The Myers Group Glossary of Terms

AHRQ – The Agency for Healthcare Research and Quality is the lead Federal agency charged with improving the quality, safety, efficiency, and effectiveness of healthcare for all Americans. The research sponsored, conducted, and disseminated by the Agency for Healthcare Research and Quality (AHRQ) provides information that helps people make better decisions about health care.
(http://www.ahrq.gov/about/profile.htm) 

Attributes are the individual questions on a survey tool that relate to a specific service area or composite.

Banner Tables (a.k.a. Crosstabs) show detailed results for each question in a survey.  Crosstabulation is a combination of two (or more) frequency tables arranged such that each cell in the resulting table represents a unique combination of specific values of crosstabulated variables. Thus, crosstabulation allows us to examine frequencies of observations that belong to specific categories on more than one variable. By examining these frequencies, we can identify relations between crosstabulated variables. Only categorical (nominal) variables or variables with a relatively small number of different meaningful values should be crosstabulated. Note that in the cases where we do want to include a continuous variable in a crosstabulation (e.g., income), we can first recode it into a particular number of distinct ranges (e.g., low, medium, high).

Benchmark Section:

Benchmarks are a set of scores compiled from numerous studies in which plans can compare their score for individual survey questions and composites.  The average is taken of the percentage distributions of identical survey questions across all plans to create the benchmark.

Quality Compass 2006 (All Plans) data benchmark is a collection of CAHPS 3.0H mean summary ratings (from reporting year 2005) for those commercial adult plans (approximately 282 plan specific samples) allowing NCQA to use their data to be compiled into an aggregate, or national summary, without releasing their plan-level scores.

Quality Compass 2006 (Public-Report) data benchmark is a collection of CAHPS 3.0H mean summary ratings (from reporting year 2005) for those commercial adult plans (253 plan specific samples) choosing to report their scores publicly, in addition to submitting their scores to be compiled anonymously into a Quality Compass aggregate, or national summary.  

Quality Compass 2006 (Regional) is a regional breakout of All Plans data that is broken into the eight Census Regions: East North Central, Middle Atlantic, Mountain, New England, Pacific, South Atlantic, South Central, and West North Central.  There is also a regional breakout by Health and Human Services Regions (HHS): Atlanta, Boston, Chicago, Dallas, Denver, Kansas City, New York, Philadelphia, San Francisco, and Seattle.
This data benchmark is a collection of CAHPS 3.0H mean summary ratings (from reporting year 2005) for those commercial adult plans allowing NCQA to use their data to be compiled into an aggregate, or national summary, without releasing their plan-level scores.  Each report shows the regional data that corresponds most closely to the health plan location, or if the plan publicly submitted to NCQA, the regional scores shown are for the region the plan was assigned to by NCQA.

Health and Human Services Regions:
Chicago - Indiana, Illinois, Michigan, Minnesota, Wisconsin, Ohio
New York - New York, New Jersey, Puerto Rico, Virgin Islands
Philadelphia – Delaware, District of Columbia, Maryland, Pennsylvania, Virginia, West Virginia
Denver - Colorado, Montana, North Dakota, South Dakota, Utah, Wyoming
Boston - Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
Seattle - Alaska, Idaho, Washington, Oregon
Atlanta - Alabama, Florida, Georgia, Kentucky, Mississippi, North Carolina, South Carolina, Tennessee
Dallas - Arkansas, Louisiana, Oklahoma, New Mexico, Texas
Kansas City - Iowa, Missouri, Nebraska, Kansas
San Francisco – American Samoa, Arizona, California, Guam, Hawaii, Nevada

U.S. Census Bureau Regions:
East North Central - Ohio, Indiana, Illinois, Michigan, Wisconsin
Middle Atlantic - New Jersey, New York, Pennsylvania
Mountain - Arizona, Colorado, Idaho, Montana, New Mexico, Nevada, Utah, Wyoming
New England - Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
Pacific - Alaska, California, Hawaii, Washington, Oregon
South Atlantic - Delaware, Washington D.C., Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, West Virginia
South Central (West and East) - Alabama, Arkansas, Kentucky, Louisiana, Mississippi, Oklahoma, Tennessee, Texas
West North Central - Minnesota, Iowa, Missouri, North Dakota, South Dakota, Nebraska, Kansas

Quality Compass 2006 (Medicaid Adult - All Plans) data benchmark is a collection of CAHPS 3.0H mean summary ratings (from reporting year 2005) for those Medicaid adult plans (an estimated 84 samples) allowing NCQA to use their data to be compiled into an aggregate, or national summary, without releasing their plan-level scores.

Quality Compass 2006 (Medicaid Adult- Public Report) data benchmark is a collection of CAHPS 3.0H mean summary ratings (from reporting year 2005) for those Medicaid adult plans (74 samples) choosing to report their scores publicly, in addition to submitting their scores to be compiled anonymously into a Quality Compass aggregate, or national summary.

Quality Compass 2005 (Medicaid Child, CAHPS® Booklet) data benchmark is a collection of CAHPS 3.0H mean Summary Ratings (from reporting year 2004) for those Medicaid Child plans (36 samples) allowing NCQA to use their data to be compiled into an aggregate, or national summary, without releasing their plan-level scores.

Quality Compass 2005 (Medicaid Child CCC, CAHPS® Booklet) data benchmark is a collection of CAHPS 3.0H mean Summary Ratings (from reporting year 2004) for those Medicaid Child with Chronic Care Conditions plans (18 samples) allowing NCQA to use their data to be compiled into an aggregate, or national summary, without releasing their plan-level scores.  The Medicaid Child CCC Benchmark data is only available for composites and ratings, and does not include attributes.

The 2006 CAHPS PPO Benchmark is a collection of CAHPS 3.0H mean summary ratings for those commercial adult plans (41 non-FEHB samples) choosing to report their scores anonymously into a Quality Compass aggregate, or national summary.  Scores shown in this report reflect the mean Summary Rates from these plan means.

The 2006 National CAHPS Benchmarking Database (NCBD) Commercial Adult Benchmark Note: NCBD Summary Rate definitions are not the same as NCQA Summary Rate definitions.  The health plan’s Summary Rates are recalculated on the comparison page to match NCBD Summary Rate definitions.  The NCBD data presented includes summary-level distributions of 271 commercial adult (HMO/POS) CAHPS 3.0 Health Plan Survey results from 2006.  The majority of submissions come from the U.S. Office of Personnel Management (OPM), the Federal agency that sponsors health benefits for the civilian government work force.  Other sources of commercial submissions include state employers, state health data commissions, and individual health plans.

The 2006 National CAHPS Benchmarking Database (NCBD) Medicaid Adult Benchmark Note: NCBD Summary Rate definitions are not the same as NCQA Summary Rate definitions.  The health plan’s Summary Rates are recalculated on the comparison page to match NCBD Summary Rate definitions.  The NCBD data presented includes summary-level distributions of 119 Medicaid adult CAHPS Medicare Managed Care Survey results from 2006.

The 2006 National CAHPS Benchmarking Database (NCBD) Medicaid Child Benchmark Note: NCBD Summary Rate definitions are not the same as NCQA Summary Rate definitions.  The health plan’s Summary Rates are recalculated on the comparison page to match NCBD Summary Rate definitions.  The NCBD data presented includes summary-level distributions of 95 Medicaid child CAHPS Medicare Managed Care Survey results from 2006.

The 2006 National CAHPS Benchmarking Database (NCBD) Medicare Benchmark Note: NCBD Summary Rate definitions are not the same as NCQA Summary Rate definitions.  The health plan’s Summary Rates are recalculated on the comparison page to match NCBD Summary Rate definitions.  The NCBD data presented includes summary-level distributions of 273 Medicare adult CAHPS Medicare Managed Care Survey results from 2006.

Further Information on NCBD:
In total, the CAHPS Database currently contains 9 years of data from the CAHPS Health Plan Survey. The 2006 database holds survey results for 327,662 adults and children enrolled in commercial, Medicaid, SCHIP, and Medicare plans. 

The Myers Group received the “National CAHPS Benchmarking Database: 2006 CAHPS Health Plan Survey Chartbook” in October of 2006.  The data presented in this chartbook includes the following populations:

• Commercial Adult,
• Commercial Child,
• Medicaid Adult,
• Medicaid Child,
• State Children’s Health Insurance program (SCHIP), and
• Medicare Adult Managed Plans.

The 2005 National HCAHPS Benchmarking Database National Benchmark was created by the National CAHPS Benchmarking Database (NCBD) in conjunction with the Agency for Healthcare Research and Quality (AHRQ) to provide hospitals with comparative information on the quality of patient hospital care and services. The national benchmark consists of the mean Summary Rates of data voluntarily submitted by 254 test-site hospitals from across the nation to the National HCAHPS Benchmarking Database in 2005.

The National HCAHPS Benchmarking Database Regional Benchmark was calculated according to parameters as defined by the American Hospital Association and contains data from the 254 test-site hospitals that submitted data to NCBD in 2005. Regions and included states are as follows:

• Mid Atlantic/New England (n=7,480) – Maine, New Hampshire, Vermont, Massachusetts, Rhode Island, Connecticut, New York, New Jersey, Pennsylvania
• South Atlantic (n=20,642) – Delaware, Maryland, District of Columbia, Virginia, West Virginia, North Carolina, South Carolina, Georgia, Florida
• East Central (n=26,906) – Ohio, Indiana, Illinois, Michigan, Wisconsin, Kentucky, Tennessee, Alabama, Mississippi
• West North Central (n=6,614) – Minnesota, Iowa, Missouri, North Dakota, South Dakota, Nebraska, Kansas
• West South Central (n=6,578) – Arkansas, Louisiana, Oklahoma, Texas
• Mountain (n=6,322) – Montana, Idaho, Wyoming, Colorado, New Mexico, Arizona, Utah, Nevada
• Pacific (n=10,236) – Washington, Oregon, California, Alaska, Hawaii

CAHPS (Consumer Assessment of Healthcare Providers and Systems Survey) The Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys are a set of standardized survey tools developed to assess patient satisfaction with their health plan. Developed jointly by the Agency for Healthcare Research and Quality (AHRQ) and NCQA, the CAHPS 3.0H survey is the most comprehensive tool available for assessing consumers’ experiences with their health plans. The CAHPS 4.0 Commercial Survey has recently been approved for use in 2007, while the remainder product line type 4.0 survey versions are currently in development and testing.

Composites represent an overall aspect of plan quality and are comprised of similar questions. For each composite, an overall score is computed. The composite score is the average of the Summary Rate Scores of the questions comprising a composite. For example, the Customer Service composite in the Provider Satisfaction Survey is the average of the Summary Rate Scores of Questions 1 and 2.

Confidence Interval tells you how sure you can be that a statement is true. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. For example, if the Summary Rate Score is 75%, and the 95% confidence interval is +/- 6%, then we are 95% confident that the true Summary Rate Score is between 69% and 81% (75% + 6% = 81%; 75% - 6% = 69%).

Correlation Analysis - Correlations can be utilized to test the relationship that a particular plan attribute has with the responses to ratings of overall satisfaction (i.e. health plan or healthcare). The strength of the correlation, or relationship, is given by the correlation coefficient which values can range from –1.00 to +1.00. A correlation coefficient of 0 implies there is no relationship between the variables. As the correlation coefficient increases, so does the strength of the relationship between the two variables.

Database – a database file is a collection of confidential member-level information provided by the client for a particular TMG project. This data usually consists of a member’s name, address, phone number, plan type, age, gender, and additional variables for use in data analysis for the Final Report. A random, stratified, or other type of draw is performed on the database, and the members in the final sample are contacted for survey completion (attempts can be made by mail, phone, or internet).

Demographic Categories are the grouping of respondents by age, gender, race, etc. The Myers Group will often collapse several of the respondent characteristic variables into fewer segments than those defined by the survey. The consolidation of these categories with small samples allows for more valid between-group statistical comparisons.

Disposition (Disposition Category) is the final status assignment given to a respondent within the sample. The category signifies both the survey administration protocol used to complete the survey (M=mail, and T=phone) and the status of the record (M10= mail complete, T22= phone, language barrier). All record code assignments of “10” are considered valid responses.

Global Proportions are a graphical presentation of the percentage of members who responded to each response choice, organized by composite category and the attributes contained within each.

HEDIS® (Health Plan Employer Data and Informaton Set) is the most widely used set of standardized performance measures designed to ensure that purchasers and consumers have the information they need to reliably compare the performance of managed healthcare plans. It is part of an integrated sysem to establish accountability in managed care across the nation. The performance measures in HEDIS are related to many significant public health issues such as cancer, heart disease, smoking, asthma and diabetes. HEDIS also includes a standardized survey of consumers' experiences that evaluates plan performance in areas such as customer service, access to care and claims processing. HEDIS is sponsored, supported and maintained by NCQA. (http://www.ncqa.org/programs/hedis/index.htm)

NCQA (National Committee for Quality Assurance) is an independent, 501(c)(3) not-for-profit organization committed to assessing, reporting on, and improving the quality of care provided by organizing delivery systems. NCQA is governed by a Board of Directors that includes employers, consumer and labor representatives, health plans, quality experts, regulators, and representatives from the field of organized medicine. (www.ncqa.org)

Question Summaries (a.k.a. Frequency Distributions) are the proportion of respondents that fall into each response category for all questions. For most reports a section entitled Question Summaries is included. This section includes: Question category, Question number, Question verbiage, Valid number of responses, and Response options. Other options that may be included are Summary Rate Scores, Trend data, Benchmark scores, and Significance testing.

Range is the percentage point difference between Summary Rate Score percentages for two or more segment groups within one population sample. The larger the number, the greater the difference in Summary Rate Scores between segment groups for any given item. Summary Rate Scores with a range that are greater than or equal to 10% may be cause for further inspection. This variance could indicate a preference for a certain response option(s) within a particular segment group.

Rating Questions use an 11-point scale with “0” representing the worst rating and “10” representing the best rating.

Raw Data File (Member-Level Data File) is either an Excel or SPSS data file that includes a project’s data before any statistical analysis has been applied, and does not include any member identifying variables. A data file may include the survey ID, individual ID, disposition code (mail, phone, internet), individual question responses coded numerically, open-ended question text responses, and additional variables such as region, clinic, provider, plan type (HMO, POS, PPO), or disease type. These additional variables are provided by the client in their original database sample, thereby allowing the respondent data to be matched with these variables in the raw data file.

Regression Analysis - Opportunity Analysis can also use Regression Analysis to identify Key Drivers. Regression estimates are measures of the relationship between composite scores and Overall Satisfaction. Regression Analysis takes into consideration all of the interrelationships between attribute/composites when determining the strength of the relationship between attribute/composites and Overall Satisfaction. The numbers reported next to each significant composite are Beta coefficients. The higher the Beta coefficient, the larger the effect the composite has on overall satisfaction.

Response Rate is only calculated for those respondents who were eligible and able to respond. According to NCQA protocol, ineligible members include those who are deceased, do not meet the eligible population criteria, have a language barrier, or are either mentally or physically incapacitated. Non-respondents include those members who have refused to participate in the survey, could not be reached due to a bad address or telephone number, or members that reached a maximum attempt threshold and were unable to be contacted during the survey time period. NCQA has also considered surveys that have been returned with less than 80% of the questions answered a non-response; however, this rule will be eliminated in 2007.

Rounding of Numerical and Percentage Data
Typically, when percentages are calculated in our report applications, all decimal places are computed, but only the first decimal place is actually shown. As such, adding rounded single-digit decimals may not equal 100%. If the same figures were taken out an additional decimal place, however, they would add to exactly 100%. Through consultation with a number of our clients, The Myers Group has determined that using a single decimal place in the reporting of percentages provides an adequate level of detail. Finally, when rounding, TMG employs the standard practice of rounding down any number from 1 to 4, and rounding up any number from 5 to 9.

Sampling Error can be thought of as the extent to which survey results may differ from what would be obtained if every eligible member in the sample had been surveyed. The size of such error depends largely on the percentage distributions (i.e., the number of respondents selecting each answer category) and the number of members surveyed. The more disproportionate the percentage distributions or the larger the sample size, the smaller the error will be.

The following tables may be used in estimating approximate sampling error. The first table shows the range (plus or minus the figure shown) within which the population percentage could be expected to lay 95 out of 100 times a sample of that size and percentage distribution would be selected. The second table shows the range (plus or minus the figure shown) within which the population percentage could be expected to lay 90 out of 100 times a sample of that size and percentage distribution would be selected.

The sampling error table is used in the following manner.  Assume that “overall satisfaction with the health plan” received a Summary Rate Score of 70% from a sample of 500 valid responses.  For a 95% confidence interval, look at the first table where the sample size of 500 intersects the percentage distribution of 70/30.  The margin of error for this sample size is four percentage points (4%).  Therefore, on average, in 95 out of 100 similar samples, the 95% confidence interval (e.g., 66% to 74%) will span the true unknown population percentage.

Statistical Significance is not necessarily related to the amount one score is higher or lower than another score.   The number of respondents and the percentage distribution of response options also influence statistical significance.  One score is statistically different from another score when the difference between the scores is more than would be expected by sampling error (margin of error).  Statistical significance is the likelihood that conclusions resulting from a sample also hold true for the population from which the sample was taken.  For example, if the difference between a plan’s overall rating scores for two consecutive years is statistically significant at the .05 level, you can be 95% confident that the difference between the two scores would also be observed if all members were surveyed for both years.

Summary Rate Score represents the percentage of respondents who chose the most favorable response option(s).  For example, one question’s Summary Rate Score is computed using the following proportion:

Survey Administration Protocol describes the process in which the data was collected for the survey.

Trend Comparisons show how your health plan’s current year composite, attribute, and rating Summary Rate Scores compare to your scores from previous years.

Z-Test is a statistical inference test used to determine if the difference between two proportions (Summary Rates) is large enough to be statistically significant.  Statistical testing is done to draw conclusions about differences in proportions between a sample and a set constant (e.g., a national benchmark) or between different samples (e.g., a Summary Rate for this year versus a Summary Rate for last year). When checking for significant differences between proportions various conditions must be meet: The sample must be from a simple random sample and the population from which the sample was drawn must have a normal variance (bell-shaped curve).  If it is not known that the population has a normal variance, it suffices to have a sufficiently large sample.