|
||||||||
|
HEALTH POLICY Health Plan Survey Shown To Reveal Plan DifferencesChoosing the best health plan can be a guessing game. It is difficult to know which plan has the most satisfied members, or even which aspects of the plan make it the "best"—the friendliest staff, the most attentive doctors, the least hassling paperwork. The Consumer Assessments of Health Plans Survey (CAHPS) is becoming a standard for evaluating health plans to help consumers choose their health care based on the aspects of the plan that matter. The CAHPS survey asks people to rate their plans by answering questions about access to care, interactions with doctors and staff, and availability of appointments and services. A new paper, whose lead author is Alan Zaslavsky, HMS associate professor of statistics in the Department of Health Care Policy, begins a new phase of the project by looking at data from the surveys to figure out which factors really do matter to consumers, validating CAHPS as a way of determining differences among plans. The paper, which appears in the February Medical Care, uses data from a nationwide implementation of CAHPS by the Health Care Financing Administration on users of Medicare managed care, collecting information on more than 200 plans from nearly 90,000 Medicare recipients, the most extensive use of the survey to date.
Alan Zaslavsky and his colleagues are helping to interpret and report qualitative differences among health care plans, a project that will allow consumers to choose their plans wisely. Culmination of CAHPS Begun in 1995, CAHPS is a five-year project led by Harvard, RAND, and Research Triangle Institute, and sponsored by the Agency for Healthcare Research and Quality. The first phase of CAHPS focused on development and testing of the survey. Researchers refined the instrument using psychometrics—how people respond to survey questions and how responses vary according to individual differences. For instance, older people tend to rate their health care more favorably, and more educated people tend to give lower ratings. But the ultimate goal of CAHPS is to evaluate plans, not their members. "If you want to focus on things that are going on at the plan level, the number of data points is the number of plans, not the number of people," Zaslavsky says. The wealth of data from Medicare has given researchers enough information to start looking at how plans differ. For CAHPS to work, it must find differences among plans regardless of the individual differences of plan members. Zaslavsky adjusted the data for individual differences, essentially redistributing the ratings to approximate what would be found if each plan had an identical membership. He found that this adjustment caused the disparities among plans to be more pronounced, showing that CAHPS measures substantive differences. The CAHPS survey is designed to evaluate specific aspects of a plan's quality, rather than just measuring overall satisfaction. The number of items is too large, however, to be useful either for reporting to consumers or for analyzing in research. "You can't analyze 38 questions in relation to something else," Zaslavsky says. Using statistical techniques of factor analysis, the research group looked for correlations in responses to survey questions that would allow them to group the relevant questions into a few broad categories. The approach is part math and part judgment—looking over the results, they found four composites that accounted for the strongest correlations and made sense according to the types of questions they contained. Key Categories The analysis showed statistically what had previously been a matter of conjecture. The four composites that the researchers found are delivery, which concerns the direct provision of care and the interactions with doctors and staff; access to specialists, prescription drugs, office visits, and services; the customer component of the interaction, including payments, paperwork, and customer service; and provision of advice about health. The groupings demonstrate that CAHPS does distinguish among factors that vary from plan to plan. For instance, there was a strong correlation between ratings of interactions with doctors and interactions with office staff, although on an individual basis there is no reason for the similarity. "The relationships were generally stronger at the plan level than at the individual level," Zaslavsky says, suggesting that something about the plan itself is affecting communication. Summarizing the results according to these factors would help consumers make sense of the survey data and facilitate their use in research. Other ongoing research projects are analyzing how regional differences account for variations in quality and determining plan characteristics that are predictive of quality performance. —Courtney Humphries |
