Voices Rise Over Surgical Volume–Quality Connection

Photo by Steve Gilbert Shukri Khuri believes that carefully collected clinical data about patients, like the kind used in the VA system, should be the measure of quality of surgical care, not patient volume.

Does practice necessarily make perfect in surgical care? For more than two decades, evidence has accumulated that hospitals with higher volumes of surgical procedures have better results, and surgeons who perform the most operations have fewer patient deaths. The Leapfrog Group, a coalition of corporate health care purchasers, has even set volume thresholds for specific surgeries in its ratings for hospitals that provide services to its employees, and other groups are starting to do the same.

But not everyone believes that more is better, and some surgeons and researchers argue against using what they call an imperfect measure of quality. They are looking for better measures to separate the good hospitals from the bad ones and gather data that will help hospitals improve their care.

Since 1979, a series of studies has found a positive relationship between hospital volume and patient outcomes. In the largest study to date, John Birkmeyer, then at Dartmouth and now a professor of surgery at the University of Michigan, used data from 2.5 million Medicare patients who underwent one of 14 cancer or cardiovascular procedures and found that higher volume centers had lower mortality rates for all of the procedures, though the scale varied widely. Birkmeyer and other groups have even used information about volume to estimate the number of lives that could be saved by selectively referring patients to high-volume hospitals.

No one argues that both surgeons and hospitals need to perform a minimum number of procedures to be considered proficient, though that number depends on the complexity of the procedure. Yet beyond that minimum, is more always better? Shukri Khuri, HMS professor of surgery at the VA Boston Healthcare System, has been the most adamant opponent of using volume to measure quality. In the Sept. 8 issue of the World Journal of Surgery, and in an upcoming chapter in the Advances in Surgery, Khuri lays out his argument. He believes that data from a long-term quality improvement program at the VA calls into question the relationship between volume and surgical outcomes. He argues that even if a relationship does exist, it is not a predictive one and hospitals should be using better measures to assess their quality of care.

Systemwide Data
Khuri heads the National Surgical Quality Improvement Program (NSQIP) at the VA, which was launched in 1994 to gather information about the quality of care in the system’s hospitals. The program relies on a dedicated trained nurse at each clinical center who prospectively collects information on patient risk factors, care during the patient’s stay, and 30-day outcomes for more than 100,000 major operations every year. Based on this information, Khuri and his colleagues can calculate both the expected and observed morbidity and mortality rates for different patient populations. The ratio between the observed outcomes and expected outcomes (O/E) is used as a measure of overall quality for each institution. The program has been highly successful; within a decade, surgical mortality rates and complications have dropped, and the VA system is widely noted for its high quality.

“Most people feel that volume in and of itself shouldn’t be used directly as a quality measure.”

When a reorganization within the VA prompted a call to close smaller surgical centers, Khuri and his colleagues began looking at volume. Khuri said he fully expected to see better care at higher volume centers, but the data did not show a clear relationship. For example, in one year, the center that had the highest volume of colectomies had one of the worst O/E ratios for that procedure. If volume were used as a basis for referral, “we would be sending the colectomy patients to the hospital that has the worst risk outcomes of the whole system,” Khuri said.

Instead, he believes the system of care is important. Physicians and nurses conducted site visits at different VA hospitals as part of a validation study, and found that hospitals with very good O/E ratios, or low outliers, generally had better systems of care than high outliers. “We’re now confident that when we say this hospital is a high outlier, that hospital is likely to have inferior processes and structures of care,” he said.

A Referral Hitch
Khuri cannot prove that the volume–outcome relationship is unreliable outside the VA system; critics have pointed out that the VA has a unique structure and relatively homogenous patient population, and more importantly, that its hospitals have a much lower volume than a typical sample from the private sector.

John Birkmeyer does not believe that the VA study outweighs the accumulated evidence showing a volume–outcome relationship. He does concede that volume is only a proxy measure for quality and is imperfect compared with a direct measure such as mortality rates. Where this sort of information is available, Birkmeyer said, it is wise to use it. But outcomes measures are difficult to apply to infrequent procedures. For these, Birkmeyer’s bet is on the hospital with higher volume. “There’s no question that for many operations, getting patients to higher-volume hospitals or higher-volume surgeons would save not hundreds but thousands of lives,” he said. The problem, he believes, is a practical one. The health care system does not have a way to reorganize on a large scale the process by which patients are referred to hospitals.

Measuring Quality
Sharon-Lise Normand, HMS professor of health care policy (biostatistics), has studied the volume–outcome relationship in cardiac surgery and other coronary interventions. She said that while there is a relationship between volume and quality, “most people feel that volume in and of itself shouldn’t be used directly as a quality measure.” Even the Leapfrog Group, which has set volume thresholds for certain complex procedures, uses outcomes and other measures in addition to volume whenever the information is available. The trick is finding measures of quality that are practical to collect and accurate to apply and to convince hospitals to report their data.

Normand said that current research is working on defining process measures—steps that should be taken during a patient’s care—as well as outcomes measures. Many people, however, are still not convinced that these criteria could fairly compare hospitals. “Where the disagreement arises,” she said, “is really whether or not information could be collected that would sufficiently describe how sick the patients were [before the intervention].” Without adjusting for patient risk factors, doctors and hospitals with the healthiest patients will look better.

Khuri believes that the VA program and its data are superior because they strictly adjust for patient risk using clinical rather than administrative sources. Patient information is collected by a nurse at each institution using established standards and clinical definitions.

The American College of Surgeons is extending the NSQIP to private hospitals, with the goal of improving quality and collecting the kind of risk-adjusted outcomes data that can be used to better make decisions. Birkmeyer said that the cost of gathering clinical data “is several orders of magnitude higher than relying on data that we already have,” and it is unclear whether the benefit is worth the cost. Now, however, he is co-director of a pilot program to implement the NSQIP in 15 Michigan hospitals. He said that physicians place much more trust in clinical information, and that alone will make it more useful.

R. Scott Jones, director of the American College of Surgeons Division of Research and Optimal Patient Care, who is leading the organization’s effort, said that in the surgical field, “risk-adjusted outcomes will be the gold standard for measuring quality.” It will take a long time to get there, however, and in the meantime, Jones said, the college is using volume standards in several new bariatric surgery centers it is accrediting in hospitals around the country. Their rationale is that “if we set a volume standard, we’re not going to hurt any patients,” he said. Yet the centers will also carefully collect outcomes data. And as more hospitals start doing the same, which Jones hopes will happen over the next decade or so, the question of volume will gradually lose relevance.

—Courtney Humphries

top