PUBLIC HEALTH

New Look at Classic Study Shows Overemphasis on Individual-level Data

A lot of what we know about what makes us sick and what keeps us well comes from following large groups of individuals over time. These studies reveal telling associations between disease and individual characteristics, such as diet, smoking, exercise and genes.

Photo by Graham Ramsay

“People make places and places make people; such a multilevel perspective needs to be more routinely incorporated into epidemiologic analysis,” said S.V. Subramanian, who revisited a classic paper to show the value of such analysis.

But where people live also matters—and sometimes may matter more than who they are, say HSPH researchers who recently revisited the profoundly influential statistics paper at the root of the big epidemiological datasets that populate 21st century science.

The historic paper was first published in the June 1950 American Sociology Review by W.S. “Bill” Robinson of the University of California, Los Angeles. Robinson’s article marks the intellectual watershed that broadly diverted academic attention away from investigations of large-scale influences in favor of individual data for the last 60 years.

“Individual behaviors don’t happen in isolation but in context,” said first author S. V. Subramanian, HSPH associate professor of society, human development, and health. “Individuals are part of groups, and groups are made of individuals. The classic biomedical and epidemiological model is very individualistic. This could, at best, lead to inadequate descriptions of the relationships of interest and, at worst, be misleading.”

In an extensive new study, Subramanian and his co-authors reanalyzed Robinson’s data in a broader context to show that individual correlations are not sufficient to explain individual outcomes. Their cautionary tale about the error of omitting important levels of data was published online Jan. 28 in the International Journal of Epidemiology.

“We’re saying you have to think about the individual and the context—both, not either–or,” said co-author Nancy Krieger, HSPH professor of society, human development, and health. “Sometimes the context doesn’t matter, and it’s just about individuals. Sometimes the context is the only thing that matters. It is an empirical question that needs to be systematically investigated.”

The findings may bolster efforts to study cultural, social, economic and environmental influences, such as why the same person might grow up obese in Mississippi but lean in Colorado, might have less risk of a heart attack living in a city that bans transfats in restaurants, or might have a greater chance of asthma living in a poor neighborhood.

In fact, these are the kinds of studies the HSPH researchers conduct. Last year, for example, Subramanian found that a neighborhood with more widows and widowers lessens the “widowhood effect”—the increased individual risk of death among the recently bereaved.

An Overinterpretation
Robinson’s original paper made a simple point: group level data cannot be used to infer individual characteristics. Based on 1930 census data, states with a larger fraction of blacks had higher illiteracy rates, Robinson found in a strong group-level correlation. But that did not mean you could conclude blacks were illiterate. In fact, the individual-level data showed a weak correlation between being black and being illiterate.

Since then, ecological analysis routinely has been charged with the methodological crime of “ecological fallacy.” Many scientists rightly harbor deep skepticism about the validity of group-level data to draw inferences about individuals, Krieger said.

The HSPH team has no quarrel with Robinson’s calculations. “Flawless,” said Subramanian. The problem lies in Robinson’s conclusion and how researchers subsequently interpreted it.

“It is sometimes difficult to separate what Robinson actually said from a caricature of what he said,” said Glenn Firebaugh, distinguished professor of sociology and demography at Penn State University, who wrote a commentary on the HSPH study. “Robinson’s contribution was to show the danger of inferring individual-level relationships from aggregate-level associations, a common practice at the time he was writing. People overinterpreted [Robinson] to mean the only sensible analysis is to use individual-level data.”

In contrast, the HSPH re-analysis shows the additional insights of group-level data. The researchers found strong effects of Jim Crow education laws on individual literacy. States with legalized segregation had higher illiteracy rates among both whites and blacks, but the effect was stronger on blacks.

The bottom line, Subramanian said, is that single-level analysis, whether at the group or individual level, is inadequate. “What is needed is a multilevel perspective that incorporates both.”

Firebaugh illustrates the importance of both the Robinson and HSPH papers. In the 1968 U.S. presidential election, George Wallace, a well-known segregationist at the time, received a higher share of votes in regions with higher percentages of blacks. Contrary to those figures, blacks were less likely to vote for Wallace, according to postelection surveys of voters. “It must have been the case that nonblacks were more likely to vote for Wallace when they lived in black districts,” Firebaugh wrote. In other words, racial context mattered at both the individual level (for black votes) and at the regional level (for white votes).

Patriotic Methodology
The paper’s lasting influence on the scientific enterprise can also be attributed to its social and political context, Krieger said. At the time, individualism was as all-American as hot dogs, baseball and apple pie. The widespread scholarly embrace of individual actions as the primary framework of population research came during the Cold War conflict between capitalism and communism.

Courtesy U.S. Library of Congress

A new study ties lower literacy rates among blacks in Southern states to the Jim Crow laws that permitted racial discrimination. White literacy rates also suffered in states with Jim Crow laws, but not as badly. Illiteracy appears to have been influenced by an individual’s race, state Jim Crow laws, and an interaction between the two, report HSPH researchers in a paper that supports group-level analysis in the face of ongoing skepticism about this approach.

“During what has come to be known as the McCarthy era, U.S. academics who seriously or publicly questioned the individualistic assumptions associated with free-market ideology found themselves variously marginalized, denied funding, or fired from their jobs, including in economics, sociology, and medicine and public health,” she and her co-authors write.

That may explain why Robinson and other scientists did not raise the obvious context of Southern states’ Jim Crow laws, which were also in the news thanks to Harry Truman’s challenge of legalized segregation in the first-ever presidential address on civil rights to Congress in 1948.

Robinson’s data comes from page 1219 of the 1930 census, but ignores the motherlode of data two pages later about the enormous state variation in black versus white rates of illiteracy, which the HSPH team used for its analysis.

The take-home lesson goes beyond the necessity of multilevel analysis, Krieger said. The battle to eliminate legalized racial discrimination was raging on the front pages of newspapers when the Robinson study was published without consideration of the effect of Jim Crow laws on illiteracy rates. “The ideas were available at the time, but no one squawked about it,” she said. “The conduct of rigorous science requires careful attention to how experience shapes not only the data but our thinking and thus the very stuff of our science.”

—Carol Cruzan Morton

Students may contact S.V. Subramanian at svsubram@hsph.harvard.edu for more information.

Conflict Disclosure: The authors report no conflicts of interest.

Funding Sources: National Institutes of Health

top