Computational Biology
Microarrays Prove Reliable in Cross-platform Tests
Immunology
T Cell Misfits May Spell Autoimmunity
Pathology
Prostate Tumor Chemistry
Reveals Early Disease
Medical Education
Dienstag Named Medical Education Dean
Student Research Takes Stage at Soma Weiss Day
Minorities Appear to Use CAM Less Than Whites
Blueprint Drawn for Ebola Infection
Director Appointed for Research Compliance at HMS
Abelardo Morell Photograph Donated
Stem Cell Institute Awards 12 Seed Grants
Connors Gift Renames MGH Building After Thiers
Zelen Award and Talk Announced
Rabkin Fellows Chosen for ’05–’06
Honors and Advances
In Memoriam
Surgery Center Joins Push for Quality Improvement
Microarrays Prove Reliable in Cross-platform Tests
Variance in Lab Procedures Far Greater than in Gene Chip Performance, Studies Find
Microarray platforms can be made to perform reliably, but only if scientists all follow exacting standards for preparing samples, conducting experiments, and collecting and processing data. So say three papers published in the May Nature Methods and one in the March 29 Proceedings of the National Academy of Sciences. The studies, involving more than two dozen laboratories, assuage some recent doubts about the trustworthiness of data from experiments performed with microarrays.
Photo by Graham Ramsay
Despite the common perception that gene expression data are not reproducible across different microarrays, John Quackenbush and his colleagues, working with hypertensive mice, found comparable cardiac expression patterns measured with different platforms.
“This is really the starter’s gun that says the technology is working and it’s time to pursue more rigorous applications,” said John Quackenbush, HSPH professor of computational biology and bioinformatics at the Dana–Farber Cancer Institute and co-author of two of the studies.
Microarray technology has profoundly changed the biomedical research paradigm by putting the first stage of the experiment ahead of the hypothesis. Gene chips expose the activity of thousands of genes at once, providing a panoramic view of differences between healthy and diseased tissue and often revealing surprising molecular players. The different signature patterns can help scientists probe the molecular mechanisms of disease, explore the role of different cell types, identify potential drug targets, develop new diagnostic tools, and assess the prognosis of patients. The technology has heralded a new era of “personalized” medicine.
A
Burst of Array Studies
The number of gene array studies has exploded, from eight in 1997 to 3,000
in 2003, according to Science magazine. Many researchers attribute the
enthusiasm for microarrays to the 1999 study by Todd Golub and his colleagues
at DFCI
and the Whitehead Institute that distinguished between two types of leukemia
solely on their gene expression patterns.
| “This is really the starter’s gun that says the technology is working and it’s time to pursue more rigorous applications.” |
“As a diagnostic tool, if one lab using one platform says you have cancer, and the other lab using another platform says you’re fine, that’s a problem,” said Peter Park, HMS instructor in medicine at Children’s Hospital Boston, who consults on microarray experiments for HMS colleagues at the Harvard Medical School–Partners Center for Genetics and Genomics. Park and his colleagues are midway through a comprehensive microarray cross-platform analysis. “We advise people doing the same study to avoid using different platforms,” said Park. Even a gene chip upgrade from the same company can cause problems.
The Chips Fall Where They May
The new papers tackle these issues head on. Researchers in Quackenbush’s
lab compared gene expression results across two microarray platforms—a
commercial gene chip that assesses a single sample and a spotted cDNA array
that compares the experimental sample and the control sample together. They
took care to treat the biological samples and data exactly the same way at
each step that was not specific to the platforms. They found a 90 percent agreement
across the two platforms of the RNA expression patterns of 11,170 genes in
a comparison of heart tissue from eight mice with induced hypertension and
matched healthy controls.
“We showed conclusively that signals on arrays are dominated by the biology, not the technology,” said Quackenbush. The experiments were conducted at the Institute for Genome Research in Maryland. He moved his lab to Boston in March, where the analysis was completed.
Image courtesy of John Quackenbush Tracking gene expression. Two different microarrays (noted as affy and TIGR) agreed more than they disagreed. Both detected similar biological differences in mice exposed to a hypertension-inducing compound for either 24 hours (acute angiotension, or AA) or 14 days (chronic angiotensin, or CA). |
The second study was led by Rafael Irizarry, a statistician at Johns Hopkins Bloomberg School of Public Health in Baltimore. One rainy day, Irizarry drove identical RNA samples from four human cell lines to the ten area labs that had agreed to be guinea pigs in his experiment. He dropped a sample off with Quackenbush, a co-author on this paper.
Results from three widely used platforms showed good agreement, contradicting some previously published results. The researchers found that the labs had a larger effect than the platform—for example, the best and worst performing labs used the same platform.
“Microarrays have gotten a bum rap,” Irizarry said. The harshest critics are likely to harbor three misconceptions about microarray data, he said. First, they make the mistake of comparing absolute expression measurements, which can vary according to the different probes embedded in different platforms that purport to measure the same gene.
The more critical studies also did not account for the significant effect of different ways of preprocessing, or adjusting, the raw data before analysis. Previous comparisons of data used inferior algorithms at that stage, he said. Finally, other comparison studies inadvertently may have attributed the lab effect to platform differences.
“We hope our study serves as motivation to create standards,” Irizarry said. The third study demonstrates the impact of such standards. Seven labs and 64 scientists in the Toxicogenomics Research Consortium participated in a before-and-after experiment. The labs ran two identical samples of mouse RNA (one from the liver and one pooled from several key organs) on a total of 12 different microarray platforms.
The initial results showed poor correlation across platforms and between laboratories, but consistency on individual platforms within labs. While about half of the differences arose from the platforms, commercial platforms showed the most consistency (greater than 90 percent), compared with the microarrays manufactured at each of the labs. The individual lab effect accounted for about 6 percent of the discrepancies in results.
Consistency in results improved dramatically on a second run, where the researchers all used standardized protocols they developed for labeling and hybridization, scanning, data preprocessing, and data analysis.
“All arrays operate on the same premise—the fact that DNA pairs with itself,” said corresponding author Brenda Weis, research coordinator for the Toxicogenomics group, funded by the National Institute of Environmental Health Sciences. “We’re saying we know what contributes to variability, and we know how to correct it. In the research community, we can accelerate the advancement of knowledge about disease processes much quicker if standards are in place.”
In the PNAS paper, scientists from eight universities reached the same verdict about procedures they developed and tested for a project designed to identify a genome signature from white blood cells that can predict which severe trauma and burn victims ultimately will develop sepsis or multiple organ failure. The collaboration is headed by Ronald Tompkins, the John Francis Burke professor of surgery at HMS and Massachusetts General Hospital.
“If you have rigorous standard operating procedures, and you can verify people are following them, the differences in biological expression will be far greater than the variance introduced by the sample collection, the labs, and the platform,” said co-author Lyle Moldawer, professor of surgery at the University of Florida College of Medicine in Gainesville.
So are microarrays ready for prime time? “We might not yet be willing to believe arrays give us definitive answers,” Quackenbush said, “but they give us a pretty good place to start looking for them.”