Common Flaws In Oncology Microarray Studies Identified By Study
January 19, 2007 – 7:54 pm | posted in Blood / Hematology, Breast Cancer, Cancer / Oncology, Lung CancerA substantial percentage of microarray-based studies in oncology contain critical flaws in analysis or in their conclusions, reports a study in the January 17 issue of the Journal of the National Cancer Institute. The study’s authors provide a checklist and a set of guidelines for performing and reporting such studies.
Microarrays are a tool used to study gene expression. Researchers can study thousands of genes at a time, all on a single glass slide. In oncology, scientists have used microarrays to study unique gene expression patterns of specific tumor types, to discover new drug targets, and to categorize unique characteristics of a particular tumor to help doctors tailor treatments to an individual patient. However, such studies produce volumes of data that is easily misinterpreted. It has been difficult to replicate such studies, which is considered the best way to validate scientific findings.
To study the statistical methods used in cancer-focused microarray studies, Alain Dupuy, M.D., and Richard M. Simon, D.Sc., of the National Cancer Institute in Bethesda, Md., reviewed 90 studies published through the end of 2004 that related microarray expression profiling to clinical outcome. The most common cancers in those studies were hematologic malignancies (24 studies), lung cancer (12 studies), and breast cancer (12 studies). The studies fell into three general categories: an outcome-related gene finding, such as searching for specific genes that are expressed differently in people who have a good versus bad prognosis; a class discovery, where researchers cluster together tumors with similar gene expression profiles; and supervised prediction, in which the gene expression profiles are used to generate an algorithm or set of rules that will predict clinical outcomes for patients based on their individual gene expression profiles.
The authors closely scrutinized the statistical methods and reporting in 42 studies published in 2004. Half of these studies (21) contained at least one basic flaw. In the 23 studies with an outcome-related gene finding, nine of them had inadequate, unclear, or unstated methods to take into account false-positive findings. In 13 of the 28 studies focused on class discovery, there were spurious claims of meaningful classifications of outcomes, in which the authors did not perform adequate analyses to reach their conclusions. Among the 28 studies reporting supervised prediction, Dupuy and Simon found that 12 of those studies used biased estimates of the accuracy of their predictions.
“…Microarray studies are a fast-growing area for both basic and clinical research with an exponentially growing number of publications,” the authors write. “As demonstrated by our results, common mistakes and misunderstandings are pervasive in studies published in good-quality, peer-reviewed journals.” To avoid such errors, Dupuy and Simon provide guidelines in the form of a list of “Do’s and Don’ts” for researchers. “We believe that following these guidelines should substantially improve the quality of analysis and reporting of microarray investigations,” the authors write.
Popularity: 45% [?]
2 Responses to “Common Flaws In Oncology Microarray Studies Identified By Study”
Microarrays are the so-called “gene chips.†These can examine gene expression in up to 50,000 different genes at once. It is mainly used for screening/gene discovery work. In other words, out of 50,000 genes, only 15 or 20 may be involved in determining sensitivity/resistance to a given drug. So you sceen 50,000 genes to discover an association and then you focus in on only a few hundred or so for more careful study for sensitivity and reproducibility by some other method like real time polymerase chain reaction (RT-PCR).
The heterogeneity of cancer means that a particular drug will rarely be effective against all tumors of a particular type, and the degree of efficacy will vary between patients. Also, as a result of their original mutations, many tumor cells acquire the ability to adapt rapidly to changes in their environment, sometimes by further mutation, often by molecular changes which induce resistance to the drugs used to treat them. Further courses of chemotherapy then select for resistant cells, and the treatment eventually fails to control the tumor.
To overcome the problems of heterogeneity and prevent rapid cellular adaptation, oncologists are able to tailor chemotherapy to individual patients. This is done by testing the tumor cells to see if they are susceptible to particular drugs, before giving them to the patient. DNA microarray work will prove to be highly complementary to the parellel breakthrough efforts in targeted therapy through cell culture assays like the EGFRxâ„¢.
Many hope that molecular tests may hold the key to success, particularly as more specific drugs are designed to hit the molecular changes that are responsible for the uncontrolled growth of cancer cells. Like testing breast cancer for the presence of hormone receptors and over-expression of growth factor receptors. However, most drugs cannot be looked at in this way and tests that are now in use have limited predictive accuracy.
So how about exposing cancer cells to the drug and testing their effect? You need to expose the cancer cells to the drugs without altering their behavior from the original tumor. It is not possible to remove the non-cancer cells from the tumor without doing this. But certain assay culture methods can get rid of the non-cancer cells before the end of the culture period. These cell culture assays have contributed to the molecular understanding of chemosensitivity and resistance.
The August 5, 2004 edition of the New England Journal of Medicine contains an article relating microarray gene expression patterns to clinical drug resistance. The investigators exposed cells to drugs and cultured in a 96 hour suspension cell culture drug resistance assay (MTT) to define sensitivity and resistance. They used the data to define gene expression patterns associated with sensitivity and resistance to each of 4 drugs commonly used in the treatment of childhood leukemia. They were able to show that the gene expression definitions of sensitivity and resistance were significantly and independently associated with treatment outcome.
This work could not have been done without prior work in more than a thousand cell culture drug resistance test assays from children with leukemia to define sensitivity and resistance for each of the four drugs. Cell culture assays are the Rosetta Stone which allows for identification of clinically relevant gene expression patterns which correlate with clinical drug resistance for different drugs in specific diseases. This shows how short-sighted it has been for the academic and clinical oncology community not to support the development and clinical application of Cell Cuture Assay Tests.
In an accompanying editorial, a review of the study findings indicated that the observed gene expression profiles represent fundamental biochemical features and suggests that gene expression profiles could be used to alter therapy instead of in vitro sensitivity testing. However, they go on to state that there is no single gene whose expression accurately predicts therapy outcome, emphasizing that cancer is a complex disease and needs to be attacked on many fronts.
A number of cell culture assay labs across the country have data from tens of thousands of fresh human tumor specimens, representing virtually all types of human solid and hematologic neoplasms, in which were tested a median of 17 drugs and/or drug combinations under very similar conditions to that of this acute lymphoblastic leukemia study. Cells were exposed to drugs and cultured in suspension for 96 hours and tested simultaneously with two different assays (MTT and DiSC). What this means is that these cell culture assay labs have the Rosetta Stone database necessary to define sensitivity and resistance for virtually all of the currently available drugs in virtually all types of human solid and hematologic neoplasms.
By Gregory D. Pawelski on Feb 24, 2007
Microarrays are the so-called “gene chips.†These can examine gene expression in up to 50,000 different genes at once. It is mainly used for screening/gene discovery work. In other words, out of 50,000 genes, only 15 or 20 may be involved in determining sensitivity/resistance to a given drug. So you sceen 50,000 genes to discover an association and then you focus in on only a few hundred or so for more careful study for sensitivity and reproducibility by some other method like real time polymerase chain reaction (RT-PCR).
The heterogeneity of cancer means that a particular drug will rarely be effective against all tumors of a particular type, and the degree of efficacy will vary between patients. Also, as a result of their original mutations, many tumor cells acquire the ability to adapt rapidly to changes in their environment, sometimes by further mutation, often by molecular changes which induce resistance to the drugs used to treat them. Further courses of chemotherapy then select for resistant cells, and the treatment eventually fails to control the tumor.
To overcome the problems of heterogeneity and prevent rapid cellular adaptation, oncologists are able to tailor chemotherapy to individual patients. This is done by testing the tumor cells to see if they are susceptible to particular drugs, before giving them to the patient. DNA microarray work will prove to be highly complementary to the parellel breakthrough efforts in targeted therapy through cell culture assays like the EGFRxâ„¢.
Many hope that molecular tests may hold the key to success, particularly as more specific drugs are designed to hit the molecular changes that are responsible for the uncontrolled growth of cancer cells. Like testing breast cancer for the presence of hormone receptors and over-expression of growth factor receptors. However, most drugs cannot be looked at in this way and tests that are now in use have limited predictive accuracy.
So how about exposing cancer cells to the drug and testing their effect? You need to expose the cancer cells to the drugs without altering their behavior from the original tumor. It is not possible to remove the non-cancer cells from the tumor without doing this. But certain assay culture methods can get rid of the non-cancer cells before the end of the culture period. These cell culture assays have contributed to the molecular understanding of chemosensitivity and resistance.
The August 5, 2004 edition of the New England Journal of Medicine contains an article relating microarray gene expression patterns to clinical drug resistance. The investigators exposed cells to drugs and cultured in a 96 hour suspension cell culture drug resistance assay (MTT) to define sensitivity and resistance. They used the data to define gene expression patterns associated with sensitivity and resistance to each of 4 drugs commonly used in the treatment of childhood leukemia. They were able to show that the gene expression definitions of sensitivity and resistance were significantly and independently associated with treatment outcome.
This work could not have been done without prior work in more than a thousand cell culture drug resistance test assays from children with leukemia to define sensitivity and resistance for each of the four drugs. Cell culture assays are the Rosetta Stone which allows for identification of clinically relevant gene expression patterns which correlate with clinical drug resistance for different drugs in specific diseases. This shows how short-sighted it has been for the academic and clinical oncology community not to support the development and clinical application of Cell Cuture Assay Tests.
In an accompanying editorial, a review of the study findings indicated that the observed gene expression profiles represent fundamental biochemical features and suggests that gene expression profiles could be used to alter therapy instead of in vitro sensitivity testing. However, they go on to state that there is no single gene whose expression accurately predicts therapy outcome, emphasizing that cancer is a complex disease and needs to be attacked on many fronts.
A number of cell culture assay labs across the country have data from tens of thousands of fresh human tumor specimens, representing virtually all types of human solid and hematologic neoplasms, in which were tested a median of 17 drugs and/or drug combinations under very similar conditions to that of this acute lymphoblastic leukemia study. Cells were exposed to drugs and cultured in suspension for 96 hours and tested simultaneously with two different assays (MTT and DiSC). What this means is that these cell culture assay labs have the Rosetta Stone database necessary to define sensitivity and resistance for virtually all of the currently available drugs in virtually all types of human solid and hematologic neoplasms.
(N Engl J Med. 2004, 351:533-542; 601-3)
(N Engl J Med. 2002, 347: 1999-2009)
(Various Bio-Assay Labs)
By Gregory D. Pawelski on Feb 24, 2007