A controversial retracted study on the toxicology of genetically modified corn and pesticide has been republished in a new journal.
In 2012, a study was published in Food and Chemical Toxicology by Séralini et al. titled “Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize.”
The authors of the study claimed that females rats fed the GM maize were 2-3 times more likely to die during the 2 year study, mostly due to mammary tumours. They also noted the males in some GM-fed groups had an increased incidence of other types of tumours.
The paper’s design, methodology and use of statistical tools attracted a great deal of criticism from the scientific community and the paper was eventually retracted by the journal.
The research group has now republished the study in the open access journal Environmental Sciences Europe.
You can read extensive commentary on the original study from New Zealand, UK, Canadian and Australian experts.
The SMC has gathered the following expert commentary. A ‘Before the Headlines’ analysis compiled by UK statistical experts is also available below.
Associate Professor Peter Dearden, Director of Genetics Otago, Laboratory for Evolution and Development, University of Otago comments:
“The republication of the Séralini study raises a number of important issues to do with the scientific process. It must be noted that the paper being published is identical to the first one, which was initially attacked on methodological bases.
“The paper is being republished because the authors feel it was unfairly retracted from Food and Chemical Toxicology. I think that the problem here is the controversial nature of the original paper.
“This was a publication that gave some interesting results, but that needed to be replicated with larger numbers of rats in the experiment and, perhaps, a more statistically robust analysis. The paper was, in my mind, inconclusive, but pointed a direction in which future research could go.
“After much public discussion the paper was withdrawn by the journal against the wishes of the authors. This is unusual. Even more unusual is the notice of retraction that states that the study was inconclusive, but there was no flaw or fraud in the original paper. Inconclusive data is no reason to retract a peer-reviewed and published paper.
“The republication of this paper, and the rebuttals presented, have not changed my opinion. I am not convinced that the original paper indicates any danger of genetically modified food. I do think, however, that this research needs to be continued.
“I am also convinced that retracting the original paper in this unusual way has not served the scientific process well. All good science is a debate, and one that should be held publically in published journals. Only through open publication, replication and exchange of scientific data can we use science effectively.
“Controversial studies should not be buried because of public argument. They should be investigated, repeated, and new data published to either disprove or support the original findings. Only then do we get a clear and robust argument.”
Prof Jack Heinemann, Molecular Biology and Genetics, University of Canterbury, comments:
“The republication of “Long-term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize” is an important demonstration of the resilience of the scientific community. The first publication of these results revealed how personal and vicious the responses are toward researchers presenting uncomfortable findings. I applaud Environmental Sciences Europe for submitting the work to yet another round of rigorous blind peer review and then bravely standing by the process and the recommendations of its reviewers, especially after witnessing the events surrounding the first publication.
“This study has arguably prevailed through the most comprehensive and independent review process to which any scientific study on GMOs has ever been subjected.
“The work provides important new knowledge that must be taken into account by the community that evaluates and reports upon the risks of genetically modified organisms, indeed upon all sources of pesticide in our food and feed chains. In time these findings must be verified by repetition or challenged by superior experimentation. In my view, nothing constructive for risk assessment or promotion of GM biotechnology has been achieved by attempting to expunge these data from the public record.
“These findings are specific to the named GM crop variety and the tested herbicide. Therefore, future work using similar (and more powerful) protocols should focus on testing of more kinds of GM crop plants, more kinds of pesticides, and especially more kinds of mixtures of chemicals that may be on either GM or non-GM foods. Other exposure pathways also need to be tested. These include inhalation and contact exposures. The latter will be especially important as a new generation of nucleic acid-based spray-on pesticides prepare to come to market.”
Prof Thomas Lumley, Department of Statistics, University of Auckland, comments:
“I do not think the republication of the Séralini paper and the responses to critics answer any of the statistical concerns I had with the original paper. The main point of the response over sample size is to argue that some standard toxicological studies also use small sample sizes, which may be true but would not be relevant.
“Although I do not find it convincing, I am pleased that the study is being republished. While I think it would have been reasonable to reject the paper initially, I was uncomfortable with a retraction that was not based on any new information or any accusation of wrongdoing, and said so at the time.
“Since the responses to critics claim that much of the opposition is a smear campaign by people funded by Monsanto and the GM crop industry, I think it is appropriate to point out that I have never received funding from Monsanto or any company involved in GM crop technology. ”
You can also read Prof Lumleys’s comments on the original study and retraction on the StatsChat Blog.
Our colleagues at the UK SMC collected the following expert commentary.
Prof Bruce Chassy, Professor Emeritus of Food Safety/Nutritional Sciences, University of Illinois, said:
“The original Seralini paper was rejected for many reasons. Perhaps the most important of these was that the design of the study and the described methods for data collection were fatally flawed in a number of ways. No amount of rewriting or excuses for faults can make the data whole again. When the data are faulty, the experiment must be repeated with proper design and methods.
“Food and Chemical Toxicology and Elsevier have acted poorly throughout this affair. It is difficult for experts to understand why Food and Chemical Toxicology published the paper since it is exceedingly challenging to find an expert peer-reviewer who cannot find numerous flaws in the paper. The journal then consumed more than a year to retract the paper.
“Among the several reasons for retraction that Food and Chemical Toxicology failed to cite was the unethical use of animals in experiments which the Committee on Publication Ethics states in a reason for retraction.
“Seralini now states that the research was not a cancer study. If that is true, then there was no reason not to euthanize animals when tumors were first detectable. There was nothing to gain or learn. This is unethical treatment of animals.”
Prof Tom Sanders, Professor of Nutrition & Dietetics, King’s College London, said:
“Republishing data that was faulty in the first place in study design and analysis does not provide redemption. Furthermore, it is now possible to publish almost anything in Open Access journals!.
“Seralini did not follow conventional methods for assessing animal toxicity and made most of the measurements at the end of life. When a very large number measurements are made statistically significant differences will occur play of chance.
“The figures of an animal with a large tumour serve no scientific purpose. There are numerous omissions of probabilities which could lead the less critical reader to infer differences that are not statistically significant.”
Prof David Spiegelhalter, Winton Professor of the Public Understanding of Risk, University of Cambridge, said:
“The article still does not appear to have had proper statistical refereeing, and the methods and reporting are obscure. The claimed effects show no dose-response, and so the conclusions rest entirely on a comparison with ten control rats of each sex. This is inadequate.”
“The study needs replicating by a truly independent laboratory using appropriate sample sizes. I agree with the authors that this whole area would benefit from greater transparency of data and improved experimental and statistical methods”
Professor Joe N. Perry, Visiting Professor of Biometry, University of Greenwich, said:
“This paper appears to be based on the same data as Séralini’s previous 2012 paper, with no real new information and only minor rephrasing and a few new references. Therefore, I doubt whether my conclusions would differ from those of the vast majority of independent members of the scientific community, who concluded in 2012 that there was insufficient evidence to justify the claims of CRIIGEN and Giles-Eric Séralini. However, I do welcome Séralini’s promise to publish his raw data and my hope is that all organisations involved in GM risk assessment will, wherever possible in the future, publish in full their raw data in the spirit of full transparency and openness.”
From the Australian Science Media Centre:
Dr Ian Musgrave, Senior Lecturer in the Faculty of Medicine, School of Medicine Sciences, within the Discipline of Pharmacology at the University of Adelaide, comments:
“A French research study that claimed that rats fed a diet which contained a Roundup-tolerant genetically modified maize died more frequently and earlier over the two year study than control groups was retracted last year after widespread criticism of its methodology and interpretation. It has now been republished. However, the major flaws in this study still remain.
1) the wrong controls were used – there should have been a non-GMO control for each level of GMO corn (i.e. there should have been an 11 per cent control for the 11 per cent GMO corn, a 22 per cent control for the 22 per cent GMO corn and 33 per cent standard corn for the 33 per cent GMO corn. As energy content, carbohydrate load and other components of the corn may affect tumour formation, this is a fundamental flaw which invalidates any conclusions.
2) there is no dose response. For a substance to be an attributable cause of cancer, being exposed to more of the substance should result in more cases of cancer this just does not happen in this study.
3) furthermore, there is no consistent response to any of the measured outcomes that would even hint at a real adverse effect. The GMO corn had no effect on the number of tumours – Roundup even decreased the number of tumours in male rats, as did the combination of roundup and GMO corn in male rats (there was no consistent effect in female rats). High levels of GMO corn and high levels of roundup both reduced spontaneous mortality and pushed back the onset of death in male rats.
“This shows that all we are seeing in these results is due to random variation in a poorly controlled experiment. It does not show that GMO corn, or roundup, even at concentrations that no human would ever be exposed to through diet, have no effect on cancer or mortality.”
Declaration of interest: No conflict of interest.
———–
Professor Andrew Bartholomaeus, Adjunct Professor at the Pharmacy School of Canberra University and the Therapeutic Research Unit of the School of Medicine at the University of Queensland, comments:
“This paper is largely a re-publication of the original article published and subsequently retracted by Food & Chemical Toxicology due to concerns around the scientific quality of the study and its interpretation, with some amendments that qualitatively address some of the criticisms of the original.
“The science of the original publication was carefully assessed by food regulatory agencies, including the European Food Safety Agency (EFSA) and Food Standards Australia NZ (FSANZ). EFSA concluded that the design, analysis and reporting is of insufficient scientific quality to be relevant in the safety assessment process. The damming criticisms of the European Society of Toxicologic Pathology (ESTP), the peak body for experts in the diagnosis and interpretation of animal pathology findings, remain most relevant. ESTP concluded that the interpretation of findings included such basic errors that they would “be considered as a disqualifying mistake at an examination for pathologists” and stated they were “shocked by the whole body photographs of animals bearing very large tumors… which should have been euthanised….much earlier…….as the authors only illustrate that Sprague Dawley rats develop mammary tumors..(which are) common background lesions” in this strain of animal.
“From a toxicological or food safety perspective the conclusions of FSANZ and international food regulatory agencies and peak scientific bodies suggest that the paper has insufficient scientific merit even to be considered controversial or provocative and it will likely be essentially irrelevant to the mainstream scientific community.
“None of the changes alter these fundamental criticisms. In short, the paper is likely to raise little more than a yawn amongst the mainstream toxicology and food regulatory communities. As an exercise in media management however the republication and associated commentary and media management such as the embargoes and limited access, reflects a masterful flair for publicity generation. Unfortunately such studies, and the associated publicity, may lead to more serious public health consequences than those purported to be found in the studies themselves, as illustrated by the vandalism of field trials of Golden rice in the Philippines, a crop being developed to alleviate the chronic disease and premature death of some of the world’s most desperate and disadvantaged children, suffering chronic vitamin A deficiency.”
Declaration of interests:I have no direct financial interest in commercial biotechnology activities, either currently or at any time in the past. Before retiring I was the Branch head for the Risk assessment Branch of FSANZ, and prior to that the chief toxicologist for the prescription medicines branch of the TGA. I currently consult, primarily to Government, on science policy and practice in regulation and perform human health risk assessments for various areas of government. I have also collaborated with ILSI (free of charge) to deliver workshops on biotechnology risk assessment for regulators around the world and to publish papers on this topic.
From the Science Media Centre of Canada:
Cami Ryan, Professional Affiliate with the Department of Bioresource Policy, Business and Economics, College of Agriculture and Bioresources, University of Saskatchewan and an independent research consultant.
“First, and most importantly, this is the same poorly designed scientific study that has been widely discredited by health and food safety agencies all over the world when it was published in 2012 (and subsequentlyretracted in 2013 ) by Food and Chemical Toxicology. Sample sizes and controls are still a problem (there are well-articulated OECD guidelines on this: and there are several holes in terms of interpretation of data (see this, lots of links to other ‘takes’ on this).”
“If Séralini’s goal here was the pursuit of good, quality science, he would have accepted the original retraction, paid mind to the broader criticisms that he received from subject-matter scientific experts and organizations and executed a new study (using an appropriate methodology) before attempting to publish again. Quality science is published in quality journals. If Séralini was really onto something here, it most certainly would have been taken up by more reputable academic journals such as Nature or Science. ”
Disclosure statement from Cami Ryan: My current work is funded through various entities including not-for-profit grower groups and organizations as well as Genome Canada’s Genome Prairie/GELS program. No conflict.
Alan McHughen, Plant Biotechnologist and Geneticist at College of Natural and Agricultural Sciences, UC Riverside, USA. He is a public sector educator, scientist and consumer advocate. He helped develop US and Canadian regulations governing the safety of genetically engineered crops and foods. He served on US National Academy of Sciences panels investigating the environmental effects of transgenic plants, a second investigating the safety of genetically engineered foods and helped review a third looking at sustainability and economic impacts of biotechnology on US agriculture.
“The number of rats used was too small to detect a meaningful difference in treatments. In this ‘new’ study, the number of rats remains the same, too small to yield meaningful results. To illustrate for those not familiar, it’s as if Seralini tossed a coin two times, and the coin came up ‘heads’ both times. With this result, Seralini is trying to convince us that he has a magic coin that only comes up ‘heads’.
“The strain of rats used (Sprague-Dawley) was inappropriate for this type of two-year long study, as these rats have a natural predisposition to form tumors, regardless of the treatment. Séralini has not and can not justify this fatal error in experimental design
“Séralini now asserts that he follows all European ethical guidelines for animal care. But he still shows rats with massive tumors, and the European ethical standards requires rats be euthanized when tumors reach 4mm diameter. Clearly the rats in the photos have tumors larger than 4mm, about the size of a small pea.
“There’s no dose response. In toxicity or carcinogenicity studies, increasing the dose of an actual toxin or carcinogen leads to greater effect. But Séralini’s data do not show such dose effects, and Séralini still does not properly explain why.
“In short, the ‘new’ paper will have the same impact as the original, retracted paper, because the original data were useless, and there is no new data. The methodology was faulty then, and, as there is no new methodology, it remains faulty now.”
Disclosure statement for Alan McHughen: I am happy to advise that I am a public sector academic scientist serving the public interest, and as such, my research program is funded entirely from public sources; I do not accept private funds. As a result, I have no research connection to either Mr Séralini (or his coauthors), or CRIIGEN, or Monsanto.
Robert Wager, Technician and faculty member in the Biology Department at Vancouver Island University. M.Sc in Biochemistry and Molecular biology (UBC 1993). His website has dozens of articles for the general public and he wrote several peer reviewed articles. He has given public talks on GMO’s and have been involved in GMO research with an emphasis on public education for 14 years.
“There are two main issues with the data I think need explanation by Séralini. First, the basic rule of toxicology is the dose makes the poison. Everything can be toxic if the dose is high enough. Therefore all proper toxicology studies show dose response curves (the higher the dose, the greater the effect). None of the data in the Séralini paper show dose response curves.
“The second point and probably more important point is the use of inappropriate strain of rats. Sprague-Dawley is a strain of rat that spontaneously generates tumors. For this reason they are extensively used in cancer research. One of the main criticisms of the original 2012 paper was the omission of the control rat data and photos. The re-release again does not show the control rats.
“It is very clear that review of the science literature show the conclusions of Séralini et al. are not supported by the vast majority of publications in this area.”
Disclosure statement for Robert Wager: I have no financial connection with any biotech company. I have never received any personal pay from any biotech company, nor does my institute receive/administer and grants from biotech companies. I have serious difference of opinion on GMO’s with Séralini et al. but have no connection to him or his institute. I am an academic who hates the impact pseudo-science is having on public policy and that is my only motivation.
Marcel Kuntz, biologist, director of research at Centre National de la Recherche Scientifique (CNRS, France) and professor at University of Grenoble-Alpes, author of two books about GMO: Les OGM, l’environnement et la santé (Ellipses, 2006) et OGM, la question politique (Presses Universitaires de Grenoble, 2014).
“The authors reach essentially the same conclusions that were already refuted and they don’t take into account the fundamental criticisms addressed to them.
“Looking specifically at the tumors: The breed of rats used is subject to spontaneous tumor development. To identify a statistically reliable increase in tumors in a group of rats requires a large number of individuals. This re-publication is still deficient on this point.
“These tumors were the most spectacular element of the media operation conducted by the authors. It should be noted that they showed photographs of three rats: a rat who used the GMO NK603, another who drank Roundup and a third absorbed both. Unlike the most basic scientific approach, no control rats (which didn’t eat GMO or drink herbicide) were shown. These control rats are still not shown in the re-publication.”
Disclosure statement for Marcel Kuntz: My only income comes from my employers mentioned above (and marginally the copyright of my books). I have no current contract with a private company, or as an individual, nor to my laboratory. My current scientific work is basic research, unrelated to the marketing of a variety of plant (GM or not). I don’t hold any patents, nor collect, nor received income as an inventor of a patent held by others. I do not identify any change in this situation in the foreseeable future.
BEFORE THE HEADLINES ANALYSIS provided by the SMC UK
Title, Date of Publication & Journal
‘Republished study: long-term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize’ by Gilles-Eric Séralini et al, published by SpringerOpen, 24 June 2014.
[Originally publication: ‘Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize’, Food and Chemical Toxicology, 19 September 2012]
Claim supported by evidence?
The conclusions of the new study are written in different language to that of the original publication. Nonetheless, the clear language regarding rats fed on a diet containing NK603 Roundup tolerant GM maize dying earlier and having more tumours than rats fed on a standard diet is not supported by statistical significance.
(Note: biochemical analysis has not been reviewed by a Before the Headlines statistician)
[Original publication: The paper does not prove the claim that rats fed on a diet containing NK603 Roundup tolerant GM maize died earlier than rats fed on a standard diet.]
Summary
Looking at the graphs of mortality for females (Fig 6), death might appear to be earlier with a GM diet than a standard diet; however this has not been proven statistically. Mortality is broadly similar for males with a GM diet and a standard diet. Similar comments apply to pathological findings.
It is evident that some treated groups have lower death rates / tumour rates than the comparable controls. This is not reported in the abstract.
There is no consistent dose-trend – if there were an effect, we would expect to see increases (e.g. in deaths) from 0 to 11 to 22 to 33. In contrast – in males, 33 (and C) have the lowest numbers of deaths.
[Same as original publication]
Strengths/Limitations
“In females, all treatment groups showed a two- to threefold increase in mortality, and deaths were earlier. This difference was also evident in three male groups fed with GM maize” * – this statement has not been subjected to standard methods of statistical analysis for survival time. The phrase “two- to threefold increase in mortality” is based on exceptionally small numbers.
The authors suggest a threshold – this rarely occurs in practice. We would expect greatest mortality/ toxicity at the highest dose in a well-designed study. With small numbers as in this study we would expect to see a general trend of mortality increasing with dose.
It seems likely that the numbers in each group are too small for standard methods of statistical analysis to find significant effects on mortality or pathological findings.
There are virtually no p-values presented. The group sizes are small. This should be interpreted with extreme caution.
It is notable that the figures do not present deaths in the control group in a similar manner (no step graph for controls). This makes it more difficult to compare the other groups with the controls.
There are many treated groups, and a number of parameters.There is obvious potential for selected reporting, selection of methods etc. In such small groups, with so many parameters this is a big issue. This issue is amplified in the abstract and further in the press release. Strong statements are issued without sufficient backing / explanation.
Deaths are compared to the control mean (for males and females). Due to the distribution of deaths (most deaths occur in old age), this is almost bound to exclude the large majority of deaths in the control groups. The 2 or 3 deaths in the control group is determined by their methods, but is inappropriately presented as a true observation.
*[In the original publication this quote was worded differently “In females, all treated groups died 2-3 times more than controls, and more rapidly. This difference was visible in 3 male groups fed GMOs.” Otherwise same as original publication.]
Before The Headlines is a service provided to the SMC by volunteer statisticians: members of the Royal Statistical Society (RSS), Statisticians in the Pharmaceutical Industry (PSI) and experienced statisticians in academia and research. A list of contributors, including affiliations, is available here.