Reporting Negative Data in Clinical Trial Results
Negative data used to often be viewed as inconsequential and, therefore, “null.”
In an environment where pharmaceutical research is expanding far into theoretical territory with genetic studies and patient-targeted therapies with narrow applications, the drive for early success continues to exert heavy pressure on the reporting of clinical trials. Increasingly, the scientific community is coming to realize the invaluable role that negative outcomes play in the understanding of medicine as a whole.
Early in clinical trial research, the choice to publish was initially left in the hands of the researchers and the publishing community who determined what was and was not newsworthy. As a result, trials with negative results were often abandoned without publication of the data collected, often on the presumption that it represented wasted time and resources. Negative data were often viewed as inconsequential and, therefore, “null.”
Health Organizations Raise Questions About Clinical Trial Data
The premise that negative results should be treated by omission gave rise to a number of ethical challenges from major sources in recent decades — including the World Health Organization (WHO), the National Institutes of Health (NIH), and the U.S. Food and Drug Administration (FDA) — to restructure clinical trial reporting to include all relevant data from all clinical trials performed.
Several objections were specifically raised to the practice of not reporting negative clinical trial results, citing that it:
- causes duplication of negative results from other investigations;
- puts new trial patients at unnecessary risk;
- distorts the evidence base; produces incomplete knowledge and creates misconceptions that get incorporated into guidelines and daily practice;
- undermines trust of patients participating in trials if the results are never published; and
- results in poor allocation of product development resources and slows drug development.
From a scientific standpoint, the main objection to nonreporting of negative results was that it contributes to a body of knowledge that was at best, incomplete, and at worst, wrong.
According to a statement by the WHO, “Researchers have a duty to make publicly available the results of their research... Negative and inconclusive as well as positive results must be published or otherwise made publicly available.”1
A cross-sectional analysis of published literature by Christopher Jones, MD, from the Cooper Medical School of Rowan University, Camden, New Jersey, and colleagues showed that the results of nearly one-third of all large-scale clinical trials with more than 500 participants registered before 2009 were never published.2 The authors reported that data collected from an estimated 299,763 participants enrolled in 171 of 585 trials (29%) conducted from November 1999 to January 2009 had not been published up to 60 months after the close of the trial.2
Additionally, results from the majority (78%, 133/171) were not reported on Clinicaltrials.gov.2 Results from nearly twice as many studies that were industry funded were not published (32%, 150/468) compared with those funded by other sources (18%, 21/117).
“We looked at very large trials completed by 2009 and found that nearly 30% remained unpublished by 2012,” Dr Christopher Jones told Rheumatology Advisor. “Since then, there has been progress on the regulatory front with respect to trial transparency and data availability. It was very encouraging to see the WHO's recent statement,1 as well as the policy released by the NIH last year,3 both of which sent clear, emphatic messages reinforcing the importance of making trial results public.”
Path Toward Balanced Reporting and Transparency
In 1997, the United States Congress passed the Food and Drug Modernization Act to establish the current NIH online registry, clinicaltrials.gov (operated by the National Library of Medicine), designed to register clinical trials of drugs, biological products, and medical devices. Authors were required to register their trials within 21 days of recruitment of the first participant, although reporting of ultimate results was not required.
In 2005, the WHO Assembly led support to this effort with the release of a resolution requiring “unambiguous identification of all interventional clinical trials,” and by the creation of the International Clinical Trials Registry Platform, stating that: “The registration of all interventional trials is a scientific, ethical and moral responsibility.”1
The subsequent Food and Drug Administration Amendments Act (FDAAA) of 2007 required basic study results to be posted to clinicaltrials.gov within 1 year of completion of the study. 4 The FDAAA requirements were written for “applicable clinical trials,” which did not specifically reflect phase 1 preclinical studies and most trials prior to application with the FDA.5,6
Poor Compliance With Regulations
These broad interpretations all conspired to create large gaps in the way requirements were viewed. A 2014 meta-analysis by Saito and Gill7 of 400 clinical trials randomly selected from the clinicaltrials.gov database found that 29% (n=118) failed to publicly report results within 4 years of completion of the trial. The mean number of days to publication among the remaining 282 studies was 647, still far in excess of the 1-year requirement.7 Saito and Gill noted that many of the studies were smaller and were industry funded. “This constitutes reporting bias and threatens the validity of the clinical research literature in the US,” they concluded.
The World Medical Association Declaration of Helisinki Ethical Principles for Medical Research Involving Human Subjects, updated in 2008,8 required that all studies involving human participants be publicly registered to a “free access” website before the first patient is enrolled. The declaration charged authors, editors, and publishers of medical research with the ethical obligations of providing comprehensive and timely reporting of all study results.8
“Unfortunately, despite these important steps, nonpublication is still an enormous problem,” Dr Jones said. “Regulations and policies aimed at addressing nonpublication have been unsuccessful in the past, likely because enforcement is challenging and there is significant variability among investigators with respect to understanding their obligations toward making data available.”
Financial pressure is a main factor, as both the Saito and Gill and Jones et al studies found large percentages of unreported trials had industry support. Research conducted by Danielle Fanelli at the University of Edinburgh revealed that competition for funding and positions often drove researchers to drop lines of investigation that did not yield significant results in favor of trials that had better chances of success and publication.9,10
“I think that until very recently neither the medical community nor the public have recognized the magnitude of the nonpublication problem and the profound impact it can have on medical decision-making. I hope that this is starting to change and that both physicians and patients will demand that investigators and study sponsors do better,” Dr Jones challenged.
- WHO Statement on Public Disclosure of Clinical Trial Results [press release]. Washington DC: World Health Organization. http://www.who.int/ictrp/results/reporting/en/. Published April 9, 2015. Accessed March 8, 2017.
- Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical trials: cross sectional analysis [published online October 29, 2013]. BMJ. doi: 10.1136/bmj.f6104
- Hudson KL, Lauer MS, Collins FS. Toward a new era of trust and transparency in clinical trials [published online October 4, 2016]. JAMA. doi: 10.1001/jama.2016.14668
- Food and Drug Administration Modernization Act (FDAMA) of 1997 [press release]. Silver Spring, Maryland: Food and Drug Administration. https://www.fda.gov/RegulatoryInformation/Legislation/SignificantAmendmentstotheFDCAct/FDAMA/default.htm. Published November 21, 1997. Accessed March 9, 2017.
- Gopal RK, Yamashita TE, Prochazka AV. Research without results: inadequate public reporting of clinical trial results. Contemp Clin Trials. 2012;33:486-491.
- Zarin DA, Tse T. Moving toward transparency of clinical trials [published online March 7, 2008]. Science. doi: 10.1126/science.1153632
- Saito H, Gill CJ. How frequently do the results from completed US clinical trials enter the public domain?—a statistical analysis of the ClinicalTrials.gov database [published online July 15, 2014]. PLoS ONE. doi: 10.1371/journal.pone.0101826
- 8. World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects [press release]. Seoul, South Korea: World Medical Association. http://www.wma.net/en/30publications/10policies/b3/17c.pdf. Published October 2008. Accessed March 9, 2017.
- Fanelli D. Do pressures to publish increase scientists' bias? An empirical support from US States Data. [published online April 21, 2010]. PLoS ONE. doi: 10.1371/journal.pone.0010271
- Fanelli D. Negative results are disappearing from most disciplines and countries [published online September 11, 2011]. Scientometrics. doi: 10.1007/s11192-011-0494-7