Home
An astonishing 64 per cent of drug or medical device side effects are left out of the published reports that clinicians so frequently base decisions on. This is the finding of a recent paper published in the journal PLOS Medicine by a team of UK researchers.
The paper looked at 28 studies dealing with the discrepancies present in hundreds of published trial results, versus their unpublished counterparts. Unpublished data was found in places such as pharmaceutical reports and clinical trial registries. This includes ClinicalTrials.gov in the US, one of the first of its kind set up to bring greater transparency to the industry.
The authors found that harmful side effects would have been missed between 43 per cent and 100 per cent of the time if only the published findings were consulted, and 64 per cent on average.
“There is strong evidence that much of the information on adverse events remains unpublished and that the number and range of adverse events is higher in unpublished than in published versions of the same study,” the authors wrote.
It is incredibly difficult to accurately assess the depth of the problem. But the authors believe it is, nevertheless, vast in scope.
“Most people in the healthcare sector like to demonstrate improvements in peoples’ lives - people only read about treatments that make them better,” co-author Yoon K Loke, professor of Medicine and Pharmacology at the University of East Anglia, told WIRED.
“The main purpose of a paper is that there is good news and that something works. Most people consider side effects to be bad news so they present the minimum possible. The journals want to publish something that is exciting and interesting. I wouldn’t say it is anyone’s fault in particular. People like to think they have the new cure for cancer. I blame the culture.”
The 28 studies the team looked at each approached the problem differently, so various specific issues were highlighted in each. One study, in particular, found that although there were fewer unpublished data sources than published among the trials studied, the total number of serious side effects was higher in the unpublished set. For example, instances of “suicide ideations, attempts, or injury, homicidal ideations, and psychiatric symptoms” all higher in the unpublished set. The side effects being dealt with in these broad studies are clearly not all trivial.
The authors are now calling for full and transparent reporting of trial results so that medical professionals can base their decisions on the wider picture.
This is far from an unknown problem. Loke says we are so frequently getting only “a small, incomplete picture” of what actually happened in a trial. In many instances the authors behind the 28 studies had to submit Freedom of Information requests to get a fuller picture.
John Ioannidis, professor in disease prevention at Stanford Medicine and academic editor on the PLOS Medicine study, believes most editors and journals are in fact not actually aware of the extent of the problem. “Reporting of harms has always been suboptimal, even worse than reporting of effectiveness outcomes that has also had substantial deficiencies,” he told WIRED. “Many journals are starting to take more seriously the need for making detailed protocols and raw data routinely available. This will hopefully help remedy some of this bias or at a minimum it will help probe its depth. But there will still remain a lot of unpublished data and their non-availability may keep distorting the literature.”
He adds that although there is some academic guidance provided to authors reporting on harmful side effects, “most journals don’t follow this guidance routinely”. “This leaves a lot of subjectivity on what/how to report among the typically large and difficult to organise amount of information that may be collected - either by plan or haphazardly - on side effects during a trial.
There have been higher profile instances of side effects being omitted from published papers that demonstrate just how grave the problem can be.
A 2014 Newsweek story highlighted how a "significant amount of negative data" from trials of the drug Tamiflu were withheld from the public. Around 70 deaths were attributed to the drug, many of them suicides - but this potential side effect was not known to the US Centre for Disease Control and Prevention or the doctors administering it.
Headway is being made to remedy the problem, but progress is slow. On September 14 the United Nations released a report from The High-Level Panel on Access to Medicines, which urged governments to “require that the unidentified data on all completed and discontinued clinical trials be made publicly available in an easily searchable public register”.
This was already being called for as far back as 2008, when the International Committee of Medical Journal Editors decided only to publish trials that had been publicly registered before they began.
"Registering trials means logging details of the trial in a publicly accessible online database so researchers can see that the trial has happened and its protocol," Sile Lane, director of campaigns and policy at independent charity Sense About Science, told WIRED.
"Anyone can look at the register entry to see what the trialists planned to do and then look at the published report to see if they have reported back honestly and completely. This helps us hold the trialists to account." ICMJE is also considering requiring every trial to submit and publish raw data.
GlaxoSmithKline has promised to make clinical study reports from all of its trials since 2000 public, while Bristol-Myers Squib andJohnson & Johnson have both agreed to make raw trial data open to researchers. However in general, says Lane, "big industry umbrella groups have not made this a priority and are not pushing to find remedies".
"History is only moving in one direction, and that's towards transparency...Some companies and some academics do this now. There's really no excuse for anyone running a trial not to do this."
http://www.wired.co.uk/article/drug-side-effects
The paper looked at 28 studies dealing with the discrepancies present in hundreds of published trial results, versus their unpublished counterparts. Unpublished data was found in places such as pharmaceutical reports and clinical trial registries. This includes ClinicalTrials.gov in the US, one of the first of its kind set up to bring greater transparency to the industry.
The authors found that harmful side effects would have been missed between 43 per cent and 100 per cent of the time if only the published findings were consulted, and 64 per cent on average.
“There is strong evidence that much of the information on adverse events remains unpublished and that the number and range of adverse events is higher in unpublished than in published versions of the same study,” the authors wrote.
It is incredibly difficult to accurately assess the depth of the problem. But the authors believe it is, nevertheless, vast in scope.
“Most people in the healthcare sector like to demonstrate improvements in peoples’ lives - people only read about treatments that make them better,” co-author Yoon K Loke, professor of Medicine and Pharmacology at the University of East Anglia, told WIRED.
“The main purpose of a paper is that there is good news and that something works. Most people consider side effects to be bad news so they present the minimum possible. The journals want to publish something that is exciting and interesting. I wouldn’t say it is anyone’s fault in particular. People like to think they have the new cure for cancer. I blame the culture.”
The 28 studies the team looked at each approached the problem differently, so various specific issues were highlighted in each. One study, in particular, found that although there were fewer unpublished data sources than published among the trials studied, the total number of serious side effects was higher in the unpublished set. For example, instances of “suicide ideations, attempts, or injury, homicidal ideations, and psychiatric symptoms” all higher in the unpublished set. The side effects being dealt with in these broad studies are clearly not all trivial.
The authors are now calling for full and transparent reporting of trial results so that medical professionals can base their decisions on the wider picture.
This is far from an unknown problem. Loke says we are so frequently getting only “a small, incomplete picture” of what actually happened in a trial. In many instances the authors behind the 28 studies had to submit Freedom of Information requests to get a fuller picture.
John Ioannidis, professor in disease prevention at Stanford Medicine and academic editor on the PLOS Medicine study, believes most editors and journals are in fact not actually aware of the extent of the problem. “Reporting of harms has always been suboptimal, even worse than reporting of effectiveness outcomes that has also had substantial deficiencies,” he told WIRED. “Many journals are starting to take more seriously the need for making detailed protocols and raw data routinely available. This will hopefully help remedy some of this bias or at a minimum it will help probe its depth. But there will still remain a lot of unpublished data and their non-availability may keep distorting the literature.”
He adds that although there is some academic guidance provided to authors reporting on harmful side effects, “most journals don’t follow this guidance routinely”. “This leaves a lot of subjectivity on what/how to report among the typically large and difficult to organise amount of information that may be collected - either by plan or haphazardly - on side effects during a trial.
There have been higher profile instances of side effects being omitted from published papers that demonstrate just how grave the problem can be.
A 2014 Newsweek story highlighted how a "significant amount of negative data" from trials of the drug Tamiflu were withheld from the public. Around 70 deaths were attributed to the drug, many of them suicides - but this potential side effect was not known to the US Centre for Disease Control and Prevention or the doctors administering it.
Headway is being made to remedy the problem, but progress is slow. On September 14 the United Nations released a report from The High-Level Panel on Access to Medicines, which urged governments to “require that the unidentified data on all completed and discontinued clinical trials be made publicly available in an easily searchable public register”.
This was already being called for as far back as 2008, when the International Committee of Medical Journal Editors decided only to publish trials that had been publicly registered before they began.
"Registering trials means logging details of the trial in a publicly accessible online database so researchers can see that the trial has happened and its protocol," Sile Lane, director of campaigns and policy at independent charity Sense About Science, told WIRED.
"Anyone can look at the register entry to see what the trialists planned to do and then look at the published report to see if they have reported back honestly and completely. This helps us hold the trialists to account." ICMJE is also considering requiring every trial to submit and publish raw data.
GlaxoSmithKline has promised to make clinical study reports from all of its trials since 2000 public, while Bristol-Myers Squib andJohnson & Johnson have both agreed to make raw trial data open to researchers. However in general, says Lane, "big industry umbrella groups have not made this a priority and are not pushing to find remedies".
"History is only moving in one direction, and that's towards transparency...Some companies and some academics do this now. There's really no excuse for anyone running a trial not to do this."
http://www.wired.co.uk/article/drug-side-effects
