Research improprieties suspected in Moradabad
Jul 29, 2005
London, UK - A slew of correspondence, editorials, commentaries and—most ominously—two "expression of concern" notices in the July 30, 2005 issues of BMJ and Lancet over dubious research originating in Moradabad, India highlights the qualms, frustrations, and constraints of the medical community when it comes to investigating one of its own.
BMJ's issue details the chronology of its editors' suspicions stemming from a 1992 article [ 1 ] by Dr Ram B Singh (Medical Hospital and Research Centre at Moradabad, India) and reinforced over the years as Singh's group (unsuccessfully) submitted additional papers to the journal. The Lancet material focuses on the publication of a study also by Singh's group in 2002 [ 2 ]. Four Lancet reviewers had vetted the paper, but as Lancet editor Dr Richard Horton describes in his editorial this week, complaints started trickling into his office as soon as the paper was in print, with one of the first coming from Dr Richard Smith, then editor of BMJ.
"When Richard picked up the phone and called me after we published this paper in 2002, [the controversy] was completely new to me, but I gather that a lot of people had been talking about it," Horton told renalwire . "What is very odd here is that nobody was willing to go on the record. This is where I think it is the responsibility of not just the journals and institutions but the whole scientific community. When the paper was published in 1992 in BMJ, why didn't more people come forward, instead of having whispered conversations in the halls of meetings? Why did they not put down their scientific concerns about the work on paper? And they didn't."
BMJ editor Dr Fiona Godlee acknowledged that rumors were swirling about Singh's studies, but only in certain circles. "There was a lot of word of mouth about Singh's group and in particular among meta-analysts who have done systematic reviews in this area, and some of them, for a long time, have just not included Singh's work in their analyses," Godlee told renalwire . "So people within the field have largely known about the problems and discounted his work as a result; you could say, well, that's the informal route and maybe that's okay, but it doesn't resolve the problem."
When the paper was published in 1992 inBMJ, why didn't more people come forward, instead of having whispered conversations in the halls of meetings?
Both the 1992 and 2002 papers reported favorable outcomes with the Indo-Mediterranean diet. Indeed, the unintended victim of the journals' actions this week may unfairly be the broader scientific appreciation of the Mediterranean-like diet and its benefits. Unfairly, because as Dr Jim Mann points out in his Lancet Comment [ 3 ], "regardless of the final judgment of the [2002 Singh et al paper], there is little doubt about the potential benefit of the dietary measures it tested."
But as Horton observed to renalwire , the body of research is inescapably tainted. "This is really sad. . . . Here's something that is very important from a public-health point of view. Then you have a study that comes along that purportedly proves it, and then you have a situation like this that raises questions, and although the questions are raised only over the one study, inevitably people then say, well, maybe we should just go back a step before we evaluate the evidence on the efficacy of this kind of diet. That would be a mistake, but in a sense, it's almost inevitable."
An outpouring of polite concern
Publication of an "expression of concern" is a strategy recommended by the International Committee of Medical Journal Editors when an editor concerned about research fraud has not been able to satisfactorily resolve his or her suspicions but has reasonable grounds for concern about "aspects of the conduct or integrity of the work." This week's simultaneous publication of the two expressions of concern is a first for both the Lancet and BMJ [ 4 , 5 ].
As Horton points out in the Lancet, the key unanswered questions about Singh et al's research include unresolved concerns over the studies themselves, as well as the seemingly "covert" discussions about these concerns. In the case of BMJ, efforts to satisfactorily settle the issue have lasted more than a decade, permitting Singh et al to continue to publish related research in other international journals.
Freelance journalist Caroline White describes the Singh saga in a comprehensive summary in this week's BMJ [ 6 ]. She notes that Singh et al's papers appeared to include a wide range of incongruities, including overly impressive outcomes, sloppy statistics, and highly heterogeneous results. In a report commissioned by BMJ to review the 1992 paper, and quoted by White, Dr Paul McKeigue (University College Dublin, Ireland) wrote, "I think that the level of inconsistencies and errors in these papers is sufficient to justify retraction by the BMJ and other journals of the papers from this source . . . even if there is no direct evidence of misconduct." Since the authors were unable to provide raw data supporting their research, including the original consent forms and study protocol—reputedly eaten by termites—there would be "reasonable grounds" for presuming that the material had been forged, McKeigue added.
McKeigue's synopsis echoes that of other contributors to both journals this week. In the Lancet, Dr CR Somban meticulously details all of his concerns with Singh et al's 2002 paper [ 7 ]. Likewise, Dr Sanaa Al-Marzouki and colleagues report in BMJ the results of a comparison between the statistical methods used in unpublished diet research by Singh et al and those of a similarly designed trial of a drug, both tested as a means of reducing CVD [ 8 ]. Al-Marzouki et al conclude, "Several statistical features of the data from the dietary trial are so strongly suggestive of data fabrication that no other explanation is likely."
Ultimately, however, without more definitive proof—which could have formed the basis for a formal retraction—both journals settled for an expression of concern. Like a retraction, an expression of concern is linked to the offending article.
As Godlee and managing editor Jane Smith note in their BMJ editorial, "An expression of concern does not resolve the suspicions, but it alerts readers, and in particular systematic reviewers, to doubts about the studies. And it may in turn prompt an organization with the capacity and standing to the take the action necessary to do the necessary investigations" [ 9 ].
A prickly predicament
Perhaps even more difficult to address is the question of how grumbled unease over Singh et al's conduct could persist for so long without resolution. At the core of the problem is the fact that, sensitive to the critical role of an unblemished reputation within medical research, very few of Singh's peers around the globe would have wanted to stick their necks out. Horton acknowledged to renalwire : "It was frustrating to go through a process of really quite detailed peer review and revision and then to find that there were these [preexisting] concerns."
But as Godlee and Smith observe in their editorial, "In practice, there's a limit to what journals can do—because they have neither the resources nor the authority to conduct investigations to resolve suspicions about data."
There is no international authority responsible for investigating allegations of research improprieties, despite the fact that international trials led by investigators in multiple countries are increasingly common. With no formal outlet for their distress about Singh, "I think many people raised their concerns with the journals but then felt they had nowhere else to go," Godlee told renalwire . "And that's where an international body of some sort would be helpful."
Retractions, she notes, can be made with confidence when an author admits to fraud or when an author's institution conducts an investigation and finds the evidence it needs. Singh, for his part, has maintained his innocence; conveniently, he is also the head of his own institution. But institutional power has its limits. In his 10-year quest to resolve the concerns over Singh, Smith sought assistance from authorities based in India, to no avail. In a separate instance, also described by Smith in this week's BMJ [ 10 ], for a paper by Dr RK Chandra, then at the Memorial University, St John's, NL, the institution was willing to help investigate BMJ's concerns about Chandra but then was unable to act once the researcher resigned.
Left to act alone, the BMJ and Lancet conducted their own investigations of Singh's research. The BMJ commissioned experts and statisticians to review the research being submitted by Singh's group, whereas the Lancet editors opted to send the corresponding author on the 2002 Singh paper, Dr Elliott Berry (Hebrew University of Jerusalem, Israel), to perform a site visit of Singh's research facility. Berry's conclusions, also published in the Lancet this week, were: "While the data collection suffered from some irregularities, I have no reason to consider that it was not performed according to the protocol reported in the Lancet" [ 11 ].
In their editorial, Godlee and Smith note that one solution in unresolved cases like Singh's might be for journal editors to keep confidential "blacklists" of suspected researchers and their submitted papers. The problem with this strategy, however, is the sheer number of journals and the implication that someone is guilty until proven innocent. Another proposed solution, they write, is that researchers be required to put their data in a "secure archive" where it can be audited if necessary. That too, however, requires an infrastructure that currently does not exist.
If you get a one- or two-page tatty piece of study protocol, well, you know that something is a bit odd.
In the meantime, says Godlee, journals "could perhaps be making better use of this expression-of-concern mechanism." An expression of concern that is later resolved can subsequently be retracted, she notes, as in the case of a recent expression of concern "resolved" by the New England Journal of Medicine [ 12 ].
But both Horton and Godlee believe sloppy or forged research is less likely to slip under editorial radar than it was in the past, even a past as recent as 2002. Statistical peer review has improved, and in general, the bar is set high for both research and reviewers. Both the Lancet and the BMJ now have as a requirement of submission that researchers include study protocols when they are reporting results from randomized trials.
"If you get a 100-page protocol with details of every single step in the conduct, design, analytic plan, and so on, then even if you don't know the authors individually, you get the sense that this is a serious group," Horton explained to renalwire . "If you get a one- or two-page tatty piece of study protocol, well, you know that something is a bit odd."
Regardless, says Horton, the Singh ordeal should serve as an important reminder to journal editors: "I don't think journals can avoid thinking about the conditions in which a particular piece of research was actually conducted anymore. That's certainly our biggest lesson."
Sources
Jul 29, 2005
London, UK - A slew of correspondence, editorials, commentaries and—most ominously—two "expression of concern" notices in the July 30, 2005 issues of BMJ and Lancet over dubious research originating in Moradabad, India highlights the qualms, frustrations, and constraints of the medical community when it comes to investigating one of its own.
BMJ's issue details the chronology of its editors' suspicions stemming from a 1992 article [ 1 ] by Dr Ram B Singh (Medical Hospital and Research Centre at Moradabad, India) and reinforced over the years as Singh's group (unsuccessfully) submitted additional papers to the journal. The Lancet material focuses on the publication of a study also by Singh's group in 2002 [ 2 ]. Four Lancet reviewers had vetted the paper, but as Lancet editor Dr Richard Horton describes in his editorial this week, complaints started trickling into his office as soon as the paper was in print, with one of the first coming from Dr Richard Smith, then editor of BMJ.
"When Richard picked up the phone and called me after we published this paper in 2002, [the controversy] was completely new to me, but I gather that a lot of people had been talking about it," Horton told renalwire . "What is very odd here is that nobody was willing to go on the record. This is where I think it is the responsibility of not just the journals and institutions but the whole scientific community. When the paper was published in 1992 in BMJ, why didn't more people come forward, instead of having whispered conversations in the halls of meetings? Why did they not put down their scientific concerns about the work on paper? And they didn't."
BMJ editor Dr Fiona Godlee acknowledged that rumors were swirling about Singh's studies, but only in certain circles. "There was a lot of word of mouth about Singh's group and in particular among meta-analysts who have done systematic reviews in this area, and some of them, for a long time, have just not included Singh's work in their analyses," Godlee told renalwire . "So people within the field have largely known about the problems and discounted his work as a result; you could say, well, that's the informal route and maybe that's okay, but it doesn't resolve the problem."
When the paper was published in 1992 inBMJ, why didn't more people come forward, instead of having whispered conversations in the halls of meetings?
Both the 1992 and 2002 papers reported favorable outcomes with the Indo-Mediterranean diet. Indeed, the unintended victim of the journals' actions this week may unfairly be the broader scientific appreciation of the Mediterranean-like diet and its benefits. Unfairly, because as Dr Jim Mann points out in his Lancet Comment [ 3 ], "regardless of the final judgment of the [2002 Singh et al paper], there is little doubt about the potential benefit of the dietary measures it tested."
But as Horton observed to renalwire , the body of research is inescapably tainted. "This is really sad. . . . Here's something that is very important from a public-health point of view. Then you have a study that comes along that purportedly proves it, and then you have a situation like this that raises questions, and although the questions are raised only over the one study, inevitably people then say, well, maybe we should just go back a step before we evaluate the evidence on the efficacy of this kind of diet. That would be a mistake, but in a sense, it's almost inevitable."
An outpouring of polite concern
Publication of an "expression of concern" is a strategy recommended by the International Committee of Medical Journal Editors when an editor concerned about research fraud has not been able to satisfactorily resolve his or her suspicions but has reasonable grounds for concern about "aspects of the conduct or integrity of the work." This week's simultaneous publication of the two expressions of concern is a first for both the Lancet and BMJ [ 4 , 5 ].
As Horton points out in the Lancet, the key unanswered questions about Singh et al's research include unresolved concerns over the studies themselves, as well as the seemingly "covert" discussions about these concerns. In the case of BMJ, efforts to satisfactorily settle the issue have lasted more than a decade, permitting Singh et al to continue to publish related research in other international journals.
Freelance journalist Caroline White describes the Singh saga in a comprehensive summary in this week's BMJ [ 6 ]. She notes that Singh et al's papers appeared to include a wide range of incongruities, including overly impressive outcomes, sloppy statistics, and highly heterogeneous results. In a report commissioned by BMJ to review the 1992 paper, and quoted by White, Dr Paul McKeigue (University College Dublin, Ireland) wrote, "I think that the level of inconsistencies and errors in these papers is sufficient to justify retraction by the BMJ and other journals of the papers from this source . . . even if there is no direct evidence of misconduct." Since the authors were unable to provide raw data supporting their research, including the original consent forms and study protocol—reputedly eaten by termites—there would be "reasonable grounds" for presuming that the material had been forged, McKeigue added.
McKeigue's synopsis echoes that of other contributors to both journals this week. In the Lancet, Dr CR Somban meticulously details all of his concerns with Singh et al's 2002 paper [ 7 ]. Likewise, Dr Sanaa Al-Marzouki and colleagues report in BMJ the results of a comparison between the statistical methods used in unpublished diet research by Singh et al and those of a similarly designed trial of a drug, both tested as a means of reducing CVD [ 8 ]. Al-Marzouki et al conclude, "Several statistical features of the data from the dietary trial are so strongly suggestive of data fabrication that no other explanation is likely."
Ultimately, however, without more definitive proof—which could have formed the basis for a formal retraction—both journals settled for an expression of concern. Like a retraction, an expression of concern is linked to the offending article.
As Godlee and managing editor Jane Smith note in their BMJ editorial, "An expression of concern does not resolve the suspicions, but it alerts readers, and in particular systematic reviewers, to doubts about the studies. And it may in turn prompt an organization with the capacity and standing to the take the action necessary to do the necessary investigations" [ 9 ].
A prickly predicament
Perhaps even more difficult to address is the question of how grumbled unease over Singh et al's conduct could persist for so long without resolution. At the core of the problem is the fact that, sensitive to the critical role of an unblemished reputation within medical research, very few of Singh's peers around the globe would have wanted to stick their necks out. Horton acknowledged to renalwire : "It was frustrating to go through a process of really quite detailed peer review and revision and then to find that there were these [preexisting] concerns."
But as Godlee and Smith observe in their editorial, "In practice, there's a limit to what journals can do—because they have neither the resources nor the authority to conduct investigations to resolve suspicions about data."
There is no international authority responsible for investigating allegations of research improprieties, despite the fact that international trials led by investigators in multiple countries are increasingly common. With no formal outlet for their distress about Singh, "I think many people raised their concerns with the journals but then felt they had nowhere else to go," Godlee told renalwire . "And that's where an international body of some sort would be helpful."
Retractions, she notes, can be made with confidence when an author admits to fraud or when an author's institution conducts an investigation and finds the evidence it needs. Singh, for his part, has maintained his innocence; conveniently, he is also the head of his own institution. But institutional power has its limits. In his 10-year quest to resolve the concerns over Singh, Smith sought assistance from authorities based in India, to no avail. In a separate instance, also described by Smith in this week's BMJ [ 10 ], for a paper by Dr RK Chandra, then at the Memorial University, St John's, NL, the institution was willing to help investigate BMJ's concerns about Chandra but then was unable to act once the researcher resigned.
Left to act alone, the BMJ and Lancet conducted their own investigations of Singh's research. The BMJ commissioned experts and statisticians to review the research being submitted by Singh's group, whereas the Lancet editors opted to send the corresponding author on the 2002 Singh paper, Dr Elliott Berry (Hebrew University of Jerusalem, Israel), to perform a site visit of Singh's research facility. Berry's conclusions, also published in the Lancet this week, were: "While the data collection suffered from some irregularities, I have no reason to consider that it was not performed according to the protocol reported in the Lancet" [ 11 ].
|
In their editorial, Godlee and Smith note that one solution in unresolved cases like Singh's might be for journal editors to keep confidential "blacklists" of suspected researchers and their submitted papers. The problem with this strategy, however, is the sheer number of journals and the implication that someone is guilty until proven innocent. Another proposed solution, they write, is that researchers be required to put their data in a "secure archive" where it can be audited if necessary. That too, however, requires an infrastructure that currently does not exist.
If you get a one- or two-page tatty piece of study protocol, well, you know that something is a bit odd.
In the meantime, says Godlee, journals "could perhaps be making better use of this expression-of-concern mechanism." An expression of concern that is later resolved can subsequently be retracted, she notes, as in the case of a recent expression of concern "resolved" by the New England Journal of Medicine [ 12 ].
But both Horton and Godlee believe sloppy or forged research is less likely to slip under editorial radar than it was in the past, even a past as recent as 2002. Statistical peer review has improved, and in general, the bar is set high for both research and reviewers. Both the Lancet and the BMJ now have as a requirement of submission that researchers include study protocols when they are reporting results from randomized trials.
"If you get a 100-page protocol with details of every single step in the conduct, design, analytic plan, and so on, then even if you don't know the authors individually, you get the sense that this is a serious group," Horton explained to renalwire . "If you get a one- or two-page tatty piece of study protocol, well, you know that something is a bit odd."
Regardless, says Horton, the Singh ordeal should serve as an important reminder to journal editors: "I don't think journals can avoid thinking about the conditions in which a particular piece of research was actually conducted anymore. That's certainly our biggest lesson."
Sources
Singh RB, Rastogi SS, Laxmi VR, et al. Randomised controlled trial of cardioprotective diet in patients with recent acute myocardial infarction: Results of one year follow up. BMJ 1992; 304:1015-1019.
Singh RB, Dubnov G, Niaz MA, et al. Effect of an Indo-Mediterranean diet on progression of coronary artery disease in high-risk patients. (Indo-Mediterranean Heart Study). A randomised single-blind trial. Lancet 2002; 360:1455-1461.
Mann J. The Indo-Mediterranean diet revisited. Lancet 2005; 366:353-354.
Horton R. Expression of concern: Indo-Mediterranean Diet Heart Study. Lancet 2005; 366: 354-356.
Expression of concern: Randomised controlled trial of cardioprotective diet in patients with recent acute myocardial infarction: results of one year follow up. BMJ 2005; 331:266.
White C. Suspected research fraud: difficulties of getting at the truth. BMJ 2005; 331:281-288.
Soman CR. Indo-Mediterranean diet and progression of coronary artery disease. Lancet 2005; 366:366-367.
Al-Marzouki S, Evans S, Marshall T, Roberts I. Are these data real? Statistical methods for the detection of data fabrication in clinical trials. BMJ 2005; 331: 267-270.
Smith J, Godlee F. Investigating allegations of scientific misconduct. BMJ 2005; 331:245-246.
Smith R. Investigating the previous studies of a fraudulent author. BMJ 2005; 331:288-291.
Berry EM. Author's reply: Indo-Mediterranean diet and progression of coronary artery disease. Lancet 2005; 366:366-367.
Drazen JM, Ingelfinger JR, Curfman GD. Removal of expression of concern: Schiffl H, et al. Daily hemodialysis and the outcome of acute renal failure. N Engl J Med 2002;346:305-10. N Engl J Med 2003; 349:1965.
SHARE
RELATED POSTS on "Health & Medical"
