Editorial and institutional blindness is facilitating scientific fraud


Exhibit A

Today (finally!) a J. Neurosci. paper I’ve been battling to get retracted for over 3 years was finally removed from the literature. Cause for celebration right?  Not exactly. The following is a quote from the former Editor in Chief of the journal back in 2013…

The SfN requested institutional investigations […] institutions produced detailed reports that included the original images used to construct the figures. The images clearly documented that the figure elements that appeared to be replicates in the article in J. Neurosci. were in fact from different gels. Because there was no evidence that any results had been misrepresented, the SfN has dismissed the case. We consider the matter closed.

What followed (outlined in part here) was a long and protracted battle between YHN and the journal, eventually resulting in the SfN ethics committee (which oversees ethics at the journal) re-engaging the case. This was followed by another couple of years of them ignoring my emails, during which time J. Neurosci got a new EiC who claimed not to know about the case. The paper has been cited 77 times during the 3 & 1/2 years it’s been allowed to pollute the literature.

The retraction notice reads:

It was brought to our attention that the Donmez et al., 2012 paper has numerous examples of unindicated splicing of gel lanes and of duplications and inversions of gel images. The prevalence of these occurrences is unacceptable and compels us to retract the paper. We offer our most sincere apologies to readers.

Apology not accepted.

How about releasing the contents of the MIT report given to the journal in 2012 and relied on so heavily by Maunsell?  It’s important to establish if Maunsell was making a decision based on the best information available to him (thereby suggesting the MIT investigation itself was flawed), OR did the MIT report contain information suggesting there may be something wrong, and it was Maunsell who made the wrong call?

There’s a chain of communication  (Authors > MIT > J. Neurosci. > Me) and the public has a right to know who along the chain was economical with the truth, which led to dismissal of this case in 2013?  If it was the authors (as I suspect those involved will claim), at the very least this suggests MIT needs to step up their investigative game!

If you’re an SfN member, please consider asking the leadership why it took more than 3 years for them to realize they were wrong?  Why was the extensive evidence presented to them prima facie in 2013 not enough to make the call? There’s really no excuse.


Exhibit B

This case brings to light another example of a paper from the same authors in Cell.  As outlined in detail here, in that case the journal responded as follows:

Thank you for your e-mails. In addition to having been informed of the results of the institutional investigation, we have also examined the implicated figure panels editorially. Despite some apparent superficial similarities, upon extensive examination we were unable to find any compelling evidence for manipulation or duplication in those panels and therefore are not taking any further action at this time.
Best wishes,
Sri Devi Narasimhan, PhD, Scientific Editor, Cell

The problem was, the editor who handled the case was a scientific “grand-child” of the lead author on the paper (trained with a former postdoc of theirs). Several angry emails regarding this confict-of-interest went unanswered by Cell EiC Emilie Marcus , so I had to get the Committee on Publication Ethics (COPE) involved. Eventually the paper was retracted after 2 years (having accrued over 200 citations). No action against the journal was taken by COPE (which, as I’ve said elsewhere, appears to be nothing more than a narcissistic trade association for the publishing industry).

Again, the Cell case highlights that Editors were simply content to take things at face value, rather than invest the effort to dig deeper. Only after a LOT of badgering (I would estimate these 2 papers account for 100+ emails on my part) did they come to the realization there might actually be a problem.


How does this blindness damage science?

There’s more fallout than just the citations accrued by these papers before they were retracted (and please note, the Cell paper has been cited another 50 times since it was retracted!)

The lead author Gizem Donmez, was the recipient of a prestigious Ellison Foundation** New Scholars in Aging Award for 2012. There can be little doubt that the award and the high impact publications in Cell and J. Neurosci. helped her to secure a faculty position at Tufts. That position is now gone, but what of the others who applied for it and were beaten out by Donmez? What a sad waste of young scientific talent, to be cheated out of a job by someone who didn’t play by the rules. Indeed, one of my former post-doctoral fellows applied for an Ellison Award the same year – maybe his career prospects would be different now if Donmez’ hadn’t cheated?

Then there’s the unknowable cost of scientific effort to try and replicate or build on these retracted studies. The millions of dollars in grant funds wasted on antibodies and other reagents purchased after reading these papers.

There’s also the effort expended by myself and several other bloggers and social media activists to get these papers retracted. This stuff doesn’t pay the bills!  Throw in all the wasted hours of the search committee at Tufts who hired Donmez based on false pretenses. What about the peer reviewers who were hoodwinked during review of these papers – how do they feel now?

All of this was preventable!

Editors: Sit up and pay attention when confronted with evidence. The current model (partly driven by fear of being sued) allows journals to place too much reliance on institutional investigations.  As I told Cell several months ago, the multi-billion dollar publishing industry needs to start spending some of their precious cash on forensic investigators. Bring the analysis in-house. Hire some out-of-work members of the post-docalypse. Have the courage to question reports provided by institutions.

Universities: Look beyond the indirect co$ts from the grant dollars held by the author in question. Respond to emails and allegations of misconduct. Make your RIO and other ethics folks more accessible. Hire more forensic investigators. Have the courage to question “original data” provided by authors.

All of this takes effort and money (which both Universities and publishers have a lot of), but the current system we have is just not working.  It should not take 3 years and ridiculous amounts of prodding by a 3rd party, to correct the scientific record!


** The Ellison Foundation completely refused to answer any of my emails or voice-mails about Donmez, not even to acknowledge receipt. However, some time between June and September 2015 they quietly removed Donmez from their website.




Open Letter to PLoS Biology – [UPDATE – Plagiarism too!]

Dear editors,

I’m writing to demand that you immediately retract this paper by Mario Saad and colleagues (PLoS Biol. 2011 9: e1001212. doi: 10.1371/journal.pbio.1001212.) due to overwhelming evidence of data manipulation that has been known to you for almost 3 years.

The paper was originally criticized in comments on the journal website in July 2013. In February 2014 I posted these and more problems to PubPeer and PubMedCommons, and forwarded allegations directly to the journal by email. The case was featured in an article in The Scientist in August 2014, and has been widely discussed in comment threads relating to other papers by Mario Saad on the Retraction Watch website. In March 2015, while reporting on a different Saad paper in PLoS One, Retraction Watch asked someone at PLoS Biology to comment. They responded…

The paper you’ve asked about is one that the PLOS Biology team is currently looking into. We can’t share any details and, as is standard, we are not willing to discuss this externally. As you are likely aware, journals have to follow a clear process when investigating issues on papers and at all the PLOS journals we follow the COPE guidelines. Out of respect for both the scientific process, and for all involved, these things take time.

That was over a year ago. Today, 4 more papers by Saad were retracted, thereby disavowing the findings of an internal University investigation, and rebutting a lawsuit between Saad and the American Diabetes Association.

Your continued inaction on this paper is beginning to border on gross negligence. According to ISI Web of Science the paper has been cited 96 times, with more than two-thirds of these citations coming after the initial concerns were raised in June 2013. This is bad for science, and very bad indeed for your journal’s reputation.


Paul S. Brookes, PhD.

P.S. I should note this is not the first time I’ve had problems getting your journal to take action on a problematic paper! That one only took 8 months to fix.

P.P.S. I recently had the good fortune to teach a class on scientific communication, which included a session on how to critique a paper (journal club style). I used this as an example of a spectacularly bad paper riddled with data problems. In publishing this letter, I’m now making my powerpoint slides freely available for teachers in science ethics and other classes to use as an example. I’m not sure this is what you envisaged for this paper, but I think we can agree this was a preventable outcome.

[UPDATE 3/28/2016]

It has just been bought to my attention that, in addition to all of the above figure problems, large sections of the text in this paper are plagiarized from another publication!  The paper in question is this one, and the two images below detail the extent of the copied passages based on a quick look.  There may be more to be revealed, via the use of software…

saad plos plagiat 2 - Copy

saad plos plagiat 1 - Copy

The journal has been informed, although I do not have high hopes, based on zero response whatsoever to tweeting the above letter to both the @PLoS and @PLoSBiology twitter accounts last week.


The self-fulfilling prophecy of a downward trend in retraction delays

As reported on Retraction Watch today, folks interested in the field of publications and retractions are “geeking out” at a new study from authors at Thomson Reuters, who mined the ISI Web of Science database to learn some new things.

Among their key conclusions, based on Figure 1c of the paper, is a downward trend in the time delay between publication and retraction…. ret tend orig

The problem is, such a conclusion is a self-fulfilling prophecy. The study was based on a 2014 version of the database, so for example there cannot possibly be any papers published in 2012 that would have more than a 2 year delay in retraction, because it hasn’t been long enough yet!  Here’s another way to look at the data, with the red line indicating the maximum possible retraction delay for a paper published in that year….

ret trend annotated

Are we to believe that in the coming years, papers from 2005 onward which take 10 years to retract, will simply not arise?  No, of course those papers are out there, and over time they will of course raise the average delay to retraction, of their year’s cohort of papers.

TL/DR – it’s too early to conclude a downward trend.


But, what could be done differently?  Well, they could have taken the average delay for the whole data set (it looks to be about 7-8 years) and just set that as the cut-off, ignoring any data between 2006 and 2014.  In my estimation, doing so would have nullified the conclusion of a downward trend.


The other major issue that impinges on this outcome, is the authors being based at Thomson Reuters, a large publishing conglomerate.  The paper contains no conflict-of-interest statement, but I do find it rather “convenient” that a group of people paid by the publishing industry found a favorable trend in the speed at which the industry deals with problems as they arise.  In my own experience (N=1 anecdote), the delay for publishers to deal with problem papers is either static or has risen recently due to increased workload of this kind.

Spring lab update & musings

First, lots of modifications to the lab website…

  • The personnel page finally got edited to include all the changes we underwent last summer.
  • The publications page is now up to date with all our latest papers (including the Slo2.1 paper that is finally out!)
  • The research page has been edited to provide some more recent details on directions that different projects are going in, and to list our funding sources more accurately.

It’s also been a busy couple of months for travel and other lab’ happenings…

  • Paul spoke at the Gaming Metrics conference at UC Davis, outlining some more recent data from the ongoing study of the impact of internet publicity on retractions/corrections of literature problems.
  • Paul was also at the Oxygen Radicals Gordon Conference, talking about our work on Sirtuins and mitochondrial metabolism.
  • We finally got our new custom antibodies against Slo2.x channel variants, from Aves labs, and are now testing them.
  • A bunch of metabolomics samples went off to Metabolon, so now we’re waiting for a flood of data!
  • Owen is off to the Biophysical Society meeting in Los Angeles.
  • 3 other papers currently in the pipeline, so it should be a good year for publications. (It had better be – we have an R01 renewal going out the door in October!

Coming up…

  • Raul Mostoslavsky (SIRT6 guru) is visiting Rochester next week for a seminar/visit in the cancer center.
  • Steven P. Jones from the Cardiovascular Center at Univ. Louisville is visiting us on April 28th.
  • There’s a new Mitochondrial Meeting announced at NIH campus, this May (these are usually great meetings, and free to register!)
  • Paul’s final (?) stint on the MIM study section this June. Also there’s UMDF grant reviewing in May (we’re always looking for more panel members!)

Finally, there have been quite a few interesting happenings around the web/twitter of late, which deserve to be highlighted…

  • The “Jingmai O’Connor” affair. For the uninitiated, this started out as an interview in Current Biology, in which Dr. O’Connor (a rising star in paleontology) made some disparaging remarks about blogging, along the lines of “those who can’t do science, blog about it“). The ensuing social media melt-down is nicely documented on Leonid Schneider’s blog post, in which Dr. O’Connor was found to have behaved in a totally unprofessional manner, sparking considerable general mockery and a parody twitter account.
  • The Talia Jane incident, in which a millenial English Major wrote a whiny letter to her boss about shitty wages in the SF Bay Area, including not being able to buy bread. She was subsequently revealed on Instagram as a total foodie, with expensive habits such as having bourbon delivered by courier. The responses have been appropriately worded. If you haven’t read Jean Twenge’s book “Generation Me” for a deeper understanding of the millennial phenomenon, it’s a good read.
  • The ASAPBio hashtag on Twitter provided updates on the eponymously named conference, where “influence makers” tried to sell the rest of the life-sciences community on the idea that we should all ignore glamour journals (Cell/Nature/Science et al.) and publish our work instantly as pre-prints. Part of the resistance to this idea is some of its proponents have built careers to date on glam-humping, so who are they to tell the rest of us to give up chasing high impact journals and just do everything open access / pre-print from now on?  Back in the real world, we’re unlikely to find success any time soon in convincing tenure/promotion committees to ignore publication venue as a factor in assessing a candidate’s CV. That’s not to say I’m anti-open access (we have plenty of papers in OA journals), I just think there are currently a lot of barriers to non-BSD labs going full-OA without career consequences for the PI and the trainees.


2016 update – effect of InterNet publicity on correction of the scientific literature

Some of you may recall the events of late 2012 / early 2013, in which an anonymous blog I was running to highlight data problems in the life sciences literature, was rather unceremoniously shuttered by legal threats.

In late 2013, to try and salvage something of lasting benefit to the scientific community out of these unfortunate events, I conducted a post-hoc’ study to see what happened to the 274 papers I had blogged about – i.e., were they corrected or retracted from journals?  As a control group for the study I used a set of 223 papers I had acquired during the course of running the blog, but not yet got around to writing about.

The results, which were published in April 2014 in this PeerJ paper, showed that papers blogged about were acted upon at a significantly greater rate than those whose data problems were kept private.  Specifically, the blogged papers were retracted 6.5-fold more, and corrected 7.5-fold more. Thus, the overall conclusion was that exposure of problem data in a public forum had a large impact on whether the journals/institutions/authors actually chose to do anything about it.

So, why the update?

One of my lingering doubts about this study, was that the private papers were received across an ever-so-slightly later time window than the blogged ones. Specifically, the blogged papers were received June ’12 – December ’12, while the private papers were received November ’12 – January ’13.  Even though the data for the study were collected right up until final submission of the paper in February 2014, there was always the nagging possibility that the private papers may have been acted on less, because less time had elapsed for them. Given sufficient time, would the private papers catch up with the blogged ones?

I think it can now be argued that sufficient time has elapsed.  In addition, the dramatic rise of PubPeer, PubMedCommons, and post-publication peer review in general, means there have been multiple other opportunities for the people who sent me these papers to criticize them on-line by now. So, knowing (as I do) that several of the private papers have indeed appeared on PubPeer, one might predict they’d catch up with the blogged ones?

However, it turns out that my nagging self-doubt was ill-founded. As of today (January 2016) the blogged papers have continued to be retracted and corrected at a much greater level than the private ones!

Specifically, the blogged papers have racked up another 14 errata and 10 retractions in the intervening 2 years, bringing their totals to 61 errata and and 26 retractions (an almost 32% overall rate of action). In contrast during the same period the private papers have garnered another 2 errata and 1 retraction, for respective totals of 7 and 3, or a 4.5% overall action rate.

In other words, over 3 years since the shuttering of the blog, the blogged papers continue to be acted on at a compounded rate that is 7-fold greater than the private papers.

So, the previous advantage (?) conferred to a paper by having been blogged about, seems to have held up over the intervening period.  The private papers didn’t catch up, and while I’ll hesitate to predict they never will, the overall pattern doesn’t show any signs of changing.  Bottom line – exposure works, and perhaps even some kinds of exposure (the snarky blog variety) work better than others.

FYI, I’ll be speaking about these data at the upcoming “Gaming Metrics” conference at UC Davis, next month.