The self-fulfilling prophecy of a downward trend in retraction delays

As reported on Retraction Watch today, folks interested in the field of publications and retractions are “geeking out” at a new study from authors at Thomson Reuters, who mined the ISI Web of Science database to learn some new things.

Among their key conclusions, based on Figure 1c of the paper, is a downward trend in the time delay between publication and retraction…. ret tend orig

The problem is, such a conclusion is a self-fulfilling prophecy. The study was based on a 2014 version of the database, so for example there cannot possibly be any papers published in 2012 that would have more than a 2 year delay in retraction, because it hasn’t been long enough yet!  Here’s another way to look at the data, with the red line indicating the maximum possible retraction delay for a paper published in that year….

ret trend annotated

Are we to believe that in the coming years, papers from 2005 onward which take 10 years to retract, will simply not arise?  No, of course those papers are out there, and over time they will of course raise the average delay to retraction, of their year’s cohort of papers.

TL/DR – it’s too early to conclude a downward trend.

——————–

But, what could be done differently?  Well, they could have taken the average delay for the whole data set (it looks to be about 7-8 years) and just set that as the cut-off, ignoring any data between 2006 and 2014.  In my estimation, doing so would have nullified the conclusion of a downward trend.

——————–

The other major issue that impinges on this outcome, is the authors being based at Thomson Reuters, a large publishing conglomerate.  The paper contains no conflict-of-interest statement, but I do find it rather “convenient” that a group of people paid by the publishing industry found a favorable trend in the speed at which the industry deals with problems as they arise.  In my own experience (N=1 anecdote), the delay for publishers to deal with problem papers is either static or has risen recently due to increased workload of this kind.

Spring lab update & musings

First, lots of modifications to the lab website…

  • The personnel page finally got edited to include all the changes we underwent last summer.
  • The publications page is now up to date with all our latest papers (including the Slo2.1 paper that is finally out!)
  • The research page has been edited to provide some more recent details on directions that different projects are going in, and to list our funding sources more accurately.

It’s also been a busy couple of months for travel and other lab’ happenings…

  • Paul spoke at the Gaming Metrics conference at UC Davis, outlining some more recent data from the ongoing study of the impact of internet publicity on retractions/corrections of literature problems.
  • Paul was also at the Oxygen Radicals Gordon Conference, talking about our work on Sirtuins and mitochondrial metabolism.
  • We finally got our new custom antibodies against Slo2.x channel variants, from Aves labs, and are now testing them.
  • A bunch of metabolomics samples went off to Metabolon, so now we’re waiting for a flood of data!
  • Owen is off to the Biophysical Society meeting in Los Angeles.
  • 3 other papers currently in the pipeline, so it should be a good year for publications. (It had better be – we have an R01 renewal going out the door in October!

Coming up…

  • Raul Mostoslavsky (SIRT6 guru) is visiting Rochester next week for a seminar/visit in the cancer center.
  • Steven P. Jones from the Cardiovascular Center at Univ. Louisville is visiting us on April 28th.
  • There’s a new Mitochondrial Meeting announced at NIH campus, this May (these are usually great meetings, and free to register!)
  • Paul’s final (?) stint on the MIM study section this June. Also there’s UMDF grant reviewing in May (we’re always looking for more panel members!)

Finally, there have been quite a few interesting happenings around the web/twitter of late, which deserve to be highlighted…

  • The “Jingmai O’Connor” affair. For the uninitiated, this started out as an interview in Current Biology, in which Dr. O’Connor (a rising star in paleontology) made some disparaging remarks about blogging, along the lines of “those who can’t do science, blog about it“). The ensuing social media melt-down is nicely documented on Leonid Schneider’s blog post, in which Dr. O’Connor was found to have behaved in a totally unprofessional manner, sparking considerable general mockery and a parody twitter account.
  • The Talia Jane incident, in which a millenial English Major wrote a whiny letter to her boss about shitty wages in the SF Bay Area, including not being able to buy bread. She was subsequently revealed on Instagram as a total foodie, with expensive habits such as having bourbon delivered by courier. The responses have been appropriately worded. If you haven’t read Jean Twenge’s book “Generation Me” for a deeper understanding of the millennial phenomenon, it’s a good read.
  • The ASAPBio hashtag on Twitter provided updates on the eponymously named conference, where “influence makers” tried to sell the rest of the life-sciences community on the idea that we should all ignore glamour journals (Cell/Nature/Science et al.) and publish our work instantly as pre-prints. Part of the resistance to this idea is some of its proponents have built careers to date on glam-humping, so who are they to tell the rest of us to give up chasing high impact journals and just do everything open access / pre-print from now on?  Back in the real world, we’re unlikely to find success any time soon in convincing tenure/promotion committees to ignore publication venue as a factor in assessing a candidate’s CV. That’s not to say I’m anti-open access (we have plenty of papers in OA journals), I just think there are currently a lot of barriers to non-BSD labs going full-OA without career consequences for the PI and the trainees.

 

2016 update – effect of InterNet publicity on correction of the scientific literature

Some of you may recall the events of late 2012 / early 2013, in which an anonymous blog I was running to highlight data problems in the life sciences literature, was rather unceremoniously shuttered by legal threats.

In late 2013, to try and salvage something of lasting benefit to the scientific community out of these unfortunate events, I conducted a post-hoc’ study to see what happened to the 274 papers I had blogged about – i.e., were they corrected or retracted from journals?  As a control group for the study I used a set of 223 papers I had acquired during the course of running the blog, but not yet got around to writing about.

The results, which were published in April 2014 in this PeerJ paper, showed that papers blogged about were acted upon at a significantly greater rate than those whose data problems were kept private.  Specifically, the blogged papers were retracted 6.5-fold more, and corrected 7.5-fold more. Thus, the overall conclusion was that exposure of problem data in a public forum had a large impact on whether the journals/institutions/authors actually chose to do anything about it.

So, why the update?

One of my lingering doubts about this study, was that the private papers were received across an ever-so-slightly later time window than the blogged ones. Specifically, the blogged papers were received June ’12 – December ’12, while the private papers were received November ’12 – January ’13.  Even though the data for the study were collected right up until final submission of the paper in February 2014, there was always the nagging possibility that the private papers may have been acted on less, because less time had elapsed for them. Given sufficient time, would the private papers catch up with the blogged ones?

I think it can now be argued that sufficient time has elapsed.  In addition, the dramatic rise of PubPeer, PubMedCommons, and post-publication peer review in general, means there have been multiple other opportunities for the people who sent me these papers to criticize them on-line by now. So, knowing (as I do) that several of the private papers have indeed appeared on PubPeer, one might predict they’d catch up with the blogged ones?

However, it turns out that my nagging self-doubt was ill-founded. As of today (January 2016) the blogged papers have continued to be retracted and corrected at a much greater level than the private ones!

Specifically, the blogged papers have racked up another 14 errata and 10 retractions in the intervening 2 years, bringing their totals to 61 errata and and 26 retractions (an almost 32% overall rate of action). In contrast during the same period the private papers have garnered another 2 errata and 1 retraction, for respective totals of 7 and 3, or a 4.5% overall action rate.

In other words, over 3 years since the shuttering of the blog, the blogged papers continue to be acted on at a compounded rate that is 7-fold greater than the private papers.

So, the previous advantage (?) conferred to a paper by having been blogged about, seems to have held up over the intervening period.  The private papers didn’t catch up, and while I’ll hesitate to predict they never will, the overall pattern doesn’t show any signs of changing.  Bottom line – exposure works, and perhaps even some kinds of exposure (the snarky blog variety) work better than others.

FYI, I’ll be speaking about these data at the upcoming “Gaming Metrics” conference at UC Davis, next month.

It’s Slo2.1 baby!

We’ve been struggling for more than a couple of years, to find a publication home for our finding that the potassium channel Slo2.1 (aka Slick, KCNT2) is the one in the mitochondrion required for anesthetic preconditioning (APC).  Finally it was accepted last week in Anesthesiology, and should be in press fairly soon. Here are a few key points…

1) Slo2.1 is required for APC. Knockout mice can’t be protected by volatile anesthetics. This establishes Slo2.1 as a novel drug target to protect the heart against ischemic injury.

2) This lays to rest once-and-for-all the cacophony regarding a potential role for Slo1 in APC. There were a LOT of papers linking Slo1 to APC based on problematic pharmacology and the fact that it’s a KCa channel. Slo2.1 is a KNa channel.

3) We can’t rule out that there might still be Slo1 in mitochondria. I do have some issues with the papers published in this area, but even if mito’ Slo1 does exist, it’s not important for APC, as we showed in 2011 (and people still don’t seem to get).

 

Single use website privacy in academia

It’s grad’ school admissions time again (yay!) which means I’m knee-deep in requests for letters of recommendation. Just as an example, one student has requested letters for 7 different institutions. Writing the actual letters is not a big deal (hey Nick you owe me a beer!) but the websites some Universities use for admissions are a royal P.I.T.A.

One such website (used by a mid-size college in that big metropolis near Cape Cod MA) is “LiasonCAS“.  The premise seems rather innocent and simple at first – the University uses such a site to coordinate the upload of application materials, so nothing gets lost along the way, and presumably this saves a lot of collating work that would have previously been done by a grad’ school administrator. The problem is, there are hundreds of these sites, all commercially operated, and all requiring their own User ID / Password combination.  This creates 2 problems…

Problem 1 – too many passwords

I’m a strong believer in never using the same password twice, hence my reliance on the open source KeePass application, which stores passwords (typically random strings of 20+ characters) in an encrypted file. All told, I have 200+ unique log in IDs. Just to emphasize the size of the problem here, I recently consolidated my logins for Elsevier journals into a single user ID, and it broke the portal set up to do this. It turns out I had reviewed for 37 separate Elsevier journals over the years, and some extensive ‘phone tech support was required to give me a single account to manage them all.

So the problem here is do I really want to register for yet another website, create a unique user ID, have all my contact info’ out there in the cloud?  Am I ever going to use this website again?  Probably not, which means it’ll just sit there waiting to be hacked a decade from now, resulting in a bunch of spam and junk-mail or ‘nuisance phone calls. Over the years, on top of the 200+ login UN/PW combo’s mentioned above, I would guess I’ve accumulated at least that number again in single-use website visits.  This is not good for security.  And no, it’s not as simple as me going to the trouble of making another 200+ unique UN/PW combos for all these sites. I don’t want a KeePass database filled with junk.

Problem 2 – Draconian privacy policies

Like most admissions portals, the one mentioned above requires that you agree to their privacy policy as a condition of signing up for the privilege of submitting materials.  Most people breeze through these things when signing up for a site, which is no big deal if you’re going to use it regularly and it’s a necessity for your job. Apple’s iTunes EULA is notoriously squirrely about privacy issues, but if you wanna use an iPhone, deal with it.

However, in the case of a single-use site, these policies need a bit more scrutiny. What exactly are you agreeing to, for this one-time use?  Here’s the section from LiasonCAS’ policy on what they (the site owner and the University by extension) can do with your information. There’s a bunch of guff about using your contact info’ for contests, surveys and promotions, which is worrying in itself. But then there’s this:

3.10. Other Uses. In addition to the uses specifically identified in this Section 3 (Our Uses of Your Personal Information), we may use Personal Information you submit in any other manner we reasonably deem necessary in order to provide you with the information, products and services you request from us….

Essentially what they’re saying here, is they can do what the hell they like with your data, so long as they can write it off as “necessary” for the service you request.  What you’re requesting of course, is the privilege of uploading stuff for admissions. And the price you pay is them having the freedom to shill your data out to their spammy partners under the guise of necessity.

This is not cool.

So what to do instead? In this case I found the email address of the Dean for the graduate school in question, and emailed them the letter directly. Sure, it took another 5 minutes but at least all my personal contact info isn’t sitting out there on some nondescript company’s website waiting to be sold.

New Year’s Resolution for academics – fight the requirement to create a new User ID / Password for any website that you know you’ll probably never use again.