Ivan Oransky
Appearances
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So I got to be honest, and, you know, people in Durham may not like me saying this, but I think Duke has a lot of work to do to demonstrate that their investigations are complete and that they are doing the right thing by research dollars and for patients.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
There may be something that sounds a lot less nefarious than what I just described, but that is actually what's happening.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
People are now really fixated on what are known as paper mills. So if you think about the economics of this, right— It is worthwhile if you are a researcher to actually pay. In other words, it's an investment in your own future to pay to publish a certain paper. What I'm talking about is literally buying a paper or buying authorship on a paper.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So to give you a little bit of a sense of how this might work, You're a researcher who is about to publish a paper. So, you know, doesn't it at all have got, you know, some interesting finding that you've actually written up and a journal has accepted. It's gone through peer review. And you're like, great. And that's good for you.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
You know, you wear red, you must be angry, or if it says that this is definitely a cure for cancer.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
But you actually also want to make a little extra money on the side. So you take that paper that essentially it's not a paper. It's really still a manuscript. You put it up on a brokerage site or you put the title up on a brokerage site. You say, I've got a paper where, you know, there are four authors right now.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
It's going into this journal, which is a top tier or mid tier, whatever it is, et cetera. It's in this field. If you would like to be an author, you, The bidding starts now or it's just, it's $500 or 500 euros or wherever it is. And then Ivan Oransky comes along and says, I need a paper. I need to get tenure. I need to get promoted. Oh, great. Let me click on this, you know, brokerage site.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Let me give you $500. And now all of a sudden you write to the journal. By the way, I have a new author. His name is Ivan Oransky. He's at New York University. He just joined late, but he's been so invaluable to the process.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Often they don't know about it, or at least they claim they don't know about it.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
They do. I mean, there are a couple different versions of fraudulent with a lowercase f publications. There are publications that are legit in the sense that they can point to doing all the things that you're supposed to do as a journal. You have paper submitted. You do something that looks sort of like peer review. You assign it what's known as a digital object identifier.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
You do all that publishing stuff. And they're not out and out fraudulent in the sense of they don't exist and people are just making it up or they're trying to, you know, use a name that isn't really theirs. But they're fraudulent with lowercase f in the sense that they're not doing most of those things.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Then there are actual what we refer to, we and Anna Abelkina, who works with us on this, as hijacked journals. We have more than 200 on this list now. They were at one point legitimate journals. So it's a real title. that some university or funding agency or et cetera will actually recognize. But what happened was some version of the publisher sort of forgot to renew their domain.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
I mean, literally something like that. Now, there are more nefarious versions of it, but it's that sort of thing where, you know, these really bad players are inserting themselves and taking advantage of the vulnerabilities in the system, of which there are many,
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
to really print money, because then they can get people to pay them to publish in those journals, and they even are getting indexed in the places that matter.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Well, I think what it tends to correlate with is how direct or intense the publisher parish culture is in that particular area. And generally, that varies more by country or region than anything else. If you look at, for example, the growth in China of number of papers published, what's calculated is the impact of those papers which relies on things like how often they're cited.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
You can trace that growth very directly from government mandates. For example, if you publish in certain journals that are known as high impact factor, you actually got a cash bonus that was a sort of multiple of the number of the impact factor. And that can make a big difference in your life.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So Hindawi's business model was they're an open access publisher, which usually means you charge authors to publish in your journal and you charge them, you know, it could be anywhere from hundreds of dollars to even thousands of dollars per paper. And they're publishing, you know, tens of thousands and sometimes even more papers per year. So you can start to do that math.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
What happened at Hindawi was that somehow paper mills realized that they were vulnerable. So they started targeting them. They've actually started paying some of these editors to accept papers from their paper mill. And long story short, they now have had to retract something like, we're still figuring out the exact numbers when the dust settles, but in the thousands.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
It's a volume play. And when you're owned by shareholders who want growth all the time, that is the best way to grow. And these are businesses with, you know, very impressive and enviable profit margins of, you know, sometimes up to 40%. And these are not on small numbers. The profit itself is in the billions often.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
If you add up the number of papers published every year, and then you multiply that times the two or three peer reviewers who are typically supposed to review those papers, and sometimes they go through multiple rounds, it's easily in the tens of millions of peer reviews as a unit.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
And if each of those takes anywhere from four hours to eight hours of your life as an expert, which you don't really have because you've got to be teaching, you've got to be doing your own research, You come up with a number that cannot possibly be met by qualified people. Really, it can't. I mean, the math just doesn't work. And none of them are paid.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
You are sort of expected to do this because somebody will peer review your paper at some other point, which sort of makes sense until you really pick it apart. Now, peer reviewers—so even the best of them, and by best, I mean people who really sit and take the time and probe what's going on in the paper and look at all the data— But you can't always look at the data.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
In fact, most of the time, you can't look at the raw data, even if you had time, because the authors don't make it available. So, peer review, it's become really peer review light, and maybe not even that, at the vast majority of journals. So, it's no longer surprising that so much gets through the system that shouldn't.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
We have been conditioned. And in fact, because of our own attention economy, we end up covering studies overall else when it comes to science and medicine. I like to think that's changing a little bit. I hope it is. But we cover individual studies and we cover the studies that sound the most interesting or that have the biggest effect size and things like that.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
You know, you wear red, you must be angry, or if it says that this is definitely a cure for cancer.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
I essentially spend all of my nights and weekends thinking about scientific fraud, scientific misconduct, scientific integrity for that matter.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
We hear from whistleblowers all the time, people we call sleuths, who are actually out there finding these problems. And often that's pre-retraction or they'll explain to us why retraction happened. We also do things like file public records requests.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
I went to Adam and I said, what if we created a blog about this? It seems like there are all these stories that are hiding in plain sight that essentially we and other journalists are leaving on the table. And when we looked at the actual retraction notices, the information was somewhere between misleading and opaque. What do you mean by that?
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So when a paper is retracted, and it's probably worth defining that, a retraction is a signal to the scientific community or really to any readers of a particular paper. peer-reviewed journal article, that you should not rely on that anymore, that there's something about it that means you should, you know, you can not pretend it doesn't exist, but you shouldn't base any other work on it.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Right. So when you retract a paper, you're supposed to put a retraction notice on it, the same way when you correct an article in the newspaper, you're supposed to put a correction notice on it. But when you actually read these retraction notices, and to be fair, this has changed a fair amount in the 13 years that we've been doing this, sometimes they include no information at all.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Sometimes they include information that is woefully incomplete. Sometimes it's some version of you know, getting Al Capone on tax evasion. You know, they fake the data, but we're going to say they forgot to fill out this form, which, you know, is still a reason to retract, but isn't the whole story.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So we think that... Probably 2% of papers should be retracted for something that would be considered either out-and-out fraud or maybe just severe bad mistake. According to our data, which we have the most retraction data of any database, about 0.1% of the world's literature is retracted. So one in a thousand papers. We think it should be about 20 times that, about 2%.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
There's a bunch of reasons, but they come down to one. There was a survey back in 2009, which has been repeated and done differently and come up with roughly the same number, actually even higher numbers recently, that says 2% of researchers, if you ask them anonymously, they will say, yes, I've committed something that would be considered misconduct.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Of course, when you ask them how many people they know who have committed misconduct, it goes much, much higher than that. And so that's one line of evidence, which is, you know, admittedly indirect.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
The other is that when you talk to the sleuths, the people doing the real work of figuring out what's wrong with literature and letting people know about it, they keep lists of papers they've flagged for publishers and for authors and journals. And routinely, most of them are not retracted. Again, we came to 2%. Is it exactly 2%? And is that even the right number?
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
No, we're pretty sure that's the lower bound. Others say it should be even higher.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Bolt, an anesthesiology researcher, was studying something called head of starch, which was essentially a blood substitute. Not exactly blood, but something that when you were on like a heart-lung pump, a machine, during certain surgeries or you're in the ICU or something like that, it would basically cut down on the amount of blood transfusions people would need. And that's got obvious benefits.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Now, he did a lot of the important work in that area, and his work was cited in all the guidelines. It turned out that he was faking data.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
He at one point was under criminal indictment or at least criminal investigation. That didn't go anywhere. The hospital, the clinic also, which to be fair, had actually identified a lot of problems. They came under pretty severe scrutiny. But in terms of actual sanctions, pretty minimal.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So universities, they protect fraudsters in a couple different ways. One is that they are very slow to act, they're very slow to investigate, and they keep all of those investigations hidden.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
The other is that because lawyers run universities, like they frankly run everything else, they tell people who are involved in investigations, if someone calls for a reference letter, let's say someone leaves college, And they haven't been quite found guilty, but as a plea bargain sort of thing, they will leave and then they'll stop the investigation.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Then when someone calls for a reference, and we actually have the receipts on this because we filed public records requests for emails between different parties, we learned that they would be routinely told not to say anything about the misconduct.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
The most likely career path for anyone who has committed misconduct is a long and fruitful career.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
So, and I would just, not so much as a correction, but just to say that, yes, Mark Tessier-Levine was defenestrated as president. He remains, at least at the time of this discussion, a tenured professor at Stanford, which is a pretty nice position to be in.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
I've been quoted saying that the most likely career path or the most likely outcome for anyone who has committed misconduct is a long and fruitful career. And I mean that because it's true. Because most people, if they're caught at all, they skate. The number of cases we write about, which, you know, grows every year, but is still a tiny fraction of what's really going on.
Freakonomics Radio
Can Academic Fraud Be Stopped? (Update)
Dan Ariely, we interviewed Dan years ago about some questions in his research. Duke is actually, I would argue, a little bit of a singular case. really bad, significant misconduct.