Trickster Journalist Explains Why He Duped The Media On Chocolate Study
On Thursday we told you about an elaborate hoax carried out by a science journalist who wanted to teach the media a lesson about being more responsible in reporting on nutrition science.
As we reported, John Bohannon conducted a real — but deeply and deliberately flawed — study on how chocolate affects weight loss. He wrote press releases to alert the media, then sat back and watched who bit. Many news organizations around the world took the bait.
On Friday's All Things Considered, Bohannon talks with NPR's Robert Siegel about how and why he carried out this scheme, which he revealed this week in a post on i09.
"My goal was to show that scientists who do a bad job and get their work published can end up making headlines because it's us — journalists like you and me — who are failing," Bohannon tells Siegel. Because the media often fail to do due diligence, "the world is awash with junk science," he says.
Now while lots of news outlets — including Shape magazine in the U.S. — picked up the study, many other well-respected organizations, including The New York Times, the Associated Press and major broadcast networks, did not. (For the record, NPR did not report on it.) So shouldn't that be heartening?
"I wish it were that easy," Bohannon says. As he notes, the tabloids and other news outlets that ran with the bad science probably got millions of eyeballs. And this kind of junk nutrition information gets promulgated every day, he says.
"The entire nutrition beat is one of the most corrupt," Bohannon says. "The science is completely disrespected, even though what you eat affects your health, and is every bit as important as cancer and astrophysics."
9(MDE1MTIxMDg0MDE0MDQ3NTY3MzkzMzY1NA001))
Transcript
ROBERT SIEGEL, HOST:
Our next conversation is about how easy it is to consume junk - junk science. A couple of months ago, media organizations on several continents trumpeted findings of a new study from Germany. People on a low-carb diet lose weight more quickly if they eat a bar of chocolate every day.
(SOUNDBITE OF ARCHIVED RECORDINGS)
UNIDENTIFIED WOMAN #1: Calling all chocolate lovers - German chocolate may be delicious, but new research about the health benefits of chocolate by German researchers may be even more delightful.
UNIDENTIFIED MAN #1: (Foreign language spoken).
UNIDENTIFIED WOMAN #2: I come bearing good news for the chocolate lovers out there. Some great research has come out to show...
SIEGEL: The key part of this study was revealed yesterday. It was a hoax pulled off by science journalist John Bohannon. He explained how and why he did it on the website io9, and John Bohannon joins us now from the campus of Harvard University. Welcome to the program.
JOHN BOHANNON: Hello.
SIEGEL: And I noticed that John Bohannon is not a name actually taking credit for this particular study.
BOHANNON: No, that's right. I posed as Johannes Bohannon and my colleague, Peter Onneken, a journalist in Germany, is the one who actually conceived this whole study.
SIEGEL: You set up a real study, knowing that it was worthless science. I'd like you, just very briefly, to describe that study, what you did.
BOHANNON: Well, we hired people on Facebook to show up and go on a diet for three weeks. And we randomly assigned those people - about a dozen showed up - to three different regimes. One was a low-carb. Another was low-carb plus a bar chocolate every day, and then we had a control group who were instructed to just carry on business as usual.
SIEGEL: And then you measured them in what ways? What were you doing?
BOHANNON: Well, we measured a ton of things about them. First of all, they weighed themselves every day for three weeks, but we also took blood tests, measured all the standard battery of things you can measure from a drop of blood, and we had them fill out surveys about their sleep quality and well-being and physical complaints. And so we had a load of data by the end.
SIEGEL: And, in fact, that's the point here. If you have a small number of people and a large number of variables that you're testing for, you write, you're more or less guaranteeing that something will turn out to look like it's scientifically significant.
BOHANNON: That's the rub, is that if you don't say ahead of time what you're looking for exactly, then you can just pick up wherever random result you do get say, aha, this is the results. It's what we call the reproducibility crisis. If you can't reproduce your work and if you haven't designed your study to be reproducible, then how much credit can you give it in the end anyway?
SIEGEL: So you were guaranteed that you would have a, quote, "scientifically significance result." That was really junk science. Why? What was the point of doing all this?
BOHANNON: Well, my goal was to show that scientists who do a bad job and get their work published can end up making headlines because it's us, journalists like you and me, who are failing. So if we're not doing due diligence, for example, at least tapping one outside source when you're going to write about a new study, then the world is awash with junk science.
SIEGEL: One can look at the media play you got in the popular press two ways. Either, on the one hand, wow, this got picked up by the German newspaper Bild and the London Paper, the Daily Star. And it was on Cosmopolitan's German website and the German and Indian websites of Huffington Post. Or you can say, look, it didn't get on the Associated Press. It wasn't in the New York Times or a major national network. It wasn't on a - one of the quality British papers, as they say. You know, two cheers for journalism in that case.
BOHANNON: (Laughter) I wish it were so easy, but let's scale this in terms of the impact on people's decisions. If you've got 5 million readers of all of these tabloids being force-fed some diet claim versus the erudite, tiny fraction who read about it in an AP story and has the privilege of being discerning, which is the worse?
SIEGEL: So lots of eyeballs, perhaps, saw that story in Bild or the Daily Star.
BOHANNON: Just as they do every day. The entire diet-nutrition beat is one of the most corrupt. I don't know why, but the science is completely disrespected, even though what you eat affects your health is every bit as important and complex as cancer and astrophysics. And yet we treat this subject so differently.
SIEGEL: Well, John Bohannon, thanks for talking with us.
BOHANNON: And thank you.
SIEGEL: John Bohannon is a journalist and associate scientist at Harvard University. He revealed this week that he purposely conducted a bad study showing that chocolate can help with weight loss. That study went viral and was picked up by some news organizations this spring. Transcript provided by NPR, Copyright NPR.
300x250 Ad
300x250 Ad