The article by John Bohannon that I have chosen for this week can be thought of as a follow up to or even an initiation to the book of the month for May. What is even better here is that it is a first person account of how badly designed scientific experiments get legitimacy and popularity if they only translate into headlines that people want to believe. It is a sting operation of sorts into the functioning of a large section of scientific research and publications and into how science reporting works in popular media, even the more creditable ones.

The Bild story quotes the study’s lead author, Johannes Bohannon, Ph.D., research director of the Institute of Diet and Health: “The best part is you can buy chocolate everywhere.”

I am Johannes Bohannon, Ph.D. Well, actually my name is John, and I’m a journalist. I do have a Ph.D., but it’s in the molecular biology of bacteria, not humans. The Institute of Diet and Health? That’s nothing more than a website.

Other than those fibs, the study was 100 percent authentic.

So, if the study was authentic, how was the public “fooled”?

Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.

.
.
.

With our 18 measurements, we had a 60% chance of getting some“significant” result with p < 0.05. (The measurements weren’t independent, so it could be even higher.) The game was stacked in our favor.

It isn’t a good idea to be on cutting edge in the fields where you lack the time or competence to critically examine every new fad. So most of us would do well to avoid reading diet advice in media (or even science journals).

The key to success in popular media is rather disappointing.

The key is to exploit journalists’ incredible laziness. If you lay out the information just right, you can shape the story that emerges in the media almost like you were writing those stories yourself. In fact, that’s literally what you’re doing, since many reporters just copied and pasted  text.

Then there is more bad news. There isn’t much hope of getting conclusive dietary advice in the near future.

But science will figure it out, right? Now that we’re calling obesity an epidemic, funding will flow to the best scientists and all of this noise will die down, leaving us with clear answers to the causes and treatments.

Or maybe not. Even the well-funded, serious research into weight-loss science is confusing and inconclusive, laments Peter Attia, a surgeon who co-founded a nonprofit called the Nutrition Science Initiative. For example, the Women’s Health Initiative—one of the largest of its kind—yielded few clear insights about diet and health. “The results were just confusing,” says Attia. “They spent $1 billion and couldn’t even prove that a low-fat diet is better or worse.” Attia’s nonprofit is trying to raise $190 million to answer these fundamental questions. But it’s hard to focus attention on the science of obesity, he says. “There’s just so much noise.”

So, there! Just stick to the basics when it comes to lifestyle choices. If it sounds exotic, when it comes to dietary advice, it probably is wrong.

Read the complete article on io9.

—

Like our recommendations? Get them in your inbox. Once a month.