Science-- there's something for everyone

Monday, September 3, 2012

Fact-checking for science

Anyone who follows the latest ‘breakthroughs’ in health knows that these studies are constantly contradicting themselves. Is coffee good for you or bad? What about wine? And what’s the superfood de jour? This isn’t necessarily a bad thing, rather it’s the very nature of scientific inquiry. Someone does an experiment and gets a result, someone else repeats the experiment and gets a different result, and over time a consensus forms about what’s really going on.

The problem is that researchers like to be the ones doing daring, novel experiments, and not the ones repeating other people’s daring experiments. It’s not just a matter of ego, but also of attracting grants and other sources of funding. This means that many studies simply don’t get replicated. And unfortunately, unconfirmed studies have a bad habit of being wrong.

What can be done? Elizabeth Iorns of the University of Miami has one possible solution. Her company (Science Exchange) provides a new service called the Reproducibility Initiative. Researchers submit their protocol to the Initiative and get matched with a company that has the necessary technology to repeat that experiment. If they get the same result, the original authors will be able to state that their data has been independently corroborated.

It’s too early to tell how effective this project will be. One possible drawback is that the original research team will have to pay companies to reproduce their results. And of course, some experiments are so complex that no commercial enterprises are available to repeat them. You can’t just send samples out to your local supercollider.

Even if this specific initiative doesn’t pan out, it’s still a great idea. I hope it does lead to more attempts to confirm results.

More on this from Carl Zimmer at Slate.