In a class I’m teaching we’re had some good discussions of the New Yorker article “The Truth Wears Off,” which looks at how such factors as regression to the mean, selective reporting, and publication bias can produce a “decline effect” in scientific findings. The article concludes with this:
The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.
One of the researchers discussed in the article recommends more transparency in the scientific process by “the establishment of an open-source database, in which researchers are required to outline their planned investigations and document all their results.”
The Chronicle recently spoke with Sayeed Choudhury of the Data Conservatory about the greater transparency and documentation mandated by NSF’s new data management plan requirements.
