Crowdsourcing Science

"We are…moving to post-publication peer review where the scientific community judges what matters…connected globally through the internet."- Richard Smith Director of United Health Group's chronic disease initiative

Meh, so some guy blogs about the demise of peer-reviewed journals.

Only, I think he's right.

And the guy's opinion should carry some weight—he is a former editor of the British Medical Journal (BMJ), one of the preeminent peer-reviewed medical journals that has followed the peer review model since 1840.

In my opinion, the train left the station on this one a long time ago. It is a slow moving train, however. It is not in the best interests of those making money on the peer-reviewed journals industry to let go of their income source.

According to Smith, "Scientific journals began in the 17th century…Before that…scientists went to meetings and presented their studies. The assembled scientists would then discuss and critique the studies…This was the original peer review: immediate and open."

So, over time, the stewards of scientific information became a relatively small group of the scientific community, with some key publishers and their peer reviewers being the arbiters of "reliable" scientific communications. Not a bad way to do things for 170 odd years.

But the connectedness of the digital age changes everything. Everyone can be author, publisher, reviewer, promoter, as well as reader.

Even before Nature checked the data, I had faith in crowdsourcing as an imperfect but relatively sound way of maintaining an adequate level of factual integrity. The debate over whether or not to use Wikipedia as a source of information raged for years in the library realm because it isn't peer reviewed. I have always believed that for most substantive entries, there is statistically a high enough number of people who read it and a subset of them who will weigh in if the information is blatantly untrue. Experts are everywhere. Of course, there are also pretenders. I guess my feeling is that the information is likely to be accurate enough for many purposes. And I have never believed everything I read, even in peer-reviewed journals. Best to have several unique corroborating sources, maybe some data of your own, and a good dose of common sense on weighty matters.

So, will the validity of scientific information of the future be digitally crowdsourced, debated, and rated like a product on Amazon rather than peer reviewed? What impact will this have on future work based on information found in non-peer-reviewed sources? How will it impact the robustness and supportability of intellectual property? What are the implications on education of new scientists and how we teach them to evaluate data?

In my mind, I suddenly see Socrates surrounded by a mass of intellectuals and students peer reviewing

the old fashioned way—in a dynamic, real-time exchange. Isn't that exactly what we could do digitally, globally, and with more experts weighing in? Or, as Smith said, "It's back to our roots."

What Socrates could have done with a smart phone…

Technorati tags: Richard Smith, peer review, scientific publications, BMJ