Peer Review

So many people swear by it and use it as a standard that tells them what information to accept or dismiss. It is like people have checked their own brains and decided to let someone else do their thinking for them. or they are just too lazy to analyze a given paper and simply look to see if it was peer reviewed before accepting its content.

Here is a good comment on the peer review system:

“The benefit of published work is that if they have passed the muster of peer review future researchers can have faith in the results”, writes a commenter at The Economist. Such statements are commonplace.

I couldn’t disagree more. Nothing is more fatal to the scientific endeavour than having “faith” in a previously published result — as the string of failed replications in oncology and in social psychology is showing. See also the trivial but crucial spreadsheet error in the economics paper that underlies many austerity policies.

Studies have shown that peer-reviewers on average spend about 2-3 hours in evaluating a paper that’s been sent their way. There is simply no way for even an expert to judge in that time whether a paper is correct: the best they can do is say “this looks legitimate, the authors seem to have gone about things the right way”.

Now that is a useful thing to be able to say, for sure. Peer review is important as a stamp of serious intent. But it’s a long way from a mark of reliability, and enormous damage is done by the widespread assumption that it means more than it does.

Remember: “has passed peer review” only really means “two experts have looked at this for a couple of hours, and didn’t see anything obviously wrong in it”. (http://svpow.com/2013/05/03/peer-review-does-not-mean-we-can-trust-a-published-paper/)

To me, peer review is nothing but glorified proofreading meant to give credibility where credibility may not have been earned or the truth discerned. A peer reviewed article does not mean that the author of the article has found the truth or is correct in his findings; it simply means that the peer who reviewed it possibly liked what he read.

Evolutionary papers are peer reviewed, usually by like-minded scientists which tells you that peer review can be used as a rubber stamp and not critical analysis. It is not hard to manipulate the peer review process thus to hang one’s hat on that qualification means one is looking to be misled.

Make sure you use your own brain and study the source. An unbeliever writing an unbelieving article which is peer reviewed by an unbeliever doesn’t change the unbelieving paper into a God led work or the truth. The source is still unbelieving.

Jim West has this to say about peer review:

Peer review is only as reliable as the people doing it.  And, gosh golly, sometimes that’s just not reliable at all.  Sometimes, just admit it, peer review is as useful as teats on a male.  Its exaltation to the status of some sort of infallible instrument in the search for truth is utterly stupid.  And blind.  Willfully.

[You would think that academics would be honest people.  But the sad fact is, some will defend peer review the same way they defend ‘accreditation’: blindly and without questioning]. (http://zwingliusredivivus.wordpress.com/2013/05/04/peer-review-is-sometimes-as-useful-as-teats-on-a-male/)

I am not a fan of peer review as you could accomplish more just by having someone proofread your work.