I recently discovered an article published back in 2006 in the Journal of the Royal Society of Medicine regarding issues with the peer-review process. Since I highlighted some of these in my recent article on the evolutionary suggestion of the octopus being sent here from outer space, I decided to dissect this article and attempt to explain what peer review is, why it is used, and what issues are associated with it.
Peer review, when referring to work in the scientific fields at least, is essentially just a double-check mechanism. At least that is how it is presented to the general public. However, often peer review serves a different purpose. If a paper appears in a major scientific journal, it significantly increases the researcher’s chances of getting additional and larger government grants, necessary to continue researching. One of the stated goals of these scientific magazines is to help ensure the best projects get funded. Another is the aforementioned double-check system, to ensure what is published is accurate.
Journals use peer review because it acts as something of a protection for them. If they publish a paper that turns out to be errant, as they did with the recently retracted RNA world paper, they can point to the peer review process as something of a shield. Further, it is something of a necessary evil. The journals must check what they publish but to do so on their own would be incredibly expensive and time-consuming. The peer-review process thrusts the responsibility for an accurate scientific study into the hands of other scientists. On the surface, this sounds like a great idea. However, there are a lot of issues with the peer-review process.
Perhaps the primary issue with peer review stems from a lack of clarity about the process. As the article rightly points out, no one is entirely sure what peer review means. Who defines whether someone is a peer or not? What constitutes a review? Are the reviewers subjective or objective? If two people are racing to a discovery, might not one give the other a bad review to buy himself time to publish first? These are all questions the article considers and they are all valid.
Defining a peer is troubling because, in a sense, all scientists are peers. However, obviously one would not want a physicist reviewing a microbiology paper right? Well maybe not. Sometimes different scientific disciplines offer a different, useful perspective, as was the case with the discovery of the DNA double helix. Yet they are not considered. Instead, a peer reviewer generally has the same field of study as the parties publishing the paper, and occasionally is even competing with them. To make matters worse, most reviewers don’t actually check the paper’s conclusions. Instead, most simply scan the paper to see if they agree with the results and if the paper is well written. There is little to no attempt to duplicate experimental results, resulting in frauds being easily perpetrated upon the scientific community. Sometimes reviewers will make negative comments about a paper due to a personal dislike of the author. Some journals will also publish some of the more well-known scientists, no matter what kind of reviews their paper gets or what the contents are. The article makes all these points to come to a very definitive conclusion. The process is flawed to the point of being utterly unreliable.
The article does not come to this conclusion lightly. The author is a former editor of a major medical periodical. During his tenure as editor, he tried numerous methods to improve the peer-review process. All of them failed. The reason for this is not because of a lack of effort or poorly designed experiments. Instead, the process is too broken to be fixed. It needs to be replaced. As it stands, the burden is entirely on the periodicals to doublecheck the articles. That is an issue that needs to be addressed. The scientists need to be incentivized to publish honestly.
One of the possible ways to fix peer review is to completely abandon it. The system to replace it would have to motivate scientists to publish good studies. The way to do this is twofold. First, force the scientists to pay for the right to submit to these journals. This would eliminate scientists publishing falsified studies because it would cost them money, grant money that they need to fund their research. As a double-check to this, the journals should use the funds from submissions to finance a staff of scientists to repeat studies sent to them. This would help ensure that there would be no further falsification of data.
The scientific community not only discovered inertia, but they are also handicapped by it. They have used peer review for so long that it is doubtful that they will abandon it, no matter the evidence that it does not work. In this respect, it differs little from the evolutionary theory. Peer review is a relic of the past that has no supporting evidence and should be dispensed with for a newer and more accurate method of checking.
Article in Journal of the Royal Society of Medicine
In His Image 
3 Comments