The ID Report

by Denyse O'Leary
ARN correspondent

Peer review did not really become widely used until the middle of the 20th century, after World War II. As the Editors of The New Atlantis have pointed out in a recent article, "Rethinking Peer Review"(Summer 2006),

In the nineteenth century, many science journals were commandingly led by what Ohio State University science historian John C. Burnham dubbed "crusading and colorful editors," who made their publications "personal mouthpieces" for their individual views. There were often more journals than scientific and medical papers to publish; the last thing needed was a process for weeding out articles.

Albert Einstein's key 1905 papers were published in the German review Annalen der Physik without any review other than the approval of editors Max Planck and Wilhelm Wien. Of course, it can justly be argued that Planck, known for Planck's Constant, and Wien, known for Wien's law, were Einstein's peers. But with the flurry of science publishing that the twentieth century ushered in, that situation could not last. Increasing specialization in sciences meant that by the early twentieth century, journal editors were no longer able to evaluate the merits of all the articles they wanted to publish. They began to direct papers to experts who served on editorial boards affiliated with their journals.

After World War II, specialized science research boomed. Researchers followed up on a number of useful discoveries made during the war (for example, antibiotics and nuclear technology). The subsequent Cold War, and then the space race, kept alive the drive for further military advances as well.

But only specialists could assess the specialized papers that resulted, so editors increasingly deferred to them. The development of the Xerox copier in 1959 made the new system practical, so all the major journals adopted peer review. As the New Atlantis Editors point out, the journal editors were pretty sure they were onto a good thing:

In recent times, the term "peer reviewed" has come to serve as shorthand for "quality." To say that an article appeared in a peer-reviewed scientific journal is to claim a kind of professional approbation; to say that a study hasn't been peer reviewed is tantamount to calling it disreputable.

Indeed, that sounds so reasonable, what could go wrong?

Well, first, as New Atlantis points out, "peer review is not simply synonymous with quality." Physicist Frank Tipler offers a number of instances of landmark papers or major findings on such subjects as lasers and black holes that were rejected by peer reviewers. He also offers:

On the Nobel Prize web page one can read the autobiographies of recent laureates. Quite a few complain that they had great difficulty publishing the ideas that won them the Prize. One does not find similar statements by Nobel Prize winners earlier in the century.

Indeed, Tipler suggests,

Today, Einstein’s papers would be sent to some total nonentity at Podunk U, who, being completely incapable of understanding important new ideas, would reject the papers for publication. "Peer" review is very unlikely to be peer review for the Einsteins of the world. We have a scientific social system in which intellectual pygmies are standing in judgment of giants.

Whereas in Einstein's day, a paper introducing a new idea in physics stood a good chance of being reviewed by a Nobel Prize winner like Planck, today, the researcher would have to write several hundred papers to score a Nobelist as a reviewer. Indeed, the researcher might seldom encounter an original thinker, just dozens of people carrying out the tasks of mainstream "normal" science, as Thomas Kuhn described it.

Major bodies such as the Cochrane Collaboration and the Royal Society, as well as others have added volume to the dissatisfaction with peer review. According to Dr. Altman , "Studies have found that journals publish findings based on sloppy statistics," though

None of the recent flawed studies have been as humiliating as an article in 1972 in the journal Pediatrics that labeled sudden infant death syndrome a hereditary disorder, when, in the case examined, the real cause was murder.

Some have responded that the peer reviewer never gets enough time to look at the statistics, et cetera. While that observation is doubtless true, it hardly restores one's eroded confidence in the system.

Generally, the two most common complaints are that peer review fails to safeguard quality, which was its original purpose and that it punishes new ideas, regardless of merit.

Next: Part Two: How bad can it get? Pretty bad.