Intelligent Design is Peer-Reviewed, but is Peer-Review a Requirement of Good Science?

Although belief in a young earth is not in the list of things we young-earth creationists have in common with intelligent design groups, the intelligent design proponents have mounted a commendable attack against the dogma of evolution in the last several years. Casey Luskin is a Research Coordinator for the Center for Science and Culture at the Discovery Instititue, a notable intelligent design research group. Luskin holds graduate degrees in both science and law. He earned his BS and MS in Earth Sciences from the University of California, San Diego. His law degree is from the University of San Diego. Luskin recently published an article at evolutionnews.org for Evolution News and Views presenting the fallacy that science is only good science if it is published in peer-reviewed literature. The organization provides original reporting and analysis about the debate over intelligent design and evolution, including breaking news about scientific research. The following is Luskin’s article first published at http://www.evolutionnews.org/2012/02/intelligent_des056221.html

Intelligent Design Is Peer-Reviewed, but Is Peer-Review a Requirement of Good Science?

By Casey Luskin, February 10, 2012

As may be seen from our newly updated page listing Peer-Reviewed & Peer-Edited Scientific Publications Supporting the Theory of Intelligent Design, the ID movement has developed a diverse research program bearing fruit in the form of more than 50 peer-reviewed scientific papers. Beyond doubt, ID proponents have published a significant body of legitimate peer-reviewed research.

In the past, critics charged that ID cannot be taken seriously until it has published just such “peer-reviewed research.” Yet it’s important to understand that being recognized in the peer-reviewed literature is not an absolute requirement to demonstrate an idea’s scientific merit.

Good Science without Peer-Review

Some of the most important and groundbreaking work in the history of science first appeared in published form not in peer-reviewed scientific journal articles but in scientific books. That includes Copernicus’ De Revolutionibus and Newton’s Principia. Einstein’s original paper on relativity was published in a scientific journal (Annalen der Physik), but did not undergo formal peer-review. 1 Indeed, Darwin’s own theory of evolution was first published in a book for a general and scientific audience -- his Origin of Species—not in a peer-reviewed paper.

Moreover, important scientific work has not uncommonly been initially rejected by peer-reviewed journals. As a 2001 article in Science observed, “Mention ‘peer review’ and almost every scientist will regale you with stories about referees submitting nasty comments, sitting on a manuscript forever, or rejecting a paper only to repeat the study and steal the glory.” 2 Indeed, an article in the journal Science Communication by Juan Miguel Campanario notes that top journals such as “Science and Nature have also sometimes rejected significant papers,” and in fact “Nature has even rejected work that eventually earned the Nobel Prize.” 3 In an amusing letter titled “Not in our Nature,” Campanario reminds the journal of four examples where it rejected significant papers:

(1) In 1981, Nature rejected a paper by the British biochemist Robert H. Michell on signalling reaction by hormones. This paper has since been cited more than 1,800 times.

(2) In June 1937, Nature rejected Hans Krebs’s letter describing the citric acid cycle. Krebs won the 953 Nobel prize in physiology or medicine for this discovery.

(3) Nature initially rejected a paper on work for which Harmut Michel won the 1988 Nobel prize for chemistry; it has been identified by the Institute of Scientific Information as a core document and widely cited.

(4) A paper by Michael J. Berridge, rejected in 1983 by Nature, ranks at number 275 in a list of the most-cited papers of all time. It has been cited more than 1,900 times. 4

Elsewhere, Campanario lists “instances in which 36 future Nobel Laureates encountered resistance on the part of scientific journal editors or referees to manuscripts that dealt with discoveries that on later dates would assure them the Nobel Prize.” 5 Likewise, Tulane University physicist Frank Tipler offers the following anecdotes:

“Another example is Günter Blobel, who in a news conference given just after he was awarded the Nobel Prize in Medicine, said that the main problem one encounters in one’s research is ‘when your grants and papers are rejected because some stupid reviewer rejected them for dogmatic adherence to old ideas.’ According to the New York Times, these comments ‘drew thunderous applause from the hundreds of sympathetic colleagues and younger scientists in the auditorium.’“

“[W]hen [Stephen] Hawking submitted to Nature what is generally regarded as his most important paper, the paper on black hole evaporation, the paper was initially rejected. I have heard from colleagues who must remain nameless that when Hawking submitted to Physical Review what I personally regard as his most important paper, his paper showing that a most fundamental law of physics called ‘unitarity’ would be violated in black hole evaporation, it, too, was initially rejected.”

“Today it is known that the Hawaiian Islands were formed sequentially as the Pacific plate moved over a hot spot deep inside the Earth. The theory was first developed in the paper by an eminent Princeton geophysicist, Tuzo Wilson: ‘I ... sent [my paper] to the Journal of Geophysical Research. They turned it down.... They said my paper had no mathematics in it, no new data, and that it didn’t agree with the current views. Therefore, it must be no good.’“

“On the Nobel Prize web page one can read the autobiographies of recent laureates. Quite a few complain that they had great difficulty publishing the ideas that won them the Prize.” 6

In light of these kinds of examples, Campanario concludes:

Something is wrong with the peer review system when an expert considers that a manuscript is not of enough interest and it later becomes a classic in its discipline (or, even worse, when the work reported in a rejected paper earns the Nobel Prize). ... Contrary to reports by the American Association for the Advancement of Science and the National Academy of Sciences, publication in a peer-reviewed journal is not necessarily the best means of identifying valid research. 3

The Supreme Court Agrees

Even the U.S. Supreme Court has recognized that good science will not always be published in a peer-reviewed journal. In the landmark 1993 case Daubert v. Merrell Dow Pharmaceuticals, Inc., the Court observed that while publication in peer-reviewed journals can be an indicator of legitimate science, it is not necessarily an indicator of good science:

Publication (which is but one element of peer review) is not a sine qua non of admissibility; it does not necessarily correlate with reliability, and in some instances well-grounded but innovative theories will not have been published. Some propositions, moreover, are too particular, too new, or of too limited interest to be published. 7

While the Court noted that “submission to the scrutiny of the scientific community is a component of ‘good science,’“ it ultimately held that “publication (or lack thereof) in a peer reviewed journal” is merely “a relevant, though not dispositive, consideration in assessing the scientific validity” of a claim. 8

The point that scientific progress depends upon considering minority views and unpopular ideas that may not have been published in peer-reviewed journals was made emphatically and eloquently by Stephen Jay Gould, writing with other scientists and historians of science, in an amicus brief submitted to the Supreme Court regarding the Daubert case:

Judgments based on scientific evidence, whether made in a laboratory or a courtroom, are undermined by a categorical refusal even to consider research or views that contradict someone’s notion of the prevailing “consensus” of scientific opinion. Science progresses as much or more by the replacement of old views as by the gradual accumulation of incremental knowledge. Automatically rejecting dissenting views that challenge the conventional wisdom is a dangerous fallacy, for almost every generally accepted view was once deemed eccentric or heretical. Perpetuating the reign of a supposed scientific orthodoxy in this way, whether in a research laboratory or in a courtroom, is profoundly inimical to the search for truth. A categorical refusal even to examine and consider scientific evidence that conflicts with some ill-defined notion of majority opinion is a recipe for error in any forum.

Unable or unwilling to investigate scientific methodology and determine just what is orthodox and “generally accepted,” the Ninth Circuit instead seized upon publication in a peer-reviewed scientific journal as the badge of respectability, the sine qua non of admissible “good science.” The court thereby converted that editorial tool into something no scientist or journal editor ever meant it to be: a litmus test for scientific truth. This is not the way scientists work in their laboratories and symposia, and it is not the way that science should be used in the courtroom if the goal is to ensure the most accurate and valid judgments possible. [...]

As scientists, physicians, historians of science and sociologists of science who are members of the “scientific community,” amici can assure the Court that this is not how scientists work in their pursuit of truth. Amici challenge the Ninth Circuit’s premise that the only “good science” is that which is “generally accepted” and published in peer-reviewed journals, and reject the notion that scientific analysis and conclusions that might diverge from what a court deems the published “consensus” are so unreliable as to be wholly unworthy of consideration. The quality of a scientific approach or opinion depends on the strength of its factual premises and on the depth and consistency of its reasoning, not on its appearance in a particular journal or on its popularity among other scientists. Even if it were possible to determine the existence and character of a “consensus,” which is itself a task fraught with difficulty, prevailing views and conventional wisdom have all too often been consigned to the dust heap of the history of science. If the purpose of the Federal Rules of Evidence is to enable the fact-finder to make the most informed decision possible, by providing the assistance of qualified experts who possess “scientific, technical or other specialized knowledge [that] will assist the trier of fact to understand the evidence or to determine a fact in issue,” Fed.R.Evid. 702, then it would be a grave mistake to require that all scientific analysis be supported by a consensus and published in a particular form in order to be considered. 9

Thus, there are many compelling arguments from both the theory and practice of the scientific enterprise that demonstrate peer-review is not an absolute requirement of good science.

The Peer-Review System Under Attack

Arguments against peer-review generally come in two forms: (1) that the system wrongly rejects scientifically valid papers, and (2) that the system wrongly accepts scientifically flawed papers. A 2008 paper in the Financial Times summarized these dual criticisms:

[T]he process is under assault from critics who say it is ineffective at filtering out poor research, while it perpetuates predictable work at the expense of more imaginative thinking. In the long run we all suffer, argues Don Braben of University College London, because economic growth depends on unpredictable scientific advances. 10

Similarly, in 2006 journalist and medical doctor Lawrence K. Altman wrote in The New York Times that the peer-reviewed system has been severely weakened by allowing the publication of flawed or even fraudulent studies:

Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system. ... Virtually every major scientific and medical journal has been humbled recently by publishing findings that are later discredited. The flurry of episodes has led many people to ask why authors, editors and independent expert reviewers all failed to detect the problems before publication. 11

Likewise Richard Smith, former editor of the British Medical Journal, comments that peer review is “very slow, expensive, a considerable lottery, completely hopeless at detecting errors and fraud, and there’s evidence of bias.” 12 Indeed, a 2001 article in Science reported that in one study, “researchers found little evidence that peer review actually improves the quality of research papers.” 2

As a result of such criticisms, a 2012 article in The New York Times noted that scientists are becoming increasingly skeptical of the peer-review process:

For centuries, this is how science has operated -- through research done in private, then submitted to science and medical journals to be reviewed by peers and published for the benefit of other researchers and the public at large. But to many scientists, the longevity of that process is nothing to celebrate.

The system is hidebound, expensive and elitist, they say. Peer review can take months, journal subscriptions can be prohibitively costly, and a handful of gatekeepers limit the flow of information. It is an ideal system for sharing knowledge, said the quantum physicist Michael Nielsen, only “if you’re stuck with 17th-century technology.” 13

A 2011 article by David Colquhoun, Professor of Pharmacology at University College in London also argues that the peer-review system “doesn’t work very well any more, mainly as a result of the enormous number of papers that are being published (an estimated 1.3 million papers in 23,750 journals in 2006).” 14 In Colquhoun’s view, “There simply aren’t enough competent people to do the job” of reviewing papers, and thus “Any paper, however bad, can now get published in a journal that claims to be peer-reviewed.” 14 Complicit in this problem, Colquhoun says, is the “publish-or-perish culture”:

The blame for this sad situation lies with the people who have imposed a publish-or-perish culture, namely research funders and senior people in universities. To have “written” 800 papers is regarded as something to boast about rather than being rather shameful. University PR departments encourage exaggerated claims, and hard-pressed authors go along with them. 14

The Myth of the Objective Expert

Despite these criticisms, non-experts often mistakenly believe that individual scientists who serve as editors or referees on papers are always open-minded and completely objective in reviewing papers. According to this naïve assumption, peer-review is a flawless gold-standard which guarantees the legitimacy of a paper. Stephen Jay Gould sought to dispel these myths when he observed that “[t]he stereotype of a fully rational and objective ‘scientific method,’ with individual scientists as logical and interchangeable robots, is self-serving mythology.” 15 Likewise, Lawrence Altman’s article in the New York Times explains:

Many nonscientists perceive reviewers to be impartial. But the reviewers, called independent experts, in fact are often competitors of the authors of the papers they scrutinize, raising potential conflicts of interest. 11

Unfortunately, public perceptions of perfectly “objective” and “impartial” reviewers are reinforced by some who wield “peer-review” as a political tool to shut down minority dissenting views. According to Altman, many have overstated the integrity of the peer-review process, thus “creat[ing] a widespread misimpression that passing peer review is the scientific equivalent of the Good Housekeeping seal of approval.” 11 He continues that in reality, the canonization of the peer-review process amounts to a power-grab by scientific journals:

By promoting the sanctity of peer review and using it to justify a number of their actions in recent years, journals have added to their enormous power. 11

Altman paints a disturbing picture of the peer-review system, where journals have huge economic interests in preserving the current flawed system, and research scientists gladly play along because peer-reviewed papers are necessary for them to maintain their positions:

Despite its flaws, scientists favor the system in part because they need to publish or perish. The institutions where the scientists work and the private and government agencies that pay for their grants seek publicity in their eagerness to show financial backers results for their efforts.

The public and many scientists tend to overlook the journals’ economic benefits that stem from linking their embargo policies to peer review. Some journals are owned by private for-profit companies, while others are owned by professional societies that rely on income from the journals. The costs of running journals are low because authors and reviewers are generally not paid.

A few journals that not long ago measured profits in the tens of thousands of dollars a year now make millions, according to at least three editors who agreed to discuss finances only if granted anonymity, because they were not authorized to speak about finances.

Any influential system that profits from taxpayer-financed research should be held publicly accountable for how the revenues are spent. Journals generally decline to disclose such data. 11

Given this lack of perfect objectivity, another major problem with the peer-review system is its non-double-blind nature. When articles are sent out for review, the reviewers often (though not always) are told who authored the paper they are reviewing. And if they aren’t told, they often can figure it out simply by being aware of who is working on what projects within the field. Reviewers who perceive the authors as competitors or enemies might be biased against the paper, and more likely to reject the work even if it has merit. Reviewers who perceive the authors as allies or friends are more likely to accept the work, even if it has weaknesses. Given the pressure on everyone to “publish or perish,” reviewers may feel tempted to approve papers for publication, hoping that when they want their own papers published, others in the field will do the same for them.

Political concerns, economic factors, lab-rivalry, support for one’s friends, and other normal human elements are never completely divorced from the peer-review process.

Playing the “Peer-Review” Card to Shut Down Dissent

Despite the deficiencies in the peer-review system, “peer-review” is increasingly used as a rhetorical weapon, enlisted for the purpose of silencing dissenting, minority scientific viewpoints. University of Kent sociologist Frank Furedi has explained the alarming rise of what he calls “advocacy science,” which defends itself not by citing data but by advocating the myth of infallible peer-review:

[I]n recent years the most disturbing threat to the integrity of the peer-review system has been the growing influence of advocacy science. In numerous areas, most notably in climate science, research has become a cause and is increasingly both politicized and moralized. Consequently, in climate research, peer review is sometimes looked upon as a moral project, where decisions are influenced not simply by science but by a higher cause. 16

Furedi believes that peer-review is wielded as a political weapon:

Increasingly, peer review is cited as kind of unquestioned and unquestionable authority for settling what are in fact political disputes. Consequently, the findings of peer review are looked upon, not simply as statements about the quality of research or of a scientific finding, but as the foundation for far-reaching policies that affect everything from the global economy to our individual lifestyles.

Increasingly, peer review has been turned into a quasi-holy institution, which apparently signifies that a certain claim is legitimate or sacred. And from this perspective, voices which lack the authority of peer review are, by definition, illegitimate. Peer review provides a warrant to be heard—those who speak without this warrant deserve only our scorn.

You can almost visualize peer-review dogmatists waving their warrant and demanding that their opponents be silenced. For someone like George Monbiot, the British climate-change alarmist, peer review is the equivalent of a holy scripture. Boasting of his encounter with an opponent, who challenged him to a debate on speed cameras, Monbiot wrote: “I accepted and floored him with a simple question.” Predictably, the question was: “Has he published his analysis in a peer-reviewed journal?” 16

A revealing anecdote of those who play the “peer review” card to shut down dissent comes from the debate over claims made by NASA scientists in 2010 that they had discovered arsenic-based life. The controversy began when the NASA scientists announced their findings in a press conference where they promised, through much media hype, to reveal “an astrobiology finding that will impact the search for evidence of extraterrestrial life.” As we discussed here, their claims were critiqued by many scientists.

Initially, those criticisms came in the form of non-peer-reviewed rebuttals posted on the Internet or made in interviews for news articles. The NASA scientists who had made the claims of arsenic-based life refused to respond to those criticisms other than making comments like “we hope to see this work published in a peer-reviewed journal, as this is how science best proceeds.”

Evolutionary biologist Jonathan A. Eisen recently wrote a post titled “Stop deifying ‘peer review’ of journal publications,” calling out the hypocrisy of those NASA scientists. In short, they hyped their claims in the media -- and thus promoted their views outside of the peer-reviewed literature -- but then refused to answer critics who hadn’t published rebuttals in the peer reviewed literature. Eisen wrote the following about the refusal of the NASA scientists to address the criticisms:

This was amazing since, well, they were the ones who held the overhyped press conference. And then I (and others) found it appalling that they in essence would not response to critiques because they were not “peer reviewed.” ... Whether they were right or not in their claims, they are now hypocritical if they say that the only response should be in the scientific literature.

Eventually the critics did publish their rebuttals in the peer-reviewed literature (we also covered the peer-reviewed responses here). But this episode reveals the hypocrisy of those push their views upon the public through non-peer reviewed venues like the media, but then try to shut down critics for responding in non-peer-reviewed venues.

But is it always feasible for those who dissent from the majority viewpoint to gain access to publication in peer-reviewed journals?

On Scientific Bias, Lessons from the History of Science

Historians of science understand perfectly well why minority scientific viewpoints have difficulty being published in peer-reviewed scientific papers. A paper in the Journal of the American Society for Information Science observes that scientists can be biased against accepting new ideas:

Sometimes scientists encounter strong resistance from peers to their new ideas. The scientific community often finds it difficult to accept new ideas or methods and unexpected observations. The greatest and most harmful source of resistance from scientists to scientific discovery comes precisely from those peers whose mission is to preserve the quality of scientific work: The editors and referees of scientific journals. 17

Similarly, the famous historian of science Thomas Kuhn wrote in his seminal book The Structure of Scientific Revolutions:

No part of the aim of normal science is to call forth new sorts of phenomena; indeed those that will not fit the box are often not seen at all. Nor do scientists normally aim to invent new theories, and they are often intolerant of those invented by others. 18

As a new scientific theory that challenges much conventional wisdom, intelligent design faces this very type of intolerance. In one case, pro-ID biochemist Michael Behe submitted an article for publication in a scientific journal but was told it could not be published because “your unorthodox theory would have to displace something that would be extending the current paradigm.” 19 Behe concluded:

The take-home lesson I have learned is that, while some science journal editors are individually tolerant and will entertain thoughts of publishing challenges to current views, when a group (such as the editorial board) gets together, orthodoxy prevails. 19

Science journalist Denyse O’Leary agrees, writing that “The overwhelming flaw in the traditional peer review system is that it listed so heavily toward consensus that it showed little tolerance for genuinely new findings and interpretations.” 20 In Frank Tipler’s view, intelligent design theory faces such knee-jerk intolerance:

It is impossible to get any member of the National Academy of Sciences to consider it seriously. The typical reaction of such scientists is to foam at the mouth when the phrase “intelligent design” is mentioned. I have recently experienced this. In the fall of 2002, I arranged for Bill Dembski to come to Tulane to debate a Darwinian on the Tulane faculty. (This faculty member was appropriately named Steve Darwin!) Bill presented only the evidence against Darwinism in the debate, while Steve’s response unfortunately had quite a few ad hominem remarks. Steve has continued to be friendly to me personally. But ever since the Dembski/Darwin debate, another evolutionist on the Tulane faculty -- who shall remain nameless! -- glares at me every time he sees me. Before the debate he and I were friends. Now he considers me a monster of moral depravity.

...If my Tulane University ex-friend ever reads these words, he would want to do more than glare. He would want to strangle me for writing such a heresy! He definitely would not approve of these words were he to be the referee of a paper of mine wherein this argument is repeated. He would definitely reject any grant proposal I would make that contains these words. 1

Indeed, some of the most prominent scientific journals have all-but declared that they would never publish an ID-friendly paper. In 2002, the American Association for the Advancement of Science (AAAS), which publishes Science, issued an official declaration that “the ID movement has failed to offer credible scientific evidence.” 21 But when some of the AAAS board members were surveyed, it was found that they voted to declare “intelligent design as unscientific without actually reading for themselves the academic books and articles by scientists proposing the theory.” 22 Evidently, when it comes to intelligent design, politics rather than science drives the behavior of many journals.

Despite the attempted lockout, ID proponents have published their ideas in peer-reviewed scientific journals. This shows that ID has academic legitimacy whether or not one applies the dubious “peer-review” test of good science.

• 1 a b See Frank Tipler, “Refereed Journals: Do They Insure Quality or Enforce Orthodoxy?,” International Society for Complexity, Information, and Design Archives (June 30, 2003).
• 2 a b Martin Enserink, “Peer Review and Quality: A Dubious Connection?,” Science, Vol. 293:2187-2188 (September 21, 2001).
• 3 a b Juan Miguel Campanario, “On Influential Books and Journal Articles Initially Rejected Because of Negative Referees’ Evaluations,” Science Communication, Vol. 16(3):304-325 (March, 1995).
• 4Juan Miguel Campanario, “Not in our Nature,” Nature, Vol. 361:488 (February 11, 1993).
• 5See Juan Miguel Campanario, “Rejecting Nobel class articles and resisting Nobel class discoveries”
• 6Frank Tipler, “Refereed Journals: Do They Insure Quality or Enforce Orthodoxy?,” International Society for Complexity, Information, and Design Archives (June 30, 2003) (internal citations removed).
• 7Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593-594 (1993) (internal citations removed).
• 8Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 594 (1993) (emphasis added).
• 9Brief Amici Curiae of Physicians, Scientists, and Historians of Science in Support of Petitioners, Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
• 10Clive Cookson and Andrew Jack, “Science stifled?,” Financial Times (June 12, 2008).
• 11 a b c d e Lawrence K. Altman, “For Science’s Gatekeepers, a Credibility Gap,” New York Times (May 2, 2006).
• 12Richard Smith, quoted in Clive Cookson and Andrew Jack, “Science stifled?,” Financial Times (June 12, 2008).
• 13Thomas Lin, “Cracking Open the Scientific Process,” New York Times (January 16, 2012).
• 14 a b c David Colquhoun, “Publish-or-perish: Peer review and the corruption of science,” The Guardian (September 5, 2011).
• 15Stephen Jay Gould, “In the Mind of the Beholder,” Natural History, Vol. 103 (2):15 (1994).
• 16 a b Frank Furedi, “Turning peer review into modern-day holy scripture,” Spiked (February 23, 2010).
• 17Juan Miguel Campanerio, “Have Referees Rejected Some of the Most-Cited Articles of All Times?,” Journal of the American Society for Information Science, Vol. 47(4):302-310 (1996).
• 18Thomas Kuhn, The Structure of Scientific Revolutions, p. 24 (2nd Ed, University of Chicago Press, 1970).
• 19 a b See Michael Behe, “Correspondence with Science Journals: Response to critics concerning peer-review” (August 2, 2000).
• 20Denyse O’Leary, “Part Four: Conclusion - How will we know if a more open system works better?,” ARN.org (November 15, 2006).
• 21See “AAAS Board Resolution on Intelligent Design Theory” (October 18, 2002).
• 22See John G. West, “Intelligent Design Could Offer Fresh Ideas on Evolution,” Seattle Post-Intelligencer(December 6, 2002).