I originally wrote this article for ScienceMediaWatch, a blog that focussed on, among other things, the shortcomings of peer review. I thought it important to take a step back and explore the roots of the peer review model.
Dr. Steven Harnard recently got back to me with some suggested readings that add new depth to the history of peer review. So much depth in fact, that I thought it warranted the article to be posted here on WHSD – along with the suggested material…..
A popular theme here on ScienceMediaWatch has been the problems of peer review and the criticisms which confront it. With these problems in mind, a growing number of researches have tried to develop new systems of peer review that provide a secure quality control, while at the same time deal with some of the problems the system faces
As Anka discussed in a previous post, the web project Philica tries to solve the problems of peer review in academic journals by publishing the papers under an open access licence, to enable all registered academics in the project to review and comment on the papers (Open Peer Commentary). This project started in 2006, but a long time before that, the first pioneers in alternative review systems made their suggestions for a different approach to scientific quality control.
One of those review pioneers is Dr Stevan Harnard, who currently works at the School of Electronics and Computer Science at the University of Southampton. His starting point was that unlike novelists for example, scientists and scholars should not have to wish to protect their material from other people making copies from it (copyright them), a point which Professor Peter Suber reemphasised in a recent interview.
Whereas novelists need this mechanism to be able to sell their books and make their living out of it, for a scientists the benefits come not from selling copies, but firstly from the scientific impact his or her paper has. In order to increase this impact, the first step would be to make the paper as widely available as possible.
Although he published his ideas at a time where a lot of scientists were still sceptical about the internet, Dr Harnard soon recognised the big potential that the web offers for scientific publishing. It would make it possible to make papers available for everyone in just a few seconds, without having to contact the author before or even walking to a university library.
One particular problem of peer review Dr. Harnard wanted to target was the huge influence of the editors in chief of most peer reviewed journals, who usually pick the reviewers for the single submissions.
As I discussed in my previous post -this is a potential source of bias, since there is not only the potential for reviewers to bring their opinion to their reviewing, but also any underlying feuds or cliques.
Also, if the editor in chief picks reviewers who don’t have enough expertise in this particular field of research, especially if it is a very small one, a lack of understanding of the research could lead to a unjustifiable rejection of the paper. In an open review process online, these problems could be avoided.
In a 2007 interview with Richard Poynder, Harnad recalled founding the Open Peer Commentary journal Behavioural and Brain Sciences (BBS):
A traditional journal first conducts peer review, and then publishes articles that successfully meet the journal’s peer-review standards. BBS (and Current Anthropology, on which BBS was modelled) first circulates (accepted, peer-reviewed) articles to about 100 researchers, across specialities and around the world, inviting them to submit a 1,000-word commentary that critiques or complements the “target article.” The author then responds to the commentaries (12-20 or more) and it is all co-published in the journal.”
It was not long before the jump to OA was made. In 1989 Harnad founded the online counterpart to BBS, a peer-reviewed Open Peer Commentary journal called Psycoloquy – the very first OA journal.
Additional reading provided by Dr. Harnard:
PEER COMMENTARY: A SUPPLEMENT TO — NOT A SUBSTITUTE FOR — PEER REVIEW
(URLs shortened to evade overzealous spam filter And there is no “Hanard” or “Harnard”)
Harnad, S (1978) BBS Inaugural Editorial Behavioral and Brain Sciences 1(1)
See: userrs ecs soton ac harnad/Temp/Kata/bbs editorial
Harnad, S (ed ) (1982) Peer commentary on peer review: A case study in scientific quality control, New York: Cambridge University Press
See: eprints ecs soton ac uk/3389/
Harnad, S (1984) Commentaries, opinions and the growth of scientific knowledge American Psychologist 39: 1497 – 1498
Harnad, Stevan (1985) Rational disagreement in peer review Science, Technology and Human Values, 10 p 55-62
See: cogprints org/2128/
Harnad, S (1986) Policing the Paper Chase (Review of S Lock, A difficult balance: Peer review in biomedical publication ) Nature 322: 24 – 5
Harnad, S (1995) Interactive Cognition: Exploring the Potential of Electronic Quote/Commenting In: B Gorayska & J L Mey (Eds ) Cognitive Technology: In Search of a Humane Interface Elsevier Pp 397-414
See: cogprints org/1599/
Harnad, S (1996) Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals In: Peek, R & Newby, G (Eds ) Scholarly Publishing: The Electronic Frontier Cambridge MA: MIT Press Pp 103-118
See: cogprints org/1692/
Harnad, S (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright Learned Publishing 11(4) 283-292
See: cogprints org/1694/
Harnad, S (1998/2000/2004) The invisible hand of peer review Nature [online] (5 Nov 1998), Exploit Interactive 5 (2000): and in Shatz, B (2004) (ed ) Peer Review: A Critical Inquiry Rowland & Littlefield Pp 235-242
See: cogprints org/1646/
Harnad, S (2002) Self-Selected Vetting vs Peer Review: Supplement or Substitute? American Scientist Open Access Forum See: users ecs soton ac uk/harnad/Hypermail/Amsci/subject #msg2341
Harnad, S (2003/2004) Back to the Oral Tradition Through Skywriting at the Speed of Thought Interdisciplines Retour a la tradition orale: ecrire dans le ciel a la vitesse de la pensee Dans: Salaun, Jean-Michel & Vendendorpe, Christian (dir) Le défi de la publication sur le web: hyperlectures, cybertextes et meta-editions Presses de l’enssib
See: eprints ecs soton ac uk/7723/
Harnad, S (2003) BBS Valedictory Editorial Behavioral and Brain Sciences 26(1)
See: www ecs soton ac uk/%7Eharnad/Temp/bbs valedict
Harnad, S (2005) A-Priori Peer Review vs Post-Hoc Comments and Citations Open Access Archivangelism 9
See: openaccess eprints org/index php?/archives/9-guid
Harnad, S (2008) Validating Research Performance Metrics Against Peer Rankings Ethics in Science and Environmental Politics 8 (11) doi:10 3354/esep00088 The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance See: eprints ecs soton ac uk/15619/
Harnad, S (2010) No-Fault Peer Review Charges: The Price of Selectivity Need Not Be Access Denied or Delayed D-Lib Magazine 16 (7/8) See: eprints ecs soton ac uk/21348/