Communications of the ACM

Science demands an overhaul of the well-established system of peer-review in scholarly communication. The current system is outmoded, inefficient, and slow. The only question is how!

Cassidy Alan

The on-line and Internet process will work itself out. The "old-line authorities" will find a way to transfer peer review to the Web but the best peer review is to let everybody have a look at it that can do so.

Peer review evolved to deliberately bottleneck the process because the real estate in the pages of journals was limited, publication was the avenue to reputation, and there are too many submissions. The Web eliminates that razon d'etre for the current model of peer review.

So while keeping out the crazy stuff, and maybe catching errors in submissions, peer review has worked as a de facto firewall against maverick ideas that have merit like Joao Magueijo's "VSL" (Variable Speed of Light) theory, explained for the layman in the book "Faster Than the Speed of Light". It almost prevented publication.

Climategate also exposed another ugly underbelly of peer review: it hides conflicts of interest behind cloaks of anonymity. We see actors with massive emotional investment and large monetary grants at stake in personal communications confessed their own role in suppressing dissent.

But never fear. Post your idea on-line, and let 10,000 "peer reviewers" get an open look at it.

Witness the speed with which peer review handled the recent proposed resolution to the "P versus NP problem". In retrospect, putting it through the traditional peer review process, even a faster on-line version, would have been almost criminal in its delay. Science and technology was served well.

The traditional process has also surely suppressed efforts by competent amateurs to contribute to the process. Some do get respect due to established reputations, gotten the hard way, like Forest Mims, who bested NASA with home-built equipment.

There's nothing to fear here as the N-NP incident shows, and everything to gain. There is Internet geography sprouting up everywhere in which groups of scientists and technologists with dissenting views that peer review in "respected journals" would reject without hesitation, sometimes before getting to reviewers.

And worse, papers have been rejected by some such journals based on unrelated views prohibited by "established science orthodoxy", only to see others later get the credit for those ideas.

There will evolve forums with managed contribution that will emulate an Internet-speed version of the current model, with enhancements provided by the medium, and this will help sort out the fringe flak from the genuine article.

The net effect is an exponentially better model, high-octane acceleration in the process for good ideas, and benefits to all of us, as long as the political authorities in the legal and the academic worlds can be restrained from enforcing censorship on dissent in this expanded world.

David Roman

Cassidy Alan rightly points to the recent swift assessment of Vinay Deolalikar's proposed P-NP solution as a positive demonstration of web-based peer review, and I share his expectation that better online processes are forthcoming. Though I think they will take years and years to evolve, independent of how flawed the 'dead-tree' model of peer-review print publications may be. The print model is well-established and well-understood, and it successfully addresses a problem that Cassidy recognizes as a goal for online processes: to "help sort out the fringe flak from the genuine article," and to do so without creating obstacles, squelching new ideas, or resorting to censorship. To its advantage, the online world has a head start on avoiding these pitfalls.