Naboj is a dynamical website that lets you review online scientific articles. Right now the only articles that are available for review are those that have been posted at Los Alamos arXiv.
It is often said that peer review is one of the pillars of scientific research. It is also well known that peer review doesn't actually do its job very well, and, every few years, people like me start writing articles about alternatives to peer review. This isn't one of those rants. Instead, I'm going to focus on something that is probably less well known: peer review actually has two jobs. It's used to provide minimal scrutiny for new scientific results, and to act as a gatekeeper for funding agencies.
What I would like to do here is outline some of the differences between peer review in these two jobs and the strengths and weaknesses of peer review in each case. This is not a rant against peer review, nor should it be—I have been pretty successful in both publications and grant applications over the last couple of years. But I think it's worth exploring the idea that peer review functions much better in the case of deciding the value of scientific research than it does when acting as a gatekeeper for scientific funding.
D. Taraborelli (2008), Soft peer review. Social software and distributed scientific evaluation, Proceedings of the 8th International Conference on the Design of Cooperative Systems (COOP 08), Carry-Le-Rouet, France, May 20-23, 2008
Drawing
on sociocultural theory, the present study investigated how children in an intensive elementary
level Grade 6 class for English as a second language (ESL) scaffolded each other while carrying
out cooperative learning tasks.
B. Cao, B. Plale, G. Subramanian, P. Missier, C. Goble, and Y. Simmhan. International Workshop on the role of Semantic Web in Provenance Management (SWPM), volume 526 of CEUR Workshop Proceedings, page 1--6. CEUR-WS.org, (October 2009)
L. Massoulié, E. Le Merrer, A. Kermarrec, and A. Ganesh. Proceedings of the twenty-fifth annual ACM symposium on Principles of distributed computing, page 123--132. New York, NY, USA, ACM, (2006)
N. Herbert. ACE '07: Proceedings of the ninth Australasian conference on Computing education, page 63--71. Darlinghurst, Australia, Australia, Australian Computer Society, Inc., (2007)
B. Cao, B. Plale, G. Subramanian, P. Missier, C. Goble, and Y. Simmhan. International Workshop on the role of Semantic Web in Provenance
Management (SWPM), volume 526 of CEUR Workshop Proceedings, CEUR-WS.org, (October 2009)