The future of peer review?

[jay gw]

Peer review - subjecting scientific research to expert scrutiny - is the only acceptable method to adequately judge the rigour and accuracy of the research. Recently, however the approach has come under fire as a series of high-profile scientific misconduct have exposed its many flaws.

The European Science Foundation (ESF) has joined the European Heads of Research Councils (EuroHORCs) and the Czech Science Foundation (Grantová agentura Èeské republiky, GA ÈR) to host an international conference on peer review, which will be held in Prague on October 12-13, 2006.

* Is the peer-review process in the present form able to identify the best frontier science and how to improve it?
* What is the best way to harmonise the peer-review process and how can new methods and IT tools contribute?
* What are the major societal, cultural and ethical challenges of the future of the peer-review process and how could they be incorporated?

http://www.eurekalert.org/pub_releases/2006-10/esf-jsp100906.php

Are there "many" flaws in the current peer review process?

 

[Cuddles]

No doubt there are many problems with the peer review, but I really can't see how it could be improved. Research is worthless unless it can be validated by other people who understand what is going on. At the moment this basically relies on voluntary work, which is the main reason it is not as robust as would be hoped. The only way around this is to pay people, but this brings problems of conflict of interest and funding. I suppose some kind of Wiki system could work, but this would be difficult to guard against fraud and would have numerous problems if differences of opinon arose.

 

[rockoon]

We tried to discuss this subject several years ago: http://www.internationalskeptics.com/forums/showthread.php?t=24225&highlight=peer

You can ignore the part where JJ and Huntsman stroke each others egos and pretend like there isnt a lot of corruption in the peer review process, since clearly there is and based on recent events it appears to be getting even worse than it was.

 

[drkitten]

We tried to discuss this subject several years ago: http://www.internationalskeptics.com/forums/showthread.php?t=24225&highlight=peer

... and unfortunately none of the so-called critics in that thread actually had any criticism to offer.

I suspect that the European conference will end similarly. No one will actually have any suggestions.

 

[Ziggurat]

I'll tell you the problem with peer review. I think this, ultimately, is the ENTIRE problem: the scholars doing the reviews do not get rewarded for their efforts. There isn't much incentive to spend significant amounts of time doing reviews, and so reviews often end up as fairly cursory affairs. Everything else stems from this, and it will not change unless there's a mechanism in place to reward reviewers for their efforts. But I don't know of a good way to implement such a mechanism, and I've never heard any proposed that could be both widespread (in terms of reviewers AND journals) and significant. And such a mechanism wouldn't come free, either. So I think we're stuck with the problems of the current system, but those problems aren't fatal by any means.