Peer reviews – are they peer?
Towards the end of last year, Nature published two articles that raised issues with the peer review system. In one, the claim was made that the reviewer of one paper had not even read the paper, because he commented on something that might usually be found in such a paper, but in this case was not. Now, naturally this would be a cause for concern for the author. However, the article then went on to a somewhat deeper issue, namely that in 2013, the articles indexed in Elsevier's Scopus rose to 2.7 million. Now not all of these would be peer reviewed, but you see the problem. There are just not enough experts to deal with this flood of material. What happens is that scientists are given papers that are more outside their specialty, and yes, the reviewer may be able to evaluate the methods and results section, but, according to the claim, lack the expertise to evaluate the introduction and discussion.
The article then made the claim that reviewers should verify the authors are quoting the right literature to support their views. Now, I dispute this. The reason for citing literature is when the papers put forward views or results that are of significance to the argument that will follow, but an introduction of a paper is not a general review of anything that is vaguely associated with the topic. So the first question is, is the work novel? The probability of recognizing plagiarism unfortunately increases very rapidly as the level of expertise and experience of the reviewer becomes more significant. Recall the word "peer"? If the work is accepted as novel, then the next question is, are there any facts or assumptions seemingly pulled out of thin air that should be referenced to some other paper? If so, those papers should be referenced, but there is a problem in that the reviewer should not be expected to do a full literature search. Surely the author has to take responsibility. The important thing is the author does not claim that to which he is not entitled. The third problem is, is there a reference that shows the work to be wrong? Again, an expert in the field should be able to show this, but how to find enough such experts? Experts, by definition, are a small fraction of the total scientists.
The second Nature article illustrated what I believe is a far more insidious problem: the peer review loop. Authors are often asked to recommend peer reviewers, so they do, within a small group of friends. In return, they know they will be recommended back. In a nice cozy circle, of course you recommend publication, with maybe the odd correction to show you did something. Even worse was the case where one author did his own reviews, and sent them to colleagues who in turn would submit them to the editor. This was caught out because everyone did it far too quickly. If you are going to cheat the system, obviously you should do it slowly, when the editor will be finally glad to have something on his table!
I believe recommendations from scientists should stop, but that then raises the question, how does the editor find reviewers? Personal knowledge is valuable, but with the great flood of papers coming in, can the editor know enough experts so that those he chooses are not overwhelmed? Then there is a problem with multi-author papers. If you look at some of the papers regarding the results from some of the NASA space probes, there may be up to fifty authors cited. In other words, anyone who knows sufficient is already an author.
But this raises the questions, why would scientists want to game the system, and why is peer review required? The answer to the first appears to be, to get more papers that go towards building up a reputation for promotion, awards, whatever. I suspect the scientific literature would become a lot more manageable if this practice were to stop. But the answer to the second one might come from something similar to the physicists' ArXiv; the authors can publish anything in e-form, but it is then open to public peer review, in which relevant comments from others can be attached. If the paper survives, it is worth keeping. If there is obvious plagiarism then the work is deleted and the plagiarist publicly identified. If some references have been left out, they can be added, although some check on relevance as opposed to self-citing would be required. If there is evidence the work is wrong, then that work can be submitted as a linked paper, and if the first paper is not adequately defended, then everyone will disregard it. Let the scientific community do the peer review. This would not have worked while journals were printed on paper, but when they are essentially e-journals, why not? What do you think?
Posted by
Ian Miller on Jul 5, 2015 11:47 PM Europe/London
