top of page

Peer Review: An Opportunity

Writer's picture: Mishkat BhattacharyaMishkat Bhattacharya

Peer review is a highly contentious topic in academia. This is because papers are published (always) and funds are granted (usually) after they have been peer reviewed. Sometimes (often?) the reviewers are in disagreement with the authors of the paper/grant proposal. This is generally fine if the disagreement is purely scientific.


However, conflicts of interest often show up in these interactions, because the professionals best equipped to judge the paper/grant are often the competitors of the authors from the same or a closely related field. They are competing with the authors for scientific recognition (professional prestige) and money.


In principle the reviewers can recuse themselves from judging, but often the prospect of striking down a competitor is simply too tempting - a plain Darwinian survival/climbing tactic. Even if this strike is delivered rather blatantly, the reviewers know that the journal editors/grant managers will be assuredly on their side because journals/agencies are overwhelmed with submissions/applications and want to quickly get rid of most of them.


Thus, pretty much the whole academic enterprise operates on the cusp of this balance of expertise and self-interest on the part of the reviewers. Some would call it asking the fox to guard the henhouse.



In order to allow for free criticism, the identities of the reviewers are usually withheld. Some journals offer 'double-blind' peer review, where the identities of the authors are also withheld. But it is not overly difficult from the contextual evidence (subject, references, etc.) to establish the authors' identities.


Some journals also allow reviewers to state their names in their reports, in case they feel comfortable doing so. Some journals offer the option, to the authors, of posting the entire review correspondence after the paper is published. This, of course, avoids the issue of making public the sometimes unfair reviews which lead to rejection.


A Different Possible Solution


I think it is high time - and a good opportunity - that professional reviewing services were offered by companies set up explicitly for that purpose. Such a company would hire professional scientists from different fields, preferably with at least a postdoctoral degree, who are unaffiliated to any institutions (and recuse themselves from judging submissions from their alma maters - some cross reviewing would have to be done internally to ensure fairness).


These scientists would, through the company, offer their services for unbiased refereeing to various journals and funding agencies. To keep up and extend their expertise, they would continually read the literature, and, if necessary, attend seminars, colloquia, workshops, and conferences.


They may, while reviewing, choose to elicit the opinions of experts practicing in the field, but without divulging the explicit contents of the paper/grant they are judging. For example, they could say to an expert: "If someone sent in a paper claiming they had done so-and-so what questions would you ask of them?" This type of consultation, with the accumulation of time and expertise, would hopefully become rare. AI could possibly be used effectively here, as an additional unbiased resource.

Postscript


If paid well, this could be a very good career option for the professionals who choose this path; they could make a good industrial salary while still remaining intimately connected to academia, and in fact playing a crucial role in advancing science.


One can think of evolving a grading rubric, with factors such as originality and impact, degree of technical advance, interest to general public...which these companies could generate, so any paper/grant processed by them would carry away a meaningful stamp useful to other agencies.


Papers (which often end up taking months for a single page of review report to arrive because the professor was busy with other duties) and grants would receive much more prompt attention. The conflicts of interest would go down dramatically. This might mean more papers/grants receiving favorable reviews, but ultimate acceptance/awardance rates could be regulated by journal editors and program managers.

Postpostcript


The company need not be very big. In 2020 there were 10,000 FTE (full-time equivalent) physics faculty in the US. If each employee of the company was able to take the reviewing load of 10 such faculty, this would lead to a company of 1000 employees; AI could probably trim this down to 200. That's just for physics...there could be a company for medicine, one for engineering, one for chemistry....

Recent Posts

See All

Comments


Responsible comments are welcome at mb6154@gmail.com. All material is under copyright ©.

© 2023 by Stories from Science. Powered and secured by Wix

bottom of page