Learning to peer review

GENETICS Editor-in-Chief Mark Johnston introduces a new peer review training program for early career scientists.

“Just tell them what you think of them.” That was the response of one of my mentors when I asked him how I should review grant applications. I was a newly-minted Assistant Professor and had been asked to sit on an NIH study section. I had only a vague idea of how to go about reviewing grant applications, so I turned to my trusted colleague for advice.

I got invited back to the study section. So I must have done something right. But it felt like being tossed into the deep end of the pool before having a swimming lesson. That’s one way to learn. But perhaps it’s not the best way.

Peer-reviewers are vital to the scientific enterprise. They provide a check-and-balance for science by critically evaluating the authors’ (their peers) stories.  They check that the data support the authors’ conclusions. Are the data convincing? Does it meet statistical standards? Have the authors done the necessary controls? By answering these questions in the affirmative, peer-reviewers validate the authors’ findings; by raising concerns about these points, peer-reviewers identify errors in the work that authors surely want to avoid. And peer-reviewers provide a check of the authors’ presentation. Is it clear? Is it persuasive? In my experience, peer-review almost always helps authors improve articles.

Peer-reviewers help editors determine which stories should enter the scientific record. Reviewers must maintain high standards to protect the integrity of the literature, but they must also have reasonable expectations of authors (their peers). Science advances incrementally, after all, and reviewers and editors need to determine how much of an advance justifies readers’ attention—to judge when a story warrants becoming a brick in the Great Wall of Knowledge. It’s a big responsibility.

You’d think such an important task would require advanced training, but there’s no formal training that I know of. Many graduate programs provide their trainees with practice reviewing manuscripts and grant applications, but the scope and effectiveness of those exercises vary widely. Some, though far from all, faculty advisors provide their students opportunities to review manuscripts, often on an informal basis. This patchy system inadvertently robs many students and postdocs of the chance to hone some of the skills central to success in science—understanding the mindset and expectations of peer reviewers and editors, critical thinking, evaluating research, and providing feedback on scientific projects not directly related to your own.

But just because we’ve always done it this way, doesn’t mean it’s the only way.

During her time serving on the GSA’s Publications Committee, Early Career Liaison Aleeza Gerstein drew our attention to this inequality and variability in peer review training. Aleeza works in a field (evolutionary genetics) in which senior students and postdocs traditionally get more opportunities for inclusion in the peer review process, so she was surprised when she learned her experience was not the norm. Across our field as a whole, students and postdocs report uneven experiences in training for peer-review. Aleeza suggested that the GSA is in a good position to help train the next generation of peer reviewers. The entire GSA community could serve as a valuable resource for our early career colleagues.

With the enthusiastic support of GENETICS Senior Editor David Greenstein (now GSA Secretary and Publications Committee Chair), the Editorial Board and the Publications Committee (particularly Elyse Hope and David Fay) are working with Sonia Hall, GSA’s Director of Engagement and Development, to develop a program that will give early career GSA members real-world peer review experience.

To pilot this program, we are currently recruiting the first group of GSA member graduate students, postdoctoral fellows, and junior faculty to serve as peer-reviewers for the journal.

Trainee reviewers will receive training on the principles, purposes, and best practices of peer-review, as well as guidelines and models for fair reviews that are helpful to both the authors and the editors. Participants will review manuscripts submitted to GENETICS that are within their areas of interest and expertise. Just like for any other peer-reviewer, the participants’ reviews will be provided to the authors and considered by the editor in making their decisions.

The trainee reviewers will receive feedback in two ways. First, they will read the other reviews and the decision letter. Seeing how other, more experienced, reviewers do the job will reveal much about the process and nuances of the task, as well as illuminate the path of an academic paper from initial submission through to final publication. And seeing how the editor weighs the reviewers’ opinions and takes their comments into account in coming to a decision on the manuscript will demonstrate what is most salient in reviewers’ comments. Second, we want the trainee reviewers to benefit from the expertise of the GENETICS’ editorial board, so editors will provide feedback to the reviewers about their reviews. I’m hoping that will consist of more than just “tell them what you think of it.”

And beyond the world of publishing, we expect participants to benefit in many ways. Good peer reviewers are skilled at communicating specialist information in an accessible way. They are able to give feedback that is constructive and fair. Chances like this to get feedback are remarkably rare, despite the fact that an important part of being a scientist is regularly critiquing peers and mentees! Participants will demonstrate their understanding of responsible publication and authorship practices, their willingness to contribute to the discipline, along with many of those hard-to-show “soft” skills like workplace etiquette, knowing when to seek advice, time management, and reliably meeting deadlines.

Peer-review is a cornerstone of science. We should not leave training for such an important activity to chance. The editors of GENETICS look forward to working with our young colleagues to develop the journal’s next generation of peer-reviewers.

Learn more about the GENETICS Peer Review Training Program.