Peer review is key for public trust of academic journals. It ensures that only rigorous research is published but also helps authors to increase the value of their manuscripts through feedback from reviewers. However, measuring the developmental value of peer review is difficult as it requires fine-grained manuscript data on various stages of the editorial process, which are rarely available. To fill this gap, we accessed complete data from Royal Society journals from 2006 to 2017, and measured manuscript changes during peer review from their initial submissions. We then estimated the effect of the number of reviewers and the evaluation of reviewers on manuscript development and their citations after publication. We found that the number of reviewers had an almost linear effect on manuscript change although with decreasing marginal effects whenever more than two reviewers were involved. This effect did not depend on the initial quality of manuscripts. We also found that changes due to reviewers tended to increase a manuscript’s probability of being cited at least once after publication. While our findings show the multiple functions of peer review for manuscript development, research with larger and more representative journal samples is needed to develop more systematic measures that reflect the complexity of peer review.

Measuring the effect of reviewers on manuscript change: A study on a sample of submissions to Royal Society journals (2006–2017) / F. Bianchi, D. García-Costa, F. Grimaldo, F. Squazzoni. - In: JOURNAL OF INFORMETRICS. - ISSN 1751-1577. - 16:3(2022 Aug). [10.1016/j.joi.2022.101316]

Measuring the effect of reviewers on manuscript change: A study on a sample of submissions to Royal Society journals (2006–2017)

F. Bianchi
Primo
;
F. Squazzoni
Ultimo
2022

Abstract

Peer review is key for public trust of academic journals. It ensures that only rigorous research is published but also helps authors to increase the value of their manuscripts through feedback from reviewers. However, measuring the developmental value of peer review is difficult as it requires fine-grained manuscript data on various stages of the editorial process, which are rarely available. To fill this gap, we accessed complete data from Royal Society journals from 2006 to 2017, and measured manuscript changes during peer review from their initial submissions. We then estimated the effect of the number of reviewers and the evaluation of reviewers on manuscript development and their citations after publication. We found that the number of reviewers had an almost linear effect on manuscript change although with decreasing marginal effects whenever more than two reviewers were involved. This effect did not depend on the initial quality of manuscripts. We also found that changes due to reviewers tended to increase a manuscript’s probability of being cited at least once after publication. While our findings show the multiple functions of peer review for manuscript development, research with larger and more representative journal samples is needed to develop more systematic measures that reflect the complexity of peer review.
Peer review; Journals; Manuscripts; Reviewers; Manuscript changes
Settore SPS/07 - Sociologia Generale
Article (author)
File in questo prodotto:
File Dimensione Formato  
am.pdf

accesso aperto

Tipologia: Pre-print (manoscritto inviato all'editore)
Dimensione 520.04 kB
Formato Adobe PDF
520.04 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/935407
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 0
social impact