This paper presents an index that measures reviewer contribution to editorial processes of scholarly journals. Following a metaphor of ranking algorithms in sports tournaments, we created an index that considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions. To test the index, we used a dataset of peer review in a multi-disciplinary journal, including 544 reviewers on 606 submissions in six years. Although limited by sample size, the test showed that the index identifies outstanding contributors and weak performing reviewers efficiently. Our index is flexible, contemplates extensions and could be incorporated into available scholarly journal management tools. It can assist editors in rewarding high performing reviewers and managing editorial turnover.

The F3-index : Valuing reviewers for scholarly journals / F. Bianchi, F. Grimaldo, F. Squazzoni. - In: JOURNAL OF INFORMETRICS. - ISSN 1751-1577. - 13:1(2019 Feb 01), pp. 78-86. [10.1016/j.joi.2018.11.007]

The F3-index : Valuing reviewers for scholarly journals

F. Bianchi
Primo
Writing – Original Draft Preparation
;
F. Squazzoni
Ultimo
Writing – Original Draft Preparation
2019

Abstract

This paper presents an index that measures reviewer contribution to editorial processes of scholarly journals. Following a metaphor of ranking algorithms in sports tournaments, we created an index that considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions. To test the index, we used a dataset of peer review in a multi-disciplinary journal, including 544 reviewers on 606 submissions in six years. Although limited by sample size, the test showed that the index identifies outstanding contributors and weak performing reviewers efficiently. Our index is flexible, contemplates extensions and could be incorporated into available scholarly journal management tools. It can assist editors in rewarding high performing reviewers and managing editorial turnover.
Computer Science Applications1707 Computer Vision and Pattern Recognition; Library and Information Sciences
Settore SPS/07 - Sociologia Generale
1-feb-2019
Article (author)
File in questo prodotto:
File Dimensione Formato  
BianchiGrimaldoSquazzoni2018F3Index.pdf

accesso aperto

Descrizione: Articolo principale
Tipologia: Publisher's version/PDF
Dimensione 1.05 MB
Formato Adobe PDF
1.05 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/626040
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 14
  • ???jsp.display-item.citation.isi??? 11
social impact