Objectives Peer review remains central to scientific publishing; yet, its reliability and true influence on scientific visibility remain debated, especially within subspecialized fields such as thoracic oncology. We aimed to evaluate reviewer agreement in manuscripts submitted to the European Journal of Cardio-Thoracic Surgery (EJCTS) and to examine whether review intensity predicts bibliometric outcomes. Methods A retrospective analysis of 144 thoracic oncology manuscripts submitted in 2021 was conducted. Each was assessed by 2-4 blinded reviewers. Reviewer recommendations were coded ordinally. Agreement was quantified using Fleiss' kappa, Cohen's kappa, and Krippendorff's alpha, while reviewer heterogeneity was assessed through entropy metrics. Citation and download counts were modelled using negative binomial regression adjusted for manuscript type and time to publication. Results Reviewer concordance was low (Fleiss' kappa = -0.03; Krippendorff's alpha = 0.041), with complete agreement observed in only 9.5% of manuscripts. Increasing the number of reviewers was associated with greater disagreement. Nevertheless, both higher reviewer count (inter-rater reliability [IRR] = 1.16; P = 0.004) and additional review rounds (IRR = 1.21; P < 0.001) independently predicted increased citation counts. Longer editorial timelines were associated with slightly reduced downloads (-0.9% per week; P = 0.048). Conclusions Reviewer consensus is uncommon in thoracic oncology submissions. However, greater reviewer engagement-both in number and iteration-was independently associated with improved bibliometric impact. These findings suggest that structured diversity of reviewer perspectives may enrich peer review and enhance scholarly dissemination, provided that editorial efficiency is preserved. This supports the view that disagreement among reviewers, when properly managed, may act as a catalyst rather than a constraint in scientific dissemination.

The price of consensus: Reviewer disagreement, editorial dynamics, and impact in thoracic oncology publishing / L. Bertolaccini, P. Falcoz. - In: EUROPEAN JOURNAL OF CARDIO-THORACIC SURGERY. - ISSN 1010-7940. - (2025). [Epub ahead of print] [10.1093/ejcts/ezaf375]

The price of consensus: Reviewer disagreement, editorial dynamics, and impact in thoracic oncology publishing

L. Bertolaccini
Primo
;
2025

Abstract

Objectives Peer review remains central to scientific publishing; yet, its reliability and true influence on scientific visibility remain debated, especially within subspecialized fields such as thoracic oncology. We aimed to evaluate reviewer agreement in manuscripts submitted to the European Journal of Cardio-Thoracic Surgery (EJCTS) and to examine whether review intensity predicts bibliometric outcomes. Methods A retrospective analysis of 144 thoracic oncology manuscripts submitted in 2021 was conducted. Each was assessed by 2-4 blinded reviewers. Reviewer recommendations were coded ordinally. Agreement was quantified using Fleiss' kappa, Cohen's kappa, and Krippendorff's alpha, while reviewer heterogeneity was assessed through entropy metrics. Citation and download counts were modelled using negative binomial regression adjusted for manuscript type and time to publication. Results Reviewer concordance was low (Fleiss' kappa = -0.03; Krippendorff's alpha = 0.041), with complete agreement observed in only 9.5% of manuscripts. Increasing the number of reviewers was associated with greater disagreement. Nevertheless, both higher reviewer count (inter-rater reliability [IRR] = 1.16; P = 0.004) and additional review rounds (IRR = 1.21; P < 0.001) independently predicted increased citation counts. Longer editorial timelines were associated with slightly reduced downloads (-0.9% per week; P = 0.048). Conclusions Reviewer consensus is uncommon in thoracic oncology submissions. However, greater reviewer engagement-both in number and iteration-was independently associated with improved bibliometric impact. These findings suggest that structured diversity of reviewer perspectives may enrich peer review and enhance scholarly dissemination, provided that editorial efficiency is preserved. This supports the view that disagreement among reviewers, when properly managed, may act as a catalyst rather than a constraint in scientific dissemination.
bibliometric analysis; editorial process; inter-rater reliability; lung cancer; peer review; reviewer agreement; scholarly dissemination
Settore MEDS-13/A - Chirurgia toracica
2025
Article (author)
File in questo prodotto:
File Dimensione Formato  
ezaf375.pdf

embargo fino al 31/10/2026

Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Licenza: Creative commons
Dimensione 1.74 MB
Formato Adobe PDF
1.74 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/1196306
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
  • OpenAlex 0
social impact