Objectives Peer review remains central to scientific publishing; yet, its reliability and true influence on scientific visibility remain debated, especially within subspecialized fields such as thoracic oncology. We aimed to evaluate reviewer agreement in manuscripts submitted to the European Journal of Cardio-Thoracic Surgery (EJCTS) and to examine whether review intensity predicts bibliometric outcomes. Methods A retrospective analysis of 144 thoracic oncology manuscripts submitted in 2021 was conducted. Each was assessed by 2-4 blinded reviewers. Reviewer recommendations were coded ordinally. Agreement was quantified using Fleiss' kappa, Cohen's kappa, and Krippendorff's alpha, while reviewer heterogeneity was assessed through entropy metrics. Citation and download counts were modelled using negative binomial regression adjusted for manuscript type and time to publication. Results Reviewer concordance was low (Fleiss' kappa = -0.03; Krippendorff's alpha = 0.041), with complete agreement observed in only 9.5% of manuscripts. Increasing the number of reviewers was associated with greater disagreement. Nevertheless, both higher reviewer count (inter-rater reliability [IRR] = 1.16; P = 0.004) and additional review rounds (IRR = 1.21; P < 0.001) independently predicted increased citation counts. Longer editorial timelines were associated with slightly reduced downloads (-0.9% per week; P = 0.048). Conclusions Reviewer consensus is uncommon in thoracic oncology submissions. However, greater reviewer engagement-both in number and iteration-was independently associated with improved bibliometric impact. These findings suggest that structured diversity of reviewer perspectives may enrich peer review and enhance scholarly dissemination, provided that editorial efficiency is preserved. This supports the view that disagreement among reviewers, when properly managed, may act as a catalyst rather than a constraint in scientific dissemination.
The price of consensus: Reviewer disagreement, editorial dynamics, and impact in thoracic oncology publishing / L. Bertolaccini, P. Falcoz. - In: EUROPEAN JOURNAL OF CARDIO-THORACIC SURGERY. - ISSN 1010-7940. - (2025). [Epub ahead of print] [10.1093/ejcts/ezaf375]
The price of consensus: Reviewer disagreement, editorial dynamics, and impact in thoracic oncology publishing
L. Bertolaccini
Primo
;
2025
Abstract
Objectives Peer review remains central to scientific publishing; yet, its reliability and true influence on scientific visibility remain debated, especially within subspecialized fields such as thoracic oncology. We aimed to evaluate reviewer agreement in manuscripts submitted to the European Journal of Cardio-Thoracic Surgery (EJCTS) and to examine whether review intensity predicts bibliometric outcomes. Methods A retrospective analysis of 144 thoracic oncology manuscripts submitted in 2021 was conducted. Each was assessed by 2-4 blinded reviewers. Reviewer recommendations were coded ordinally. Agreement was quantified using Fleiss' kappa, Cohen's kappa, and Krippendorff's alpha, while reviewer heterogeneity was assessed through entropy metrics. Citation and download counts were modelled using negative binomial regression adjusted for manuscript type and time to publication. Results Reviewer concordance was low (Fleiss' kappa = -0.03; Krippendorff's alpha = 0.041), with complete agreement observed in only 9.5% of manuscripts. Increasing the number of reviewers was associated with greater disagreement. Nevertheless, both higher reviewer count (inter-rater reliability [IRR] = 1.16; P = 0.004) and additional review rounds (IRR = 1.21; P < 0.001) independently predicted increased citation counts. Longer editorial timelines were associated with slightly reduced downloads (-0.9% per week; P = 0.048). Conclusions Reviewer consensus is uncommon in thoracic oncology submissions. However, greater reviewer engagement-both in number and iteration-was independently associated with improved bibliometric impact. These findings suggest that structured diversity of reviewer perspectives may enrich peer review and enhance scholarly dissemination, provided that editorial efficiency is preserved. This supports the view that disagreement among reviewers, when properly managed, may act as a catalyst rather than a constraint in scientific dissemination.| File | Dimensione | Formato | |
|---|---|---|---|
|
ezaf375.pdf
embargo fino al 31/10/2026
Tipologia:
Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Licenza:
Creative commons
Dimensione
1.74 MB
Formato
Adobe PDF
|
1.74 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.




