Precise measurement is difficult but essential in the generation of high-quality data, and it is therefore remarkable that often so little attention is paid to intercoder reliability. It is commonly recognized that poor validity leads to systematic errors and biased inference. In contrast, low reliability is generally assumed to be a lesser concern, leading only to random errors and inefficiency. We evaluate the intercoder reliability of our recently collected data on governance events in UN peacekeeping and show how poor coding and low intercoder reliability can produce systematic errors and even biased inference. We also show how intercoder reliability checks are useful to improve data quality. Continuous testing for intercoder reliability ex post enables researchers to create better data and ultimately improves the quality of their analyses. © Taylor & Francis Group, LLC.
Events data as bismarck's sausages? intercoder reliability, coders' selection, and data quality / A. Ruggeri, T.-. Gizelis, H. Dorussen. - In: INTERNATIONAL INTERACTIONS. - ISSN 0305-0629. - 37:3(2011), pp. 340-361. [10.1080/03050629.2011.596028]
Events data as bismarck's sausages? intercoder reliability, coders' selection, and data quality
A. Ruggeri
Primo
;
2011
Abstract
Precise measurement is difficult but essential in the generation of high-quality data, and it is therefore remarkable that often so little attention is paid to intercoder reliability. It is commonly recognized that poor validity leads to systematic errors and biased inference. In contrast, low reliability is generally assumed to be a lesser concern, leading only to random errors and inefficiency. We evaluate the intercoder reliability of our recently collected data on governance events in UN peacekeeping and show how poor coding and low intercoder reliability can produce systematic errors and even biased inference. We also show how intercoder reliability checks are useful to improve data quality. Continuous testing for intercoder reliability ex post enables researchers to create better data and ultimately improves the quality of their analyses. © Taylor & Francis Group, LLC.File | Dimensione | Formato | |
---|---|---|---|
Ruggeri_II.pdf
accesso riservato
Descrizione: Special Data Feature
Tipologia:
Publisher's version/PDF
Dimensione
312.46 kB
Formato
Adobe PDF
|
312.46 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.