In online surveys, the use of manipulation checks aimed at measuring respondents’ attentiveness has become common. More than being measures of attentiveness pertaining to a specific survey, instructional manipulation checks (IMC) could work as generic measures of the quality of the answers a person gives when completing a questionnaire. By using several waves of the ITANES (Italian National Election Study)–University of Milan online panel survey, the article shows that the outcome of an IMC predicts the quality of the answers a respondent gives in previous and subsequent waves of the panel. Moreover, through a survey experiment that randomizes the length of an IMC we show that, overall, the answers’ quality of ‘attentive’ respondents assigned to different IMCs do not substantially vary. Findings also show that IMCs are reliable measures, as the outcome of two IMCs placed in two consecutive waves proved to be highly associated.

Assessing general attentiveness to online panel surveys: the use of instructional manipulation checks / R. Ladini. - In: INTERNATIONAL JOURNAL OF SOCIAL RESEARCH METHODOLOGY. - ISSN 1364-5579. - 25:2(2022), pp. 233-246. [10.1080/13645579.2021.1877948]

Assessing general attentiveness to online panel surveys: the use of instructional manipulation checks

R. Ladini
2022

Abstract

In online surveys, the use of manipulation checks aimed at measuring respondents’ attentiveness has become common. More than being measures of attentiveness pertaining to a specific survey, instructional manipulation checks (IMC) could work as generic measures of the quality of the answers a person gives when completing a questionnaire. By using several waves of the ITANES (Italian National Election Study)–University of Milan online panel survey, the article shows that the outcome of an IMC predicts the quality of the answers a respondent gives in previous and subsequent waves of the panel. Moreover, through a survey experiment that randomizes the length of an IMC we show that, overall, the answers’ quality of ‘attentive’ respondents assigned to different IMCs do not substantially vary. Findings also show that IMCs are reliable measures, as the outcome of two IMCs placed in two consecutive waves proved to be highly associated.
attentiveness; data quality; Instructional manipulation checks; online survey; panel survey
Settore SPS/07 - Sociologia Generale
2022
feb-2021
Article (author)
File in questo prodotto:
File Dimensione Formato  
Ladini 2022 IJSRM.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 1.89 MB
Formato Adobe PDF
1.89 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/813653
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 2
social impact