The notion of consensus is frequently employed in crowdsourcing applications/systems as a solution for quality assessment of task results, either based on an explicit or implicit notion of agreement among workers. In this paper, we describe the task-design activities in our crowdsourcing systems called Argo, where consensus/agreement parameters and constraints are specified and actually enforced. The configuration of system-level parameters is discussed and experience of task-design and execution on a real case-study of web-resource labeling is presented.
Designing crowdsourcing tasks with consensus constraints / S. Castano, A. Ferrara, S. Montanelli - In: International Conference on Collaboration Technologies and Systems[s.l] : IEEE, 2016. - ISBN 9781509022991. - pp. 97-103 (( convegno CTS tenutosi a Orlando nel 2016 [10.1109/CTS.2016.33].
Designing crowdsourcing tasks with consensus constraints
S. Castano;A. Ferrara;S. Montanelli
2016
Abstract
The notion of consensus is frequently employed in crowdsourcing applications/systems as a solution for quality assessment of task results, either based on an explicit or implicit notion of agreement among workers. In this paper, we describe the task-design activities in our crowdsourcing systems called Argo, where consensus/agreement parameters and constraints are specified and actually enforced. The configuration of system-level parameters is discussed and experience of task-design and execution on a real case-study of web-resource labeling is presented.
| File | Dimensione | Formato | |
|---|---|---|---|
|
07870973.pdf
accesso riservato
Tipologia:
Publisher's version/PDF
Dimensione
543.84 kB
Formato
Adobe PDF
|
543.84 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
