In this chapter, we present the Argo crowdsourcing system and related functionalities for crowd-based summary evaluation. Then, we evaluate the pros and cons of three different approaches to single-document, multi-lingual summary evaluation: i) traditional, expertbased; ii) crowdsourcing, majority-based; and iii) crowdsourcing, Argobased. The evaluation is performed over two languages of the MultiLing-2015 Single Document Summarization (MSS) Dataset, examining result under different aspects.
Crowdsourcing in Single-document Summary Evaluation: the Argo Way / N. Pittaras, S. Montanelli, G. Giannakopoulos, A. Ferrara, V. Karkaletsis - In: Multilingual Text Analysis: Challenges, Models, and Approaches / [a cura di] M. Litvak, N. Vanetik. - [s.l] : World Scientific, 2019. - ISBN 9789813274891. - pp. 245-280 [10.1142/9789813274884_0008]
Crowdsourcing in Single-document Summary Evaluation: the Argo Way
S. MontanelliSecondo
;A. FerraraPenultimo
;
2019
Abstract
In this chapter, we present the Argo crowdsourcing system and related functionalities for crowd-based summary evaluation. Then, we evaluate the pros and cons of three different approaches to single-document, multi-lingual summary evaluation: i) traditional, expertbased; ii) crowdsourcing, majority-based; and iii) crowdsourcing, Argobased. The evaluation is performed over two languages of the MultiLing-2015 Single Document Summarization (MSS) Dataset, examining result under different aspects.
| File | Dimensione | Formato | |
|---|---|---|---|
|
ws-rv9x6.pdf
accesso riservato
Tipologia:
Pre-print (manoscritto inviato all'editore)
Dimensione
746.87 kB
Formato
Adobe PDF
|
746.87 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
