Service registration chatbot: collecting and comparing dialogues from AMT workers and service’s users

Luca Molteni, Mittul Singh, Juho Leinonen, Katri Leino, Mikko Kurimo, Emanuele Della Valle

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

22 Lataukset (Pure)

Abstrakti

Crowdsourcing is the go-to solution for data collection and annotation in the context of NLP tasks. Nevertheless, crowdsourced data is noisy by nature; the source is often unknown and additional validation work is performed to guarantee the dataset’s quality. In this article, we compare two crowdsourcing sources on a dialogue paraphrasing task revolving around a chatbot service. We observe that workers hired on crowdsourcing platforms produce lexically poorer and less diverse rewrites than service users engaged voluntarily. Notably enough, on dialogue clarity and optimality, the two paraphrase sources’ human-perceived quality does not differ significantly. Furthermore, for the chatbot service, the combined crowdsourced data is enough to train a transformer-based Natural Language Generation (NLG) system. To enable similar services, we also release tools for collecting data and training the dialogue-act-based transformer-based NLG module.
AlkuperäiskieliEnglanti
OtsikkoProceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)
Sivut116-121
Sivumäärä6
ISBN (elektroninen)978-1-952148-76-7
DOI - pysyväislinkit
TilaJulkaistu - marraskuuta 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaConference on Empirical Methods in Natural Language Processing - Virtual, Online
Kesto: 16 marraskuuta 202020 marraskuuta 2020

Conference

ConferenceConference on Empirical Methods in Natural Language Processing
KaupunkiVirtual, Online
Ajanjakso16/11/202020/11/2020

Sormenjälki

Sukella tutkimusaiheisiin 'Service registration chatbot: collecting and comparing dialogues from AMT workers and service’s users'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä