Projects per year
Experts and crowds can work together to generate high-quality datasets, but such collaboration is limited to a large-scale pool of data. In other words, training on a large-scale dataset depends more on crowdsourced datasets with aggregated labels than expert intensively checked labels. However, the limited amount of high-quality dataset can be used as an objective test dataset to build a connection between disagreement and aggregated labels. In this paper, we claim that the disagreement behind an aggregated label indicates more semantics (e.g. ambiguity or difficulty) of an instance than just spam or error assessment. We attempt to take advantage of the informativeness of disagreement to assist learning neural networks by computing a series of disagreement measurements and incorporating disagreement with distinct mechanisms. Experiments on two datasets demonstrate that the consideration of disagreement, treating training instances differently, can promisingly result in improved performance.
|Number of pages||7|
|Publication status||Published - 4 Sep 2021|
|MoE publication type||A1 Journal article-refereed|
- Assessed datasets
- Instance weight
- Neural networks
FingerprintDive into the research topics of 'Deep neural learning on weighted datasets utilizing label disagreement from crowdsourcing'. Together they form a unique fingerprint.
INTERVENE: International consortium for integrative genomics prediction
01/01/2021 → 31/12/2025
Project: EU: Framework programmes funding
Data Literacy for Responsible Decision-Making
Marttinen, P., Gröhn, T., Honkamaa, J., Kumar, Y., Ji, S., Raj, V., Ojala, F., Pöllänen, A. & Tiwari, P.
01/10/2020 → 30/09/2023
Project: Academy of Finland: Strategic research funding
Intelligent Crop Production: Data-integrative, Multi-task Learning Meets Crop Simulator
Mamitsuka, H., Nariman Zadeh, H., Strahl, J., Guvenc, B., Ji, S., Rissanen, S., Pöllänen, A., Honkamaa, J., Hiremath, S. & Ojala, F.
01/01/2018 → 31/12/2022
Project: Academy of Finland: Other research funding