Active Learning for Decision-Making from Imbalanced Observational Data

Iiris Sundin, Peter Schulam, Eero Siivola, Aki Vehtari, Suchi Saria, Samuel Kaski

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

14 Lataukset (Pure)

Abstrakti

Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action a to take for a target unit after observing its covariates x~ and predicted outcomes p^(y~∣x~,a). An example case is personalized medicine and the decision of which treatment to give to a patient. A common problem when learning these models from observational data is imbalance, that is, difference in treated/control covariate distributions, which is known to increase the upper bound of the expected ITE estimation error. We propose to assess the decision-making reliability by estimating the ITE model’s Type S error rate, which is the probability of the model inferring the sign of the treatment effect wrong. Furthermore, we use the estimated reliability as a criterion for active learning, in order to collect new (possibly expensive) observations, instead of making a forced choice based on unreliable predictions. We demonstrate the effectiveness of this decision-making aware active learning in two decision-making tasks: in simulated data with binary outcomes and in a medical dataset with synthetic and continuous treatment outcomes.
AlkuperäiskieliEnglanti
Otsikko36th International Conference on Machine Learning, ICML 2019
Sivut10578-10587
Sivumäärä10
ISBN (elektroninen)9781510886988
TilaJulkaistu - 2019
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Conference on Machine Learning - Long Beach, Yhdysvallat
Kesto: 9 kesäkuuta 201915 kesäkuuta 2019
Konferenssinumero: 36

Julkaisusarja

NimiProceedings of Machine Learning Research
KustantajaPMLR
Vuosikerta97
ISSN (elektroninen)1938-7228

Conference

ConferenceInternational Conference on Machine Learning
LyhennettäICML
MaaYhdysvallat
KaupunkiLong Beach
Ajanjakso09/06/201915/06/2019

Sormenjälki Sukella tutkimusaiheisiin 'Active Learning for Decision-Making from Imbalanced Observational Data'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

  • Laitteet

    Science-IT

    Mikko Hakala (Manager)

    Perustieteiden korkeakoulu

    Laitteistot/tilat: Facility

  • Siteeraa tätä

    Sundin, I., Schulam, P., Siivola, E., Vehtari, A., Saria, S., & Kaski, S. (2019). Active Learning for Decision-Making from Imbalanced Observational Data. teoksessa 36th International Conference on Machine Learning, ICML 2019 (Sivut 10578-10587). (Proceedings of Machine Learning Research; Vuosikerta 97).