Probabilistic user modelling methods for improving human-in-the-loop machine learning for prediction

In many machine learning applications and in particular those with only few training data, human involvement in the form of data provider or expert of the task is crucial. However, human interaction with a machine learning model is constrained by (i) the interaction channels, i.e., how human knowledge can be applied in the model, and (ii) the interaction budget, i.e., how much the user is willing to interact with the model. This thesis presents new methods to improve these constraints in human-in-the-loop machine learning. The core idea of the thesis is to jointly model the available data with a model of the human user, i.e., the user model, in a unified probabilistic model and then perform sequential probabilistic inference on the joint model to design improved interaction. The thesis contributes on two types of prediction tasks. The first task is expert knowledge elicitation for high-dimensional prediction. Experts in a field usually have information beyond training data which can help to improve the prediction performance. User models, as priors and likelihood functions, are proposed to directly connect expert knowledge about the relevance of parameters to a model responsible for prediction. The user model can account for complex user behaviour such as users updating their knowledge during the interaction. Furthermore, sequential experimental design on the joint model is employed to query the most informative expert knowledge earlier to minimize the amount of interaction. The second task is personalized recommendation where the goal is to predict the most relevant item for a user with as few interactions as possible. The interactions are based on user relevance feedback on the recommendations. The thesis proposes user models that are able to receive and integrate feedback on multiple domains and sources by providing a joint probabilistic model connecting all feedback types. Sequential inference on the joint model, using Thompson sampling, was employed to find the targeted recommendation with minimum interaction. Simulated experiments and user studies in both tasks demonstrate improved prediction performance only after few interactions with the users. The research highlights the benefits of joint probabilistic modelling of the user and prediction model in interactive tasks.