Out-of-distribution generalisation in spoken language understanding

Test data is said to be out-of-distribution (OOD) when it unex-
pectedly differs from the training data, a common challenge in
real-world use cases of machine learning. Although OOD gen-
eralisation has gained interest in recent years, few works have
focused on OOD generalisation in spoken language understand-
ing (SLU) tasks. To facilitate research on this topic, we intro-
duce a modified version of the popular SLU dataset SLURP,
featuring data splits for testing OOD generalisation in the SLU
task. We call our modified dataset SLURP For OOD gener-
alisation, or SLURPFOOD. Utilising our OOD data splits, we
find end-to-end SLU models to have limited capacity for gen-
eralisation. Furthermore, by employing model interpretability
techniques, we shed light on the factors contributing to the gen-
eralisation difficulties of the models. To improve the generali-
sation, we experiment with two techniques, which improve the
results on some, but not all the splits, emphasising the need for
new techniques.