Evaluation of the confusion matrix method in the validation of an automated system for measuring feeding behaviour of cattle

Salla Ruuska, Wilhelmiina Hämäläinen, Sari Kajava, Mikaela Mughal, Pekka Matilainen, Jaakko Mononen

Research output: Contribution to journalArticleScientificpeer-review

96 Citations (Scopus)

Abstract

The aim of the present study was to evaluate empirically confusion matrices in device validation. We compared the confusion matrix method to linear regression and error indices in the validation of a device measuring feeding behaviour of dairy cattle. In addition, we studied how to extract additional information on classification errors with confusion probabilities. The data consisted of 12 h behaviour measurements from five dairy cows; feeding and other behaviour were detected simultaneously with a device and from video recordings. The resulting 216 000 pairs of classifications were used to construct confusion matrices and calculate performance measures. In addition, hourly durations of each behaviour were calculated and the accuracy of measurements was evaluated with linear regression and error indices. All three validation methods agreed when the behaviour was detected very accurately or inaccurately. Otherwise, in the intermediate cases, the confusion matrix method and error indices produced relatively concordant results, but the linear regression method often disagreed with them. Our study supports the use of confusion matrix analysis in validation since it is robust to any data distribution and type of relationship, it makes a stringent evaluation of validity, and it offers extra information on the type and sources of errors.
Original languageEnglish
Pages (from-to)56-62
Number of pages7
JournalBEHAVIOURAL PROCESSES
Volume148
DOIs
Publication statusPublished - 1 Mar 2018
MoE publication typeA1 Journal article-refereed

Keywords

  • Confusion matrix
  • Confusion probabilities
  • Error indices
  • Feeding behaviour
  • Linear regression
  • Validation

Fingerprint

Dive into the research topics of 'Evaluation of the confusion matrix method in the validation of an automated system for measuring feeding behaviour of cattle'. Together they form a unique fingerprint.

Cite this