Novel Contract-based Runtime Explainability Framework for End-to-End Ensemble Machine Learning Serving

Tri Nguyen, Linh Truong, Tram Truong-Huu

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaKonferenssiesitysScientificvertaisarvioitu

94 Lataukset (Pure)


The growing complexity of end-to-end Machine Learning (ML) serving across the edge-cloud continuum has raised the necessity for runtime explainability to support service optimizations, transparency, and trustworthiness. That involves many challenges in managing ML service quality and engineering runtime explainability based on ML service contracts. Currently, consumers use ML services almost as a black box with insufficient explainability for not only inference decisions but also other contractual aspects, such as data/service quality and costs. The generic explainability for ML models is inadequate to explain the runtime ML usage for individual consumers. Moreover, ML-specific metrics have not been addressed in existing service contracts. In this work, we introduce a novel contract-based runtime explainability framework for end-to-end ensemble ML serving. The framework provides a comprehensive engineering toolset, including explainability constraints in ML contracts, report schemas, and interactions between ML consumers and the components of the ML serving for evaluating service quality with contract-based explanations. We develop new monitoring probes to measure ML-specific metrics on data quality, inference confidence, inference accuracy, and capture runtime ML usage. Finally, we present essential quality analyses via an observation agent. That interprets ML inferences and evaluates contributions of ML inference microservices, assisting ML serving optimization. The agent also integrates ML algorithms for detecting relations among metrics, supporting constraint developments. We demonstrate our work with two real-world applications for malware and object detection.
TilaJulkaistu - 2024
OKM-julkaisutyyppiEi oikeutettu
TapahtumaInternational Conference on AI Engineering: Software Engineering for AI - Lisbon, Lisbon, Portugali
Kesto: 14 huhtik. 202415 huhtik. 2024
Konferenssinumero: 3


ConferenceInternational Conference on AI Engineering


Sukella tutkimusaiheisiin 'Novel Contract-based Runtime Explainability Framework for End-to-End Ensemble Machine Learning Serving'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.
  • Demonstration Paper: Monitoring Machine Learning Contracts with QoA4ML

    Nguyen, T. & Truong, L., 19 huhtik. 2021, s. 169-170. 2 Sivumäärä.

    Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaPosterScientificvertaisarvioitu

    Open access
    120 Lataukset (Pure)
  • QoA4ML – A Framework for Supporting Contracts in Machine Learning Services

    Truong, L. & Nguyen, T., 15 marrask. 2021, 2021 IEEE International Conference on Web Services (ICWS). Chang, C. K., Damiani, E., Fan, J., Ghodous, P., Maximilien, M., Wang, Z., Ward, R. & Zhang, J. (toim.). IEEE, s. 465-475 11 Sivumäärä

    Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

    Open access
    2 Sitaatiot (Scopus)
    324 Lataukset (Pure)

Siteeraa tätä