Optimizing Multiple Consumer-specific Objectives in End-to-End Ensemble Machine Learning Serving

Tri Nguyen, Linh Truong, Paolo Arcaini, Fuyuki Ishikawa

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

75 Downloads (Pure)

Abstract

Optimizing the quality of machine learning (ML) services for individual consumers with specific objectives is crucial for improving consumer satisfaction. In this context, end-to-end ensemble ML serving (EEMLS) faces many challenges in selecting and deploying ensembles of ML models on diverse resources across the edge-cloud continuum. This paper provides a method for evaluating the runtime performance of inference services via consumer-defined metrics. We enable ML consumers to define high-level metrics and consider consumer satisfaction in estimating service costs. Moreover, we introduce a time-efficient ensemble selection algorithm to optimize the EEMLS with intricate trade-offs between service quality and costs. Our intensive experiments demonstrate that the algorithm can be executed periodically despite the extensive search space, enabling dedicated optimization for individual consumers in dynamic contexts.
Original languageEnglish
Title of host publication2024 IEEE/ACM 17th International Conference on Utility and Cloud Computing (UCC)
Number of pages6
Publication statusAccepted/In press - 5 Nov 2024
MoE publication typeA4 Conference publication
EventIEEE/ACM International Conference on Utility and Cloud Computing - The University of Sharjah, Sharjah, United Arab Emirates
Duration: 16 Dec 202419 Dec 2024
Conference number: 17
https://www.uccbdcat2024.org/ucc/

Conference

ConferenceIEEE/ACM International Conference on Utility and Cloud Computing
Abbreviated titleUCC
Country/TerritoryUnited Arab Emirates
CitySharjah
Period16/12/202419/12/2024
Internet address

Keywords

  • ML Serving
  • Ensemble Selection
  • Ensemble ML
  • End-to-End ML
  • Performance Evaluation

Fingerprint

Dive into the research topics of 'Optimizing Multiple Consumer-specific Objectives in End-to-End Ensemble Machine Learning Serving'. Together they form a unique fingerprint.

Cite this