The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementa- tion and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional prop- erties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the appli- cability of our approach in evaluating the fairness of different ML models.

A Methodology for Non-Functional Property Evaluation of Machine Learning Models / M. Anisetti, C.A. Ardagna, E. Damiani, P.G. Panero - In: MEDES '20: Proceedings / [a cura di] R. Chbeir, Y. Manolopoulos, E. Damiani, D. Benslimane. - [s.l] : ACM, 2020 Nov. - ISBN 9781450381154. - pp. 38-45 (( Intervento presentato al 12. convegno International Conference on Management of Digital EcoSystems (MEDES '20) tenutosi a Abu Dhabi nel 2020 [10.1145/3415958.3433101].

A Methodology for Non-Functional Property Evaluation of Machine Learning Models

M. Anisetti;C.A. Ardagna;E. Damiani;
2020

Abstract

The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementa- tion and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional prop- erties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the appli- cability of our approach in evaluating the fairness of different ML models.
Non-functional properties; Machine Learning Assurance; Multi-armed bandit
Settore INF/01 - Informatica
Book Part (author)
File in questo prodotto:
File Dimensione Formato  
main.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 705.8 kB
Formato Adobe PDF
705.8 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/841580
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact