The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementa- tion and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional prop- erties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the appli- cability of our approach in evaluating the fairness of different ML models.
A Methodology for Non-Functional Property Evaluation of Machine Learning Models / M. Anisetti, C.A. Ardagna, E. Damiani, P.G. Panero - In: MEDES '20: Proceedings / [a cura di] R. Chbeir, Y. Manolopoulos, E. Damiani, D. Benslimane. - [s.l] : ACM, 2020 Nov. - ISBN 9781450381154. - pp. 38-45 (( Intervento presentato al 12. convegno International Conference on Management of Digital EcoSystems (MEDES '20) tenutosi a Abu Dhabi nel 2020 [10.1145/3415958.3433101].
A Methodology for Non-Functional Property Evaluation of Machine Learning Models
M. Anisetti;C.A. Ardagna;E. Damiani;
2020
Abstract
The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementa- tion and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional prop- erties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the appli- cability of our approach in evaluating the fairness of different ML models.File | Dimensione | Formato | |
---|---|---|---|
main.pdf
accesso aperto
Tipologia:
Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione
705.8 kB
Formato
Adobe PDF
|
705.8 kB | Adobe PDF | Visualizza/Apri |
anisetti2020.pdf
accesso riservato
Tipologia:
Publisher's version/PDF
Dimensione
698.56 kB
Formato
Adobe PDF
|
698.56 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.