Modern software systems are required to operate in a highly uncertain and changing environment. They have to control the satisfaction of their requirements at run-time, and possibly adapt and cope with situations that have not been completely addressed at design-time. Software engineering methods and techniques are, more than ever, forced to deal with change and uncertainty (lack of knowledge) explicitly. For tackling the challenge posed by uncertainty in delivering more reliable systems, this paper proposes a novel online Model-based Testing technique that complements classic test case generation based on pseudo-random sampling strategies with an uncertainty-aware sampling strategy. To deal with system uncertainty during testing, the proposed strategy builds on an Inverse Uncertainty Quantification approach that is related to the discrepancy between the measured data at run-time (while the system executes) and a Markov Decision Process model describing the behavior of the system under test. To this purpose, a conformance game approach is adopted in which tests feed a Bayesian inference calibrator that continuously learns from test data to tune the system model and the system itself. A comparative evaluation between the proposed uncertainty-aware sampling policy and classical pseudo-random sampling policies is also presented using the Tele Assistance System running example, showing the differences in achieved accuracy and efficiency.
Online Model-Based Testing under Uncertainty / M. Camilli, C. Bellettini, A. Gargantini, P. Scandurra (PROCEEDINGS-INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING). - In: 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE) / [a cura di] S. Ghosh, R. Natella, B. Cukic, R. Poston, N. Laranjeiro. - [s.l] : IEEE, 2018 Oct. - ISBN 9781538683217. - pp. 36-46 (( Intervento presentato al 29. convegno International Symposium on Software Reliability Engineering (ISSRE) tenutosi a Memphis nel 2018 [10.1109/ISSRE.2018.00015].
Online Model-Based Testing under Uncertainty
M. Camilli
;C. Bellettini;P. Scandurra
2018
Abstract
Modern software systems are required to operate in a highly uncertain and changing environment. They have to control the satisfaction of their requirements at run-time, and possibly adapt and cope with situations that have not been completely addressed at design-time. Software engineering methods and techniques are, more than ever, forced to deal with change and uncertainty (lack of knowledge) explicitly. For tackling the challenge posed by uncertainty in delivering more reliable systems, this paper proposes a novel online Model-based Testing technique that complements classic test case generation based on pseudo-random sampling strategies with an uncertainty-aware sampling strategy. To deal with system uncertainty during testing, the proposed strategy builds on an Inverse Uncertainty Quantification approach that is related to the discrepancy between the measured data at run-time (while the system executes) and a Markov Decision Process model describing the behavior of the system under test. To this purpose, a conformance game approach is adopted in which tests feed a Bayesian inference calibrator that continuously learns from test data to tune the system model and the system itself. A comparative evaluation between the proposed uncertainty-aware sampling policy and classical pseudo-random sampling policies is also presented using the Tele Assistance System running example, showing the differences in achieved accuracy and efficiency.File | Dimensione | Formato | |
---|---|---|---|
issre2018_camera-ready.pdf
accesso aperto
Tipologia:
Pre-print (manoscritto inviato all'editore)
Dimensione
1.39 MB
Formato
Adobe PDF
|
1.39 MB | Adobe PDF | Visualizza/Apri |
08539067.pdf
accesso riservato
Tipologia:
Publisher's version/PDF
Dimensione
647.81 kB
Formato
Adobe PDF
|
647.81 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.