Compilers translate programs from a high level of abstraction into a low level representation that can be understood and executed by the computer; interpreters directly execute instructions from source code to convey their semantics. Undoubtedly, the correctness of both compilers and interpreters is fundamental to reliably execute the semantics of any software developed by means of high-level languages. Testing is one of the most important methods to detect errors in any software, including compilers and interpreters. Among testing methods, mutation testing is an empirically effective technique often used to evaluate and improve the quality of test suites. However, mutation testing imposes severe demands in computing resources due to the large number of mutants that need to be generated, compiled and executed. In this work, we introduce a mutation approach for programming languages that mitigates this problem by leveraging the properties of language product lines, language workbenches and separate compilations. In this approach, the base language is taken as a black-box and mutated by means of mutation operators performed at language feature-level to create a family of mutants of the base language. Each variant of the mutant family is created at runtime, without any access to the source code and without performing any additional compilation. We report results from a preliminary case study in which mutants of an ECMAScript interpreter are tested against the Sputnik conformance test suite for the ECMA-262 specification. The experimental data indicates that this approach can be used to create generally non-trivial mutants.

The Language Mutation Problem: Leveraging Language Product Lines for Mutation Testing of Interpreters / W. Cazzola, L. Favalli. - In: THE JOURNAL OF SYSTEMS AND SOFTWARE. - ISSN 0164-1212. - (2022), pp. 1-22. [Epub ahead of print] [10.1016/j.jss.2022.111533]

The Language Mutation Problem: Leveraging Language Product Lines for Mutation Testing of Interpreters

W. Cazzola
Primo
;
L. Favalli
Ultimo
2022

Abstract

Compilers translate programs from a high level of abstraction into a low level representation that can be understood and executed by the computer; interpreters directly execute instructions from source code to convey their semantics. Undoubtedly, the correctness of both compilers and interpreters is fundamental to reliably execute the semantics of any software developed by means of high-level languages. Testing is one of the most important methods to detect errors in any software, including compilers and interpreters. Among testing methods, mutation testing is an empirically effective technique often used to evaluate and improve the quality of test suites. However, mutation testing imposes severe demands in computing resources due to the large number of mutants that need to be generated, compiled and executed. In this work, we introduce a mutation approach for programming languages that mitigates this problem by leveraging the properties of language product lines, language workbenches and separate compilations. In this approach, the base language is taken as a black-box and mutated by means of mutation operators performed at language feature-level to create a family of mutants of the base language. Each variant of the mutant family is created at runtime, without any access to the source code and without performing any additional compilation. We report results from a preliminary case study in which mutants of an ECMAScript interpreter are tested against the Sputnik conformance test suite for the ECMA-262 specification. The experimental data indicates that this approach can be used to create generally non-trivial mutants.
Language Product Lines; Mutation Testing; Language Testing;
Settore INF/01 - Informatica
PRIN202022WCAZZ_01 - Typeful Language Adaptation for Dynamic, Interacting and Evolving Systems - CAZZOLA, WALTER - PRIN2020 - PRIN bando 2020 - 2022
Article (author)
File in questo prodotto:
File Dimensione Formato  
jss22-third.pdf

embargo fino al 12/10/2024

Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione 745.25 kB
Formato Adobe PDF
745.25 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
1-s2.0-S0164121222002096-main.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 2.71 MB
Formato Adobe PDF
2.71 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/939206
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact