NON-FUNCTIONAL CERTIFICATION OF MODERN DISTRIBUTED SYSTEMS

The ICT landscape has been deeply revolutionized in the last two decades, changing the way in which distributed systems are designed, developed, and operated. From the launch of services at the beginning of the 2000s, systems moved from monolithic and static code bases to elastic and multi-layer distributed systems, where each layer transparently provides functionalities to the upper layers. Each system is in turn a composition of multiple systems operated by different parties and implemented as miniaturized micro- and nano-services, where horizontal aspects are delegated to orchestration and composition engines. In addition, Internet of Things (IoT) and Machine Learning (ML) models are increasingly permeating these systems, providing functionalities to end-users and supporting the service life cycle management. This scenario has been radically changing distributed systems engineering, as well as the Governance, Risk, and Compliance (GRC) landscape, invading the safety, security, and privacy of people. Modern distributed systems in fact virtually support every transaction and process in our everyday life, optimizing existing processes (e.g., in smart grids) and enabling new ones (e.g., remote healthcare). The trustworthiness of distributed systems is thus the prime concern and emerges as an urgent need, looking for solutions to verify whether these systems support a given set of non-functional requirements (non-functional properties in the following). A plethora of assurance techniques have been defined and used to satisfy this need. Among them, certification has been clearly identified by policymakers, regulators, and industrial stakeholders, as the preferred assurance technique. Unfortunately, existing certification schemes are not immediately applicable to modern distributed systems, due to a number of intrinsic mismatches. First, existing certification techniques are limited to the run-time evaluation of the systems to be certified, neglecting relevant aspects that might affect the certification of system non-functional properties, such as the development process (e.g., DevOps practices). Released certificates are thus incomplete and lead to inconsistent certification-based decision-making. Second, the validity of certificates is extremely short: they are invalidated at each system release, asking for a complete system re-certification. The overhead of certification is therefore high and the low quality and usefulness of certificates undermine their practical usage. Finally, the reliance on ML models for system functionalities and life cycle management makes system behavior non-deterministic, impairing existing techniques at large. We reached a standstill: despite the increasing push coming from the society for trustworthy IT built on certification, there is a lack of sound and widely applicable certification schemes for modern systems. In this thesis, we propose a continuous certification framework that overcomes this standstill. Our framework works at the application layer, verifying those service-based applications built as part of a distributed system. It implements a novel certification scheme that releases certificates fully covering the services non-functional properties, keeping the pace with service changes and evolution. The certification scheme is integrated within the service development process to facilitate and reduce the costs of certification. It is finally validated in different real-world use cases proposed by the industry. To prove the framework potential, we also refined and applied it to the certification of ML-based applications. Our contribution is manifold. First, we design and implement a novel, multi-dimensional certification scheme that enlarges the typical scope of certification-based evaluation. It releases enhanced certificates by taking into account a number of aspects (e.g., software artifacts, development process) influencing the services non-functional properties. We compare the quality of the scheme when applied in the context of system life cycle management with respect to the state of the art. Then, we extend our scheme to implement a continuous certification process that collects and analyzes relevant service changes, to select the minimum set of activities that keep certificates up to date across these changes. Current certification schemes are in fact of little practical usage, because they statically award certificates according to a one-time process; certificates are never updated according to the service life cycle, or, worse, fully re-constructed at any minor service updates. Moreover, we propose a methodology that integrates our continuous certification scheme within a unified development process that takes certification requirements into account. This approach supports the development team in the implementation of a service that is ready to be certified, that is, it can be certified at low cost and with maximum quality. Our methodology departs from the assumption of considering certification and development processes as two disconnected, sequential activities, making certification applicable in the real world. To validate our framework and show the enhancements it brings to the overall system quality, we consider three application scenarios proposed by the industry: CaixaBank, a major financial company worldwide, Cargill, a major food supply chain worldwide, and SAP, a major software company in Europe. Finally, certification of modern systems cannot leave the impact of ML on services aside. To this aim, we propose a preliminary adaptation of our scheme for the certification of ML-based applications, proving that is general enough for this purpose. We formalize this adaptation and validate it in a real-world scenario focused on non-functional property robustness.