EARLY ASSESSMENT OF SERVICE PERFORMANCE USING SIMULATION

The success of web services is changing the way in which software is designed, developed, and distributed. The increasing diffusion of software in the form services, available as commodities over the Internet, has en- abled business scenarios where processes are implemented by composing loosely-coupled services chosen at runtime. Services are in fact continuously re-designed and incrementally developed, released in heterogeneous and distributed environments, and selected and integrated at runtime within external business processes. In this dynamic context, there is the need of solutions supporting the evaluation of service performance at an early stage of the software development process, or even at design time, to support users in an a priori evaluation of the impact, a given service might have when integrated in their business process. A number of useful performance verification and validation techniques are proposed to test and simulate web services, but they assume the availability of service code or at least of reliable information (e.g., collected by testing) on service behavior. Among these approaches, simulation-based techniques are mostly used to assess the behavior of the service and predict its behavior using historical data. Despite the benefits of such solutions, few proposals have addressed the problem of how service performance can be assessed at design time and how historical data can be replaced by simulation data for performance evaluation at early stage of development cycle. In this thesis, the notion of simulation is fully integrated within early phases of the software development process in order to predict the behavior of services. We propose model-based approaches that rely on the amount of information available for the simulation of the performance of service operations. We distinguish full-knowledge, partial-knowledge and zero-knowledge scenarios. In a full-knowledge scenario, the total execution times for each operation and the internal distributions of delays are known and used for performance evaluation. In a partial-knowledge scenario, partial testing results (i.e., the lower and upper bounds to the operation execution times) are used to simulate a service performance. In the zero-knowledge scenario, no testing results are considered; only simulation results are used for performance evaluation. The main contributions of this thesis can be summarized as follows. Firstly, we proposed a model-based approach that relies on Symbolic Transition System (STS) to describe the web services as finite state automata and evaluate their performance. This model was extended for testing and simulation. The testing model annotates model transitions with performance idioms, which allow to evaluate the behavior of the service. The simulation model extends the standard STS-based model with transition probabilities and delay distributions. This model is used to generate a simulation script that allows to simulate the service behavior. Our methodology used simulation along the design and pre-deployment phases of the web service lifecycle to preliminarily assess web service performance using coarse-grained information on the total execution time of each service operation derived by testing. We used testing results and provided some practical examples to validate our methodology and the quality of the performance measurements computed by simulation considering the full-knowledge and partial-knowledge scenarios. The results obtained showed that our simulation gives accurate estimation of the execution times. Secondly, the thesis proposed an approach that permits service developers and software adopters to evaluate service performance in a zero-knowledge scenario, where testing results and service code are not yet available. Our approach is built on expert knowledge to estimate the execution time of the service operation. It evaluates the complexity of the service operation using the input and output Simple Object Access Protocol (SOAP) messages, and the Web Service Description Language (WSDL) interface of the service. Then, the operation interval of execution times is estimated based on profile tables providing the time overhead needed to parse and build SOAP messages, and the performance inferred from the testing of some reference service operations. Our simulation results showed that our zero-knowledge approach gives an accurate approximation of the interval of execution times when compared with the testing results at the end of the development. Thirdly, the thesis proposed an application of our previous approaches to the definition of a framework that allows to negotiate and monitoring the performance Service Level Agreement (SLA) of the web service based on the simulation data. The solution for SLA monitoring is based on the STS-based model for testing and the solution for SLA negotiation is based on the service model for simulation. This work provides an idea about the SLA of the service in advance and how to handle the violations of the SLA on performance after the service deployment.