COCOS: A scalable architecture for containerized heterogeneous systems

Nowadays software systems are organized around several and heterogeneous components. For example, a modern application can be composed of different microservices, along with dedicated components for machine learning analytics and recurring batch processing jobs. While containers offer a means to deploy the system and tackle heterogeneity, these components have different execution models, can exploit different resource types (e.g., CPUs and GPUs) and result in completely different execution times (milliseconds vs hours). This complexity calls for a new, scalable architecture to allow the systems to operate efficiently. This paper presents COCOS, an architecture, based on containers and control-theory, that is able to manage large and heterogeneous software systems. The architecture is based on a three-level hierarchy of controllers that cooperatively enforce user-defined requirements on execution times and consumed resources. The paper also shows a prototype implementation of COCOS based on Kubernetes, a well-known container orchestrator. The evaluation shows the efficiency of COCOS when dealing with microservices, Spark jobs and machine learning applications.