A Framework for Data Quality and Protection Management in Service-Based Data Pipelines

We present a framework that implements a service selection process tailored for the definition of service-based data pipelines. The framework addresses the critical challenge of balancing data quality and data protection in distributed, service-based data pipelines, an issue that existing solutions overlook by treating these dimensions independently. By modeling the pipeline as a Directed Acyclic Graph (DAG), the framework extends pipelines with functional and data protection requirements. An extensive experimental evaluation measures the performance of our framework by analyzing variations in data quality across diverse datasets and configurations.