Assessing the Legality of Using the Category of Race and Ethnicity in Clinical Algorithms - the EU Anti-discrimination Law Perspective

The growing use of machine learning (ML) in medical prognostics, diagnostics, and treatment recommendations offers new powerful tools for addressing pressing health challenges. However, similarly to their knowledge-based predecessors, ML algorithms are not neutral and can mirror prevailing patterns of power and disadvantage in healthcare, entrenching racial disparities. The developers of clinical algorithms often face a paradox. Ignoring race can introduce racial bias through proxies. However, explicitly taking race into account can also replicate harmful stereotypes embedded in the category of race. This paper explores the problem of race and ethnicity in clinical algorithms from the European legal perspective, offering three main contributions. Firstly, it tackles the importance of the correct operationalization of race, arguing that since race is a social construct, developers of clinical algorithms should pay particular attention to the dimension of race that the data represents. In doing so, the paper addresses the challenges to race operationalization in the European context. Secondly, the paper analyses how race is used in the design of clinical algorithms, with a particular focus on fairness measures that take race into account at the prediction time, including race correction. Thirdly, the paper explores the legality of such measures under the Racial Equality Directive.