Enhancing rPPG pulse-signal recovery by facial sampling and PSD Clustering

In order to increase the accuracy of traditional methods for camera-based pulse rate estimation, we propose a novel systemic approach that extracts multiple remote photoplethysmography (rPPG) signals from a set of scattered facial patches and effectively separates good estimates from noisy ones via a novel unsupervised Power Spectral Density (PSD) clustering method. In contrast to commonly adopted rPPG pipelines, which are often challenged by rigid head movements, facial expressions, and rapidly changing lighting conditions, our patch-oriented solution leverages the key feature of patch recurrence in video sequences. Instead of focusing on a small group of specific Regions of Interest (ROIs), our method adaptively selects a set of patches tracked across successive frames. The spatio-temporal self-similarity among these patches provides powerful internal statistics that significantly enhance standard techniques for rPPG assessment. Our main contribution is a novel unsupervised discriminatory strategy called CIRCLECLUSTERING, which naturally separates PSDs into those with low intra-class variability from those with high intra- and inter-class inhomogeneity. Extensive experimental results demonstrate the overall superiority of our patch-based clustering method compared to both traditional signal processing-based rPPG techniques and recent supervised deep learning-based models for rPPG recovery.