FOREX Press I. J. of Electrical & Electronics Research
Support Open Access
  • Home/
  • IJEER-130424

Research Article |

A Deep Learning-Based Approach for Heart Rate Monitoring through Combined Convolutional and Generative Networks Using Facial Videos

Author(s): Jyostna J1*, and Satyanarayana Penke2

Published In : International Journal of Electrical and Electronics Research (IJEER)        Volume 13, Issue 4

Publisher : FOREX Publication

Published : 25 December 2025

e-ISSN : 2347-470X

Page(s) : 837-851

DOI: https://doi.org/10.37391/IJEER.130424

Abstract

This research presents a deep learning-based architecture that uses facial video-extracted remote Photoplethysmography (rPPG) to non-invasively estimate heart rates. The proposed system addresses limitations in signal fidelity and scalability by integrating a Conditional Generative Adversarial Network (CGAN) to enhance the quality of raw rPPG waveforms and a 1D Convolutional Neural Network (CNN) for regression-based prediction of heart rate in beats per minute (BPM). Unlike traditional single-stream models, our framework supports concurrent processing of facial video streams, improving computational efficiency and applicability in real-time, multi-subject environments. Video data is pre-processed through facial Region of Interest (ROI) detection, spatial averaging in alternative colour spaces (YUV/LAB), and temporal filtering before being subjected to CGAN-driven denoising. A mean absolute error (MAE) of 2.3 BPM, accuracy of 95% and a Pearson Correlation Coefficient (PCC) of 0.92 versus reference signals were achieved by the CNN regressor when trained on enhanced signals according to the UBFC-rPPG dataset. Experimental results demonstrate the robustness of the developed model to lighting variation, head motion, and skin tone diversity. The proposed pipeline is well-suited for applications in telemedicine, contactless fitness monitoring, and smart surveillance systems requiring real-time physiological assessment. Real-time video streams have been used to test the suggested model, which shows little variation between the ground truth and the actual heart rate values. This low prediction error demonstrates the model's resilience and appropriateness for applications involving real-time physiological monitoring.

Keywords: Remote Photoplethysmography (rPPG), Conditional Generative Adversarial Network (CGAN), Video Processing, Convolutional Neural Networks.




Jyostna J*, Research Scholar, Department of Electronics and Communication Engineering, Koneru Lakshmaiah Education Foundation, Green Fields, Vaddeswaram, Andhra Pradesh-522502, India; Department of Electronics and Communication Engineering, CVR College of Engineering, Vastunagar, Ibrahimpatnam (M), Hyderabad, Telangana-501510, India; Email: jyostnajulakanti@gmail.com

Satyanarayana Penke, Department of Electronics and Communication Engineering, Koneru Lakshmaiah Education Foundation, Green Fields, Vaddeswaram, Andhra Pradesh-522502, India

    [1] Dotsinsky, “Clifford Gari D, Azuaje Francisco, McSharry Patrick E, Eds: Advanced methods and tools for ECG analysis,” Biomedical Engineering Online, vol. 6, pp. 1–3, 2007.10.1186/1475-925X-6-18.
    [2] J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement, vol. 28, pp. R1–R39, 2007.10.1088/0967-3334/28/3/R01.
    [3] A. Jubran, “Pulse oximetry,” Critical Care, vol. 19, p. 272, 2015.https://doi.org/10.1186/s13054-015-0984-8.
    [4] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Optics Express, vol. 16, no. 26, pp. 21434–21445, 2008. doi: 10.1364/oe.16.021434. PMID: 19104573; PMCID: PMC2717852.
    [5] M.-Z. Poh, D. McDuff, and R. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Optics Express, vol. 18, pp. 10762–10774, 2010.10.1364/OE.18.010762.
    [6] D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five-band digital camera,” IEEE Transactions on Biomedical Engineering, vol. 61, no. 10, pp. 2593–2601, 2014.
    [7] X. Niu, X. Zhao, H. Han, A. Das, A. Dantcheva, S. Shan, and X. Chen, “Robust remote heart rate estimation from face utilizing spatial-temporal attention,” in Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France, 2019, pp. 1–8. 10.1109/FG.2019.8756554.
    [8] W. Chen and D. McDuff, “DeepPhys: Video-based physiological measurement using convolutional attention networks,” in Proc. European Conf. Computer Vision (ECCV), 2018, pp. 356–373.10.48550/arXiv.1805.07888.
    [9] I. Goodfellow et al., “Generative adversarial networks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 3, 2014.10.1145/3422622.
    [10] UBFC-rPPG Dataset. [Online]Available: https://www.ubfc.fr/dataset/ubfc-rppg.
    [11] J. J. and S. Penke, “An efficient approach to estimating heart rate from facial videos with accurate region of interest,” in Proc. 2024 3rd Int. Conf. Innovation in Technology (INOCON), Bangalore, India, 2024, pp. 1–7.doi:10.1109/INOCON60754.2024.10511840.
    [12] M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, pp. 7–11, 2011.doi: 10.1109/TBME.2010.2086456.
    [13] S. Tulyakov, X. Alameda-Pineda, E. Ricci, L. Yin, J. F. Cohn, and N. Sebe, “Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions,” in Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 2396–2404. 10.1109/CVPR.2016.263.
    [14] G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878–2886, Oct. 2013.doi: 10.1109/TBME.2013.2266196. Epub 2013 Jun 4. PMID: 23744659.
    [15] X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2014, pp. 4264–4271.10.1109/CVPR.2014.543.
    [16] Z. Li, K. Wang, H. Xiao, X. Liu, F. Zhou, J. Jiang, and T. Liu, “Exploring remote physiological signal measurement under dynamic lighting conditions at night: dataset, experiment, and analysis,”CoRR, abs/2507.04306, 2025.10.48550/arXiv.2507.04306.
    [17] D. McDuff, S. Gontarek, and R. Picard, “Remote measurement of cognitive stress via heart rate variability,” in Proc. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Chicago, IL, USA, 2014, pp. 2957–2960.10.1109/EMBC.2014.6944243.
    [18] H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. Freeman, “Eulerian video magnification for revealing subtle changes in the world,” ACM Transactions on Graphics (TOG), vol. 31, no. 4, pp. 1–8, 2012.10.1145/2185520.2185561.
    [19] S. Moscato, L. Palmerini, P. Palumbo, and L. Chiari, “Quality assessment and morphological analysis of photoplethysmography in daily life,” Frontiers in Digital Health, 2022.doi: 10.3389/fdgth.2022.912353. PMID: 35873348; PMCID: PMC9300860
    [20] X. Liu, J. Fromm, S. Patel, and D. McDuff, “Multi Task Temporal Shift Attention Networks for on device contactless vitals measurement,” in Advances in Neural Information Processing Systems (NeurIPS), vol.33, 2020, pp. 19400–19411. 10.48550/arXiv.2006.03790.
    [21] R. Špetlík, V. Franc, J. Čech, and J. Matas, “Visual heart rate estimation with convolutional neural network,” in Proc. British Machine Vision Conference (BMVC), 2018.
    [22] Z. Yu, W. Peng, X. Li, X. Hong, and G. Zhao, “Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement,” in Proc. IEEE International Conference on Computer Vision (ICCV), Seoul, Korea, 2019, pp. 1095–1104.
    [23] R. Stricker et al., “non-contact video-based pulse rate measurement on a mobile service robot,” in Proc. 23rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2014, pp. 1040–1045.
    [24] J. Przybyło, “A deep learning approach for remote heart rate estimation,” Biomedical Signal Processing and Control, vol. 74, 2022, Art. no. 103493.ISSN 1746-8094, https://doi.org/10.1016/j.bspc.2021.103457
    [25] M. Bian, B. Peng, W. Wang, and J. Dong, “An accurate LSTM based video heart rate estimation method,” in Proc. Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Xi’an, China, Nov. 2019, pp. 409–417. (volume 11859 LNCS)10.1007/978-3-030-31726-3_35.
    [26] J, Jyostna & P, Satyanarayana. (2025). A Feasible Heartbeat Rate Monitoring Model from Facial Videos Using Weighted Feature Fusion‐Based Adaptive Long Short‐Term Memory with Attention Mechanism. Computational Intelligence. 41. 10.1111/coin.70137
    [27] Y. Nam, J. Lee, J. Lee, H. Lee, D. Kwon, M. Yeo, S. Kim, R. Sohn, and C. Park, “Designing a remote photoplethysmography-based heart rate estimation algorithm during a treadmill exercise,” Electronics, vol. 14, no. 5, p. 890, 2025https://doi.org/10.3390/electronics14050890
    [28] J. J., B. S. Reddy, A. S. Venkateswarlu, and B. C. K. Reddy, “Deep learning for image upscaling: Exploring the potential of ESRGAN,” in Proc. 2024 3rd International Conference for Innovation in Technology (INOCON), Bangalore, India, 2024, pp. 1–7.doi: 10.1109/INOCON60754.2024.10511428.
    [29] P. Viola and M. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, pp. 137–154, 2004.10.1023/B%3AVISI. 0000013087.49260.fb.
    [30] G. Balakrishnan, F. Durand, and J. Guttag, “Detecting pulse from head motions in video,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013, pp. 3430–3437.doi:10.1109/CVPR.2013.440
    [31] W. Wang, A. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479–1491, Jul. 2017.10.1109/TBME.2016.2609282.
    [32] F. Chollet, Deep Learning with Python. Manning, 2017.
    [33] K. B. Jaiswal and T. Meenpal, “rPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion,” Biomedical Signal Processing and Control, vol. 78, 2022, Art. no. 104002.ISSN 1746-8094, https://doi.org/10.1016/j.bspc.2022.104002.
    [34] Z. Yu, X. Li, and G. Zhao, “Remote photoplethysmography signal measurement from facial videos using spatio-temporal networks,” in Proc. British Machine Vision Conference (BMVC), 2019.
    [35] X. Niu, H. Han, S. Shan, and X. Chen, “RhythmNet: end-to-end heart rate estimation from face via spatial-temporal representation,” in Proc. 2019 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Seoul, Korea, 2019, 10.48550/arXiv.1910.11515.
    [36] J. Sturekova, P. Kamencay, P. Sykora, and R. Hlavatá, “A comparison of convolutional neural network transfer learning regression models for remote photoplethysmography signal estimation,” AI, vol. 6, no. 2, p. 24, 2025.10.3390/ai6020024.

Jyostna J, Satyanarayana Penke (2025), A Deep Learning-Based Approach for Heart Rate Monitoring through Combined Convolutional and Generative Networks Using Facial Videos. IJEER 13(4), 837-851. DOI: 10.37391/IJEER.130424.