Hybrid CNN–transformer model for real-time drowsiness detection using eye aspect ratio & facial landmarks
*Sanjay SharmaCorresponding author33sanjay33@gmail.comDepartment of Computer Science and EngineeringJECRC UniversityJaipur, Rajasthan, 303905, IndiaView full profile → , Shashi Sharmapathak.shashi26@gmail.comDepartment of Computer Science and EngineeringJECRC UniversityJaipur, Rajasthan, 303905, IndiaView full profile →
* Corresponding author · click or hover a name for details
- Received:
- 01 Jan 2026
- Published Online:
- 31 Aug 2026
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- JIOS-2237
- Pages:
- 2955–2968
Abstract
This paper will present a Hybrid CNN Transformer model which is a spatial feature of the face after which temporal dynamics of the behavioral aspects are combined to provide a robust real-time drowsiness detection. Using the Public datasets including YawDD and Kaggle driver drowsiness, the model exploits the Eye Aspect Ratio (EAR) and facial landmarks to recognize the indicators of drowsiness among them protruded closeness, the frequency of blinking, and the slightest muscle alterations. The CNN part is aimed at capturing discriminative spatial features on eye and face regions, and the Transformer part is dedicated to capturing long-range temporal relations across frame sequences, which is a dynamical progression of drowsiness. The preprocessing steps, such as facial landmark normalization, adaptive histogram equalization, and temporal signal smoothing, are aimed at improving the generalization and making the block more noise tolerant. Also, there is fusion of the EAR and mouth aspect data with deep features at the middle level so as to enhance stability of prediction. The experimental scores are verified to perform higher performance than baseline CNN and LSTM models and have high accuracy, precision, and F1-scores on varying lighting and head-pose conditions.
Keywords
Subject Classifications
References
[1] N. N. Alajlan and D. M. Ibrahim, “DDD TinyML: A TinyML-based driver drowsiness detection model using deep learning,” Sensors, vol. 23, no. 12, p. 5696 (2023), doi: 10.3390/s23125696.
[2] J. Chen, S. Wang, E. He, H. Wang, and L. Wang, “Recognizing drowsiness in young men during real driving based on electroencephalography using an end-to-end deep learning approach,” Biomedical Signal Processing and Control, vol. 69, p. 102792 (2021), doi: 10.1016/j.bspc.2021.102792.
[3] V. R. R. Chirra, S. R. Uyyala, and V. K. K. Kolli, “Deep CNN: A machine learning approach for driver drowsiness detection based on eye state,” Revue d’Intelligence Artificielle, vol. 33, no. 6, pp. 461–466 (2019), doi: 10.18280/ria.330609.
[4] S. Das, S. Pratihar, B. Pradhan, R. H. Jhaveri, and F. Benedetto, “IoT-assisted automatic driver drowsiness detection through facial movement analysis using deep learning and a U-Net-based architecture,” Information, vol. 15, no. 1, p. 30 (2024), doi: 10.3390/info15010030.
[5] M. T. A. Dipu, S. S. Hossain, Y. Arafat, and F. B. Rafiq, “Real-time driver drowsiness detection using deep learning,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 7, pp. 844–850 (2021), doi: 10.14569/IJACSA.2021.0120794.
[6] M. Dua, R. Singla, S. Raj, and A. Jangra, “Deep CNN models-based ensemble approach to driver drowsiness detection,” Neural Computing and Applications, vol. 33, no. 8, pp. 3155–3168 (2021), doi: 10.1007/s00521-020-05209-7.
[7] S. A. El-Nabi, W. El-Shafai, E. S. M. El-Rabaie, K. F. Ramadan, F. E. Abd El-Samie, and S. Mohsen, “Machine learning and deep learning techniques for driver fatigue and drowsiness detection: A review,” Multimedia Tools and Applications, vol. 83, no. 3, pp. 9441–9477 (2024), doi: 10.1007/s11042-023-15054-0.
[8] T. Fonseca and S. Ferreira, “Drowsiness detection in drivers: A systematic review of deep learning-based models,” Applied Sciences, vol. 15, no. 16, p. 9018 (2025), doi: 10.3390/app15169018.
[9] O. F. Hassan, A. F. Ibrahim, A. Gomaa, M. A. Makhlouf, and B. Hafiz, “Real-time driver drowsiness detection using transformer architectures: A novel deep learning approach,” Scientific Reports, vol. 15, p. 17493 (2025), doi: 10.1038/s41598-025-02111-x.
[10] S. S. Husain, J. Mir, S. M. Anwar, W. Rafique, and M. O. Ullah, “Development and validation of a deep learning-based algorithm for drowsiness detection in facial photographs,” Multimedia Tools and Applications, vol. 81, no. 15, pp. 20425–20441 (2022), doi: 10.1007/s11042-022-12433-x.
[11] V. Kumar, S. Sharma, and R. Walia, “Driver drowsiness detection using modified deep learning architecture,” Evolutionary Intelligence, vol. 16, no. 6, pp. 1907–1916 (2023), doi: 10.1007/s12065-022-00743-w.
[12] E. Magán, M. P. Sesmero, J. M. Alonso-Weber, and A. Sanchis, “Driver drowsiness detection by applying deep learning techniques to sequences of images,” Applied Sciences, vol. 12, no. 3, p. 1145 (2022), doi: 10.3390/app12031145.
[13] G. M. Mohamed, S. S. Patel, and N. Naicker, “Data augmentation for deep learning algorithms that perform driver drowsiness detection,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 1, pp. 233–248 (2023), doi: 10.14569/IJACSA.2023.0140127.
[14] P. Mukherjee and A. H. Roy, “A novel deep learning-based technique for driver drowsiness detection,” Human Factors and Ergonomics in Manufacturing & Service Industries, vol. 34, no. 6, pp. 667–684 (2024), doi: 10.1002/hfm.21056.
[15] M. Ngxande, J.-R. Tapamo, and M. Burke, “Driver drowsiness detection using behavioral measures and machine learning techniques: A review of state-of-art techniques,” in 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech), Bloemfontein, South Africa, pp. 156–161 (2017), doi: 10.1109/RoboMech.2017.8261140.
[16] P. Panwar, P. Roshan, R. Singh, M. Rai, A. R. Mishra, and S. S. Chauhan, “DDNet—A deep learning approach to detect driver distraction and drowsiness,” Evergreen, vol. 9, no. 3, pp. 881–892 (2022), doi: 10.5109/4843120.
[17] N. N. Pandey and N. B. Muppalaneni, “Temporal and spatial feature based approaches in drowsiness detection using deep learning technique,” Journal of Real-Time Image Processing, vol. 18, no. 6, pp. 2287–2299 (2021), doi: 10.1007/s11554-021-01114-x.
[18] A.-C. Phan, N.-H.-Q. Nguyen, T.-N. Trieu, and T.-C. Phan, “An efficient approach for detecting driver drowsiness based on deep learning,” Applied Sciences, vol. 11, no. 18, p. 8441 (2021), doi: 10.3390/app11188441.
[19] A.-C. Phan, T.-N. Trieu, and T.-C. Phan, “Driver drowsiness detection and smart alerting using deep learning and IoT,” Internet of Things, vol. 22, p. 100705 (2023), doi: 10.1016/j.iot.2023.100705.
[20] D. Goyal, A. Kumar, P. Mathur, F. Sheth, and A. K. Gupta, “Robust cybersecurity through discrete and large language models for effective phishing attack detection,” Journal of Discrete Mathematical Sciences and Cryptography, vol. 28, no. 5-A, pp. 1621–1638 (2025), doi: 10.47974/JDMSC-2162.
[21] A. Noonia, D. Thakral, P. Mathur, F. Sheth, H. Shaikh, and A. K. Gupta, “A discrete mathematical model and cryptography for secure medical image analysis: Encrypted chest X-ray classification,” Journal of Discrete Mathematical Sciences and Cryptography, vol. 28, no. 5-A, pp. 1473–1486 (2025), doi: 10.47974/JDMSC-2146.
[22] D. Goyal, F. Sheth, P. Mathur, and A. K. Gupta, “Discrete mathematical models for enhancing cybersecurity: A mathematical and statistical analysis of machine learning approaches in phishing attack detection,” Journal of Discrete Mathematical Sciences and Cryptography, vol. 27, no. 2-B, pp. 569–599 (2024), doi: 10.47974/JDMSC-1893.
[23] A. Rajkar, N. Kulkarni, and A. Raut, “Driver drowsiness detection using deep learning,” in Applied Information Processing Systems: Proceedings of ICCET 2021, Singapore: Springer, pp. 73–82 (2022), doi: 10.1007/978-981-16-2008-9_7.
[24] M. Ramzan, A. Abid, M. Fayyaz, T. J. Alahmadi, H. Nobanee, and A. Rehman, “A novel hybrid approach for driver drowsiness detection using a custom deep learning model,” IEEE Access, vol. 12, pp. 126866–126884 (2024), doi: 10.1109/ACCESS.2024.3438617.
