Lightweight YOLO Models for Robust Facial Expression Detection

Authors

  • Achmad Indra Aulia Institut Teknologi Sains Bandung
  • Albert Jofrandi Hutapea Institut Teknologi Sains Bandung
  • Amril Mutoi Siregar Institut Teknologi Sains Bandung
  • Surjandy Institut Teknologi Sains Bandung

DOI:

https://doi.org/10.24036/jtip.v19i2.1120

Keywords:

Facial Expression Detection, Data Augmentation, YOLO, Confidence Threshold, Object Detection

Abstract

Facial expression recognition is a fundamental component of artificial intelligence systems, particularly in human–machine interaction. However, achieving robust detection accuracy remains challenging due to variations in lighting, facial orientation, and limited training data diversity. While recent lightweight YOLO architectures—YOLOv8n, YOLOv10n, and YOLO11n—have demonstrated strong performance in general object detection, comparative studies evaluating these models specifically for facial expression detection remain limited. This study addresses this gap by systematically comparing these three nano-variant models on a dataset of 2,000 labeled facial images across four expression categories: flat face, angry, sad, and smile. The dataset was divided into training (70%), validation (20%), and test (10%) subsets. Experiments were conducted under two scenarios—with and without data augmentation—using identical training configurations. Augmentation techniques included mosaic composition, HSV variation, geometric transformations, and flipping. Results show that augmentation improved the F1 score of YOLOv10n from 0.68 to 0.72 and YOLO11n from 0.65 to 0.72, with the latter achieving the highest overall precision of 0.82. YOLOv8n exhibited stable performance with an F1 score of 0.75 under both conditions. Confidence threshold optimization revealed distinct optimal operating points for each model, ranging from 0.1 to 0.6, confirming that per-model threshold tuning is necessary to maximize detection performance. These findings provide practical guidance for selecting and configuring lightweight YOLO models for facial expression detection in resource-constrained environments.

References

S. Li and W. Deng, “Deep Facial Expression Recognition: A Survey,” IEEE Trans. Affect. Comput., vol. 13, no. 3, pp. 1195–1215, Oct. 2018, doi: 10.1109/TAFFC.2020.2981446.

S. Ullah, J. Ou, Y. Xie, and W. Tian, “Facial expression recognition (FER) survey: a vision, architectural elements, and future directions,” PeerJ Comput. Sci., vol. 10, p. e2024, Jun. 2024, doi: 10.7717/PEERJ-CS.2024.

A. Alshammari and M. E. Alshammari, “Emotional Facial Expression Detection using YOLOv8,” Engineering, Technology & Applied Science Research, vol. 14, no. 5, pp. 16619–16623, Oct. 2024, doi: 10.48084/etasr.8433.

P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, “A Review of Yolo Algorithm Developments,” Procedia Comput. Sci., vol. 199, pp. 1066–1073, Jan. 2022, doi: 10.1016/j.procs.2022.01.135.

J. Terven, D. M. Córdova-Esparza, and J. A. Romero-González, “A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS,” Machine Learning and Knowledge Extraction 2023, Vol. 5, Pages 1680-1716, vol. 5, no. 4, pp. 1680–1716, Nov. 2023, doi: 10.3390/make5040083.

T. Diwan, G. Anirudh, and J. V. Tembhurne, “Object detection using YOLO: challenges, architectural successors, datasets and applications,” Multimedia Tools and Applications 2022 82:6, vol. 82, no. 6, pp. 9243–9275, Aug. 2022, doi: 10.1007/s11042-022-13644-y.

M. M. H. Shuvo, S. K. Islam, J. Cheng, and B. I. Morshed, “Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review,” Proceedings of the IEEE, vol. 111, no. 1, pp. 42–91, Jan. 2023, doi: 10.1109/JPROC.2022.3226481.

C.-Y. Wang and H.-Y. M. Liao, “YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems,” APSIPA Trans. Signal Inf. Process., vol. 13, no. 1, pp. 1–38, Aug. 2024, Accessed: Mar. 01, 2026. [Online]. Available: http://arxiv.org/abs/2408.09332

R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” Oct. 2024, [Online]. Available: https://doi.org/10.48550/arXiv.2410.17725

D. Novaliendry, F. Rizal, M. Anwar, and D. Irfan, “A Web Application for Classification and Detection of Tomato Leaf Diseases Using CNN and Yolo Models,” Jurnal Teknologi Informasi dan Pendidikan, vol. 19, no. 1, pp. 1179–1192, Feb. 2026, doi: 10.24036/jtip.v19i1.1073.

M. Karnati, A. Seal, D. Bhattacharjee, A. Yazidi, and O. Krejcar, “Understanding Deep Learning Techniques for Recognition of Human Emotions Using Facial Expressions: A Comprehensive Survey,” IEEE Trans. Instrum. Meas., vol. 72, 2023, doi: 10.1109/TIM.2023.3243661.

C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” Journal of Big Data 2019 6:1, vol. 6, no. 1, pp. 60-, Jul. 2019, doi: 10.1186/s40537-019-0197-0.

E. Randellini, L. Rigutini, and C. Sacca’, “Data Augmentation and Transfer Learning Approaches Applied to Facial Expressions Recognition,” Feb. 2024, doi: 10.5121/csit.2021.111912.

N. E. Khalifa, M. Loey, and S. Mirjalili, “A comprehensive survey of recent trends in deep learning for digital images augmentation,” Artificial Intelligence Review 2021 55:3, vol. 55, no. 3, pp. 2351–2377, Sep. 2021, doi: 10.1007/s10462-021-10066-4.

Rosalinaa, “‘Ekspresi Wajah’ Kaggle Dataset.” [Online]. Available: https://www.kaggle.com/datasets/rosalinaa/ekspresi-wajah/data

A. Mollahosseini, B. Hasani, and M. H. Mahoor, “AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild,” IEEE Trans. Affect. Comput., vol. 10, no. 1, pp. 18–31, Aug. 2017, doi: 10.1109/TAFFC.2017.2740923.

S. Wenkel, K. Alhazmi, T. Liiv, S. Alrshoud, and M. Simon, “Confidence Score: The Forgotten Dimension of Object Detection Performance Evaluation,” Sensors 2021, Vol. 21, Page 4350, vol. 21, no. 13, p. 4350, Jun. 2021, doi: 10.3390/s21134350.

Downloads

Published

2026-04-16

How to Cite

[1]
A. I. Aulia, A. J. Hutapea, A. M. Siregar, and Surjandy, “Lightweight YOLO Models for Robust Facial Expression Detection”, J. teknol. inf. pendidik., vol. 19, no. 2, pp. 1449–1463, Apr. 2026.