Enhancing Asynchronous Code Review in Programming Education: A Systematic Literature Review of Visual-Contextual Feedback Systems

Muhammad Rizqi Jamhari; Agus Juhana

doi:10.24036/jtip.v19i1.1091

Authors

Muhammad Rizqi Jamhari Universitas Pendidikan Indonesia
Agus Juhana Universitas Pendidikan Indonesia

DOI:

https://doi.org/10.24036/jtip.v19i1.1091

Keywords:

SLR, Code Review, Cognitive Load, Programing education, PRISMA

Abstract

Asynchronous code review in programming education frequently suffers from disorganized, text-heavy feedback that increases learners' extraneous cognitive load and complicates error identification. Despite the growing adoption of visual and contextual annotation tools, the literature remains fragmented, and no systematic synthesis has established their comparative effectiveness. Following PRISMA 2020 guidelines, we searched five databases (Scopus, Web of Science, IEEE Xplore, ERIC, and ACM Digital Library) in September 2025, yielding 987 initial records. After rigorous screening, 35 peer-reviewed studies published between 2015 and 2025 were included to evaluate the impact of these tools on learner comprehension, workflow efficiency, and instructor workload. The synthesized empirical data indicate that situating visual feedback adjacent to code significantly reduces extraneous cognitive load measured via validated instruments like NASA-TLX and accelerates syntax error remediation (RQ1, RQ2). Furthermore, automated visual grading frameworks reduce instructor evaluation latency by 12% to 58% in large cohorts while maintaining a constant grading load, though initial configuration time and a lack of pedagogical depth for nuanced design choices remain primary constraints (RQ3). These findings suggest that educators should adopt hybrid feedback architectures, pairing automated visual triage with human oversight to maximize both operational efficiency and pedagogical depth.

References

M.-C. Wu, M.-C. Hung, M.-L. Chiu, C.-H. Chuang, and Y.-B. Lin, “Impact of Visual Cues and Feedback on Learning Performance: Different Learning Styles and Cognitive Loads,” I Int. Res. J. Eng. Sci., vol. 8, no. 5, pp. 1–6, 2022, doi: 10.14303/2315-5663.2022.81.

C. H. Lai, P. W. Lin, and S. H. You, “Development and evaluation of a dynamic code visualization system for C programming education: The PVLS approach,” Soc. Sci. Humanit. Open, vol. 12, no. July, p. 101962, 2025, doi: 10.1016/j.ssaho.2025.101962.

M. Lu and Z. Hu, “Leveraging Multimodal Information for Web Front-End Development Instruction: Analyzing Effects on Cognitive Behavior, Interaction, and Persistent Learning,” Inf., vol. 16, no. 9, Sep. 2025, doi: 10.3390/info16090734.

M. B. Garcia and A. M. F. Yousef, “Cognitive and affective effects of teachers’ annotations and talking heads on asynchronous video lectures in a web development course,” Res. Pract. Technol. Enhanc. Learn., vol. 18, no. January, 2023, doi: 10.58459/rptel.2023.18020.

X. Du, M. Dai, H. Tang, J. L. Hung, H. Li, and J. Zheng, “A multimodal analysis of college students’ collaborative problem solving in virtual experimentation activities: a perspective of cognitive load,” J. Comput. High. Educ., vol. 35, no. 2, pp. 272–295, 2023, doi: 10.1007/s12528-022-09311-8.

H. Gabbay and A. Cohen, “Investigating the effect of automated feedback on learning behavior in MOOCs for programming,” Proc. 15th Int. Conf. Educ. Data Mining, EDM 2022, 2022, doi: 10.5281/zenodo.6853125.

E. L. Glassman, J. Scott, R. Singh, P. J. Guo, and R. C. Miller, “OverCode: Visualizing variation in student solutions to programming problems at scale,” ACM Trans. Comput. Interact., vol. 22, no. 2, Mar. 2015, doi: 10.1145/2699751.

M. Heintz and E. L. C. Law, “PDOtcapturer and PDOTanalyser: Participatory design online tools for asynchronous idea capturing and analysis,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Sep. 2018, pp. 622–633. doi: 10.1145/3240167.3240214.

A. Bosu, M. Greiler, and C. Bird, “Characteristics of useful code reviews: An empirical study at Microsoft,” in IEEE International Working Conference on Mining Software Repositories, IEEE Computer Society, Aug. 2015, pp. 146–156. doi: 10.1109/MSR.2015.21.

L. De Oliveira Brandão, Y. Bosse, and M. A. Gerosa, “Visual programming and automatic evaluation of exercises: An experience with a STEM course,” Proc. - Front. Educ. Conf. FIE, vol. 2016-Novem, 2016, doi: 10.1109/FIE.2016.7757621.

F. Y. Chung, “Programming Learning Platform with Visual Aids,” J. Int. Conf. Proc., vol. 8, no. 1, pp. 378–388, 2025, doi: 10.32535/jicp.v8i1.4002.

K. Kuosa et al., “Interactive visualization tools to improve learning and teaching in online learning environments,” Int. J. Distance Educ. Technol., vol. 14, no. 1, pp. 1–21, 2016, doi: 10.4018/IJDET.2016010101.

J. H. Moon, S. Nyu, F. Q. China, S. S. Tian, and J. Hargis, “Teaching and Learning Creative Coding With Conversational AI,” Glob. Lokal Distance Educ. GLOKALde, vol. 10, no. August, p. 29, 2024.

C. H. Tseng, H. C. K. Lin, A. C. W. Huang, and J. R. Lin, “Investigating the effects of PuppyCodeReview, an AI-based code review system, on students’ cognitive load,” SoftwareX, vol. 31, no. July, p. 102283, 2025, doi: 10.1016/j.softx.2025.102283.

X. Liu, S. Wang, P. Wang, and D. Wu, “Automatic grading of programming assignments: An approach based on formal semantics,” Proc. - 2019 IEEE/ACM 41st Int. Conf. Softw. Eng. Softw. Eng. Educ. Training, ICSE-SEET 2019, pp. 126–137, 2019, doi: 10.1109/ICSE-SEET.2019.00022.

X. Song, S. C. Goldstein, and M. Sakr, “Using Peer Code Review as an Educational Tool,” Annu. Conf. Innov. Technol. Comput. Sci. Educ. ITiCSE, vol. 2019, pp. 173–179, 2020, doi: 10.1145/3341525.3387370.

R. A. Rasheed, A. Kamsin, and N. A. Abdullah, “An approach for scaffolding students peer-learning self-regulation strategy in the online component of blended learning,” IEEE Access, vol. 9, pp. 30721–30738, 2021, doi: 10.1109/ACCESS.2021.3059916.

M. Grossman, M. Aziz, H. Chi, A. Tibrewal, S. Imam, and V. Sarkar, “Pedagogy and tools for teaching parallel computing at the sophomore undergraduate level,” J. Parallel Distrib. Comput., vol. 105, pp. 18–30, 2017, doi: 10.1016/j.jpdc.2016.12.026.

S. Palahan, “PythonPal: Enhancing Online Programming Education Through Chatbot-Driven Personalized Feedback,” IEEE Trans. Learn. Technol., vol. 18, pp. 335–350, 2025, doi: 10.1109/TLT.2025.3545084.

F. H. Wang, “Efficient generation of text feedback in object-oriented programming education using cached performer revision,” Mach. Learn. with Appl., vol. 13, no. June, p. 100481, 2023, doi: 10.1016/j.mlwa.2023.100481.

M. Goodwin and T. Drange, “Teaching programming to large student groups through test driven development comparing established methods with teaching based on test driven development,” CSEDU 2016 - Proc. 8th Int. Conf. Comput. Support. Educ., vol. 1, no. 281, pp. 281–288, 2016, doi: 10.5220/0005789502810288.

H. M. Qadir, R. A. Khan, M. Rasool, M. Sohaib, M. A. Shah, and M. J. Hasan, “An adaptive feedback system for the improvement of learners,” Sci. Rep., vol. 15, no. 1, pp. 1–13, 2025, doi: 10.1038/s41598-025-01429-w.

C. H. Ho, H. Q. Zhang, J. Li, and M. Q. Zhang, “Development and Application of Interactive Teaching Systems for Online Design Courses,” Int. J. Distance Educ. Technol., vol. 21, no. 2, pp. 1–28, 2023, doi: 10.4018/ijdet.317365.

T. Faas, L. Dombrowski, A. Young, and A. D. Miller, “Watch me code: Programming mentorship communities on Twitch.tv,” Proc. ACM Human-Computer Interact., vol. 2, no. CSCW, Nov. 2018, doi: 10.1145/3274319.

A. Alam, I. Astuti, and D. Suratman, “Development of Web Programming Interactive Learning Multimedia in Vocational Middle School,” JTP - J. Teknol. Pendidik., vol. 24, no. 1, pp. 50–62, 2022, doi: 10.21009/jtp.v24i1.24242.

M. Khalid, A. Akanmu, A. Afolabi, H. Murzi, I. Awolusi, and P. Agee, “Design And Usability Evaluation Of An End-User Programming Environment For Equipping Construction Students With Sensor Data Analytics Skills,” J. Inf. Technol. Constr., vol. 30, pp. 213–242, 2025, doi: 10.36680/j.itcon.2025.010.

Ş. H. Aksu, A. Adem, E. Çakıt, M. Dağdeviren, and W. Karwowski, “An examination of the interrelationships among NASA-TLX dimensions utilizing the DEMATEL method,” PLoS One, vol. 20, no. 4 April, pp. 1–19, 2025, doi: 10.1371/journal.pone.0320638.

C. Douce, “Teaching web technologies: understanding the tutor’s perspective,” Open Learn., vol. 34, no. 1, pp. 78–88, Jan. 2019, doi: 10.1080/02680513.2018.1483226.

M. Rogti, “The Effect of Mobile-based Interactive Multimedia on Thinking Engagement and Cooperation,” Int. J. Instr., vol. 17, no. 1, pp. 673–696, Jan. 2024, doi: 10.29333/iji.2024.17135a.

L. Nagel, O. Karras, S. M. Amiri, and K. Schneider, “Turning asynchronicity into an opportunity: asynchronous communication for shared understanding with vision videos,” Requir. Eng., vol. 29, no. 1, pp. 49–71, Mar. 2024, doi: 10.1007/s00766-024-00414-5.

L. Dong-Kyu, “A GPT-based Code Review System for Programming Language Learning,” 2024, [Online]. Available: http://arxiv.org/abs/2407.04722

G. Haldeman, M. Babeş-Vroman, A. Tjang, and T. D. Nguyen, “CSF: Formative Feedback in Autograding,” ACM Trans. Comput. Educ., vol. 21, no. 3, pp. 1–30, 2021, doi: 10.1145/3445983.

S. Gunawardena, E. Tempero, and K. Blincoe, “Concerns identified in code review: A fine-grained, faceted classification,” Inf. Softw. Technol., vol. 153, no. August 2022, p. 107054, 2023, doi: 10.1016/j.infsof.2022.107054.

M. Hull, C. Guerin, J. Chen, S. Routray, and D. H. Chau, “Towards Automatic Grading of D3.js Visualizations,” Oct. 2021, [Online]. Available: http://arxiv.org/abs/2110.11227

C. Garcia and N. Lemos, “International Journal On Informatics Visualization The Gamification of E-learning Environments for Learning Programming,” vol. 7, no. June, pp. 455–462, 2023, [Online]. Available: www.joiv.org/index.php/joiv