Effectiveness of AI-Based Chatbots as Virtual Tutors in Basic Programming Education

Arif Setiawan; Muhammad Fikri Irfanuddin; Ryan Rizki Adhisa

doi:10.24036/jtip.v17i2.853

Arif Setiawan Universitas Muhammadiyah Surakarta
Muhammad Fikri Irfanuddin Universitas Muhammadiyah Surakarta
Ryan Rizki Adhisa Universitas Muhammadiyah Surakarta

DOI: https://doi.org/10.24036/jtip.v17i2.853

Keywords: Chatbot AI, Generative AI, finetunning, Basic Programming Education

Abstract

The integration of AI-based chatbots as virtual tutors in basic programming education addresses the challenges posed by the diverse backgrounds of students and the complexity of the course material. This integration aims to provide personalized assistance and explanations tailored to individual student needs and understanding levels. The research methodology employed in this study adopts a Research and Design (RnD) approach, utilizing the 4D development model, which encompasses the stages of Definition, Design, Development, and Deployment. Through this methodology, the study aims to develop and evaluate the effectiveness of the chatbot in enhancing students' learning experiences. The results of the study indicate a significant increase in students' self-learning activities, as evidenced by a score of 79.4% in the "Very Good" category in the Likert scale evaluation. This suggests that the integration of AI technology in basic programming education holds great potential for enhancing accessibility and effectiveness in learning, while also enriching students' learning experiences by providing adaptive and personalized learning resources. Overall, the findings underscore the importance of leveraging AI technology to address the challenges faced in programming education, thereby fostering a more inclusive and effective learning environment

References

C. K. Y. Chan, “A comprehensive AI policy education framework for university teaching and learning,” Int J Educ Technol High Educ, vol. 20, no. 1, p. 38, Jul. 2023, doi: 10.1186/s41239-023-00408-3.

F. Yustiasari Liriwati, “Transformasi Kurikulum; Kecerdasan Buatan untuk Membangun Pendidikan yang Relevan di Masa Depan,” J.Ihsan, vol. 1, no. 2, pp. 62–71, Jul. 2023, doi: 10.61104/ihsan.v1i2.61.

A. D. O. Silva and D. D. S. Janes, “Exploring the Role of Artificial Intelligence in Education: A Comprehensive Perspective,” Rev. Artif. Intell. Educ, vol. 1, no. 00, p. e05, Dec. 2020, doi: 10.37497/rev.artif.intell.education.v1i00.5.

Josephat O Oroma, H. Wanga, and F. Ngumbuke, “CHALLENGES OF TEACHING AND LEARNING COMPUTER PROGRAMMING IN DEVELOPING COUNTRIES: LESSONS FROM TUMAINI UNIVERSITY,” 2012, doi: 10.13140/2.1.3836.6407.

J. Bennedsen and M. E. Caspersen, “Failure rates in introductory programming,” SIGCSE Bull., vol. 39, no. 2, pp. 32–36, Jun. 2007, doi: 10.1145/1272848.1272879.

M. McCracken et al., “A multi-national, multi-institutional study of assessment of programming skills of first-year CS students,” SIGCSE Bull., vol. 33, no. 4, pp. 125–180, Dec. 2001, doi: 10.1145/572139.572181.

F. French, D. Levi, C. Maczo, A. Simonaityte, S. Triantafyllidis, and G. Varda, “Creative Use of OpenAI in Education: Case Studies from Game Development,” MTI, vol. 7, no. 8, p. 81, Aug. 2023, doi: 10.3390/mti7080081.

J. Huang, S. Saleh, and Y. Liu, “A Review on Artificial Intelligence in Education,” Acad. J. Interdiscip. Stud., vol. 10, no. 3, p. 206, May 2021, doi: 10.36941/ajis-2021-0077.

L. Chen, P. Chen, and Z. Lin, “Artificial intelligence in education: A review,” Ieee Access, vol. 8, pp. 75264–75278, 2020.

C. K. Y. Chan and W. Hu, “Students’ voices on generative AI: perceptions, benefits, and challenges in higher education,” Int J Educ Technol High Educ, vol. 20, no. 1, p. 43, Jul. 2023, doi: 10.1186/s41239-023-00411-8.

M. Tedre et al., “Teaching Machine Learning in K–12 Classroom: Pedagogical and Technological Trajectories for Artificial Intelligence Education,” IEEE Access, vol. 9, pp. 110558–110572, 2021, doi: 10.1109/ACCESS.2021.3097962.

The Korean Society of Culture and Convergence, G. Lee, S. Huh, H. Chee, M. Kim, and M. Kim, “Research on the Development of Teaching and Learning Support Models for the Utilization of AI-based Learning Assistance Systems in Universitie,” Korean Soc Cult Converg, vol. 45, no. 9, pp. 181–191, Sep. 2023, doi: 10.33645/cnc.2023.09.45.09.181.

Y. Zhang, “Development and Application of Artificial Intelligence Multimedia Technology Based on Big Data,” Mobile Information Systems, vol. 2022, pp. 1–10, Jan. 2022, doi: 10.1155/2022/2073091.

Tapabrato Bandyopadhyay, Srinjoy Saha, and Debrupa Pal, “Beyond Imitation: Exploring Novelty in Generative AI,” IJARSCT, pp. 472–475, Sep. 2023, doi: 10.48175/IJARSCT-13070.

E. Chen, R. Huang, H.-S. Chen, Y.-H. Tseng, and L.-Y. Li, “GPTutor: a ChatGPT-powered programming tool for code explanation”.

S. Ruan et al., “BookBuddy: Turning Digital Materials Into Interactive Foreign Language Lessons Through a Voice Chatbot,” in Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale, Chicago IL USA: ACM, Jun. 2019, pp. 1–4. doi: 10.1145/3330430.3333643.

S. H. Mad Daud, “e-JAVA Chatbot for Learning Programming Language: A Post-Pandemic Alternative Virtual Tutor,” IJETER, vol. 8, no. 7, pp. 3290–3298, Jul. 2020, doi: 10.30534/ijeter/2020/67872020.

E. Porter, M. Murphy, and C. O’Connor, “Chat GPT in dermatology: Progressive or problematic?,” Acad Dermatol Venereol, vol. 37, no. 7, Jul. 2023, doi: 10.1111/jdv.19174.

J. N. Kather, N. Ghaffari Laleh, S. Foersch, and D. Truhn, “Medical domain knowledge in domain-agnostic generative AI,” npj Digit. Med., vol. 5, no. 1, p. 90, Jul. 2022, doi: 10.1038/s41746-022-00634-5.

Q. Zhu, X. Zhang, and J. Luo, “Biologically Inspired Design Concept Generation Using Generative Pre-Trained Transformers,” 2022, doi: 10.48550/ARXIV.2212.13196.

“Python-Bot.” Accessed: Oct. 22, 2023. [Online]. Available: https://webchat.snatchbot.me/39031b004171a74999c5474f3f41b0e2cb6302679b02722320581897196c88b1

M. Verleger and J. Pembridge, “A Pilot Study Integrating an AI-driven Chatbot in an Introductory Programming Course,” in 2018 IEEE Frontiers in Education Conference (FIE), San Jose, CA, USA: IEEE, Oct. 2018, pp. 1–4. doi: 10.1109/FIE.2018.8659282.

Sutama, Metode Penelitian Pendidikan. Surakarta: CV. Jasmine, 2019.

H. Rahmayanti, V. Oktaviani, and Y. Syani, “Development of sorting waste game android based for early childhood in environmental education,” J. Phys.: Conf. Ser., vol. 1434, no. 1, p. 012029, Jan. 2020, doi: 10.1088/1742-6596/1434/1/012029.

A. Irawan, M. Lestari, and W. Rahayu, “Konsep Etnomatematika Batik Tradisional Jawa Sebagai Pengembangan Media Pembelajaran Matematika,” Scholaria, vol. 12, no. 1, pp. 39–45, Jan. 2022, doi: 10.24246/j.js.2022.v12.i1.p39-45.

“OpenAI Platform.” Accessed: Oct. 25, 2023. [Online]. Available: https://platform.openai.com

A. Ümmühan and H. ERSOY, “The adaptation of learning motivation in computer programming courses scale into Turkish: the study of validity and reliability,” Journal of Higher Education and Science, vol. 8, no. 1, pp. 073–081, 2018.

X. Deng and Z. Yu, “A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education,” Sustainability, 2023, doi: 10.3390/su15042940.

M. Shilowaras and N. Jusoh, “Implementing Artificial Intelligence Chatbot in Moodle Learning Management System,” Engineering, Agriculture, Science and Technology Journal (EAST-J), 2022, doi: 10.37698/eastj.v1i1.122.