Benchmarking User Experience on Mobile Banking Applications in Indonesia: A Comparative Study Using the User Experience Questionnaire

Kadek Yunita Dewi; I Dewa Ayu Indah Saraswati; Yulia Mariasmi Kiuk; Gede Rasben Dantes; Gede Indrawan; I Made Agus Oka Gunawan

doi:10.24036/jtip.v19i1.1087

Authors

Kadek Yunita Dewi Universitas Pendidikan Ganesha
I Dewa Ayu Indah Saraswati Universitas Pendidikan Ganesha
Yulia Mariasmi Kiuk Universitas Pendidikan Ganesha
Gede Rasben Dantes Universitas Pendidikan Ganesha
Gede Indrawan Universitas Pendidikan Ganesha
I Made Agus Oka Gunawan Politeknik Negeri Bali

DOI:

https://doi.org/10.24036/jtip.v19i1.1087

Keywords:

Benchmarking global, Comparative Study, Mobile Banking Application, System Information, User Experience Questionnaire (UEQ)

Abstract

Intense competition in digital banking services demands the optimization of User Experience (UX); however, the UX quality across mobile banking applications has not yet been comprehensively mapped. This study conducts a comparative analysis of mobile banking applications in Indonesia through a preliminary observation using a purposive sampling technique. The evaluation was carried out using the User Experience Questionnaire (UEQ) to measure six UX dimensions, with reliability testing confirming that the instrument was reliable. The results show all applications achieved positive scores across the six UEQ dimensions. BRImo (BRI) ranked highest overall, particularly in the efficiency (2.225) and dependability (2.188) dimensions, BNI Mobile Banking excelled in the novelty dimension, with the highest score (2.088). In addition, BNI Mobile Banking recorded higher efficiency (2.113) and dependability (1.913) scores compared to Livin’ by Mandiri and BCA Mobile. Meanwhile, BCA Mobile outperformed Livin’ by Mandiri in the perspicuity (1.988) and stimulation (1.925) dimensions, whereas Livin’ by Mandiri only surpassed BNI Mobile Banking and BCA Mobile in the attractiveness dimension (1.900). However, the results of the Kruskal–Wallis test indicate that the differences in scores among the applications were not statistically significant. Based on the global UEQ benchmark, all four applications demonstrate very good UX quality, predominantly in the excellent category. BRImo is the only application that achieved the excellent category across all dimensions, while Livin’ by Mandiri obtained two dimensions in the good category, followed by BNI Mobile Banking and BCA Mobile, each with one dimension in the good category.

References

N. Noerlina, D. C. Freda, G. Mayo, Y. K. Tirtawijaya, and J. Samuel, “The Future Prospect of Mobile Banking Technology Enhancement in Indonesia,” Bus. Econ. Commun. Soc. Sci. J., vol. 5, no. 1, pp. 25–34, January 2023, doi: 10.21512/becossjournal.v5i1.8743.

F. N. Syahrani, A. N. Daulay, and N. Nurbaiti, “The Influence of Self Service Technology (SST) on Customer Satisfaction of Digital Services Users at Indonesia Islamic Bank,” Airlangga J. Innov. Manag., vol. 4, no. 1, pp. 84–99, August 2023, doi: 10.20473/ajim.v4i1.45458.

I. P. Y. Agus Ariwanta, I. M. A. O. Gunawan, and G. Indrawan, “Evaluasi User Experience Sistem Informasi Akademik Pada Website mahasiswa.pkkb.ac.id Menggunakan User Experience Questionnaire (UEQ),” J. Inf. Syst. Res., vol. 5, no. 2, pp. 363–373, Januari 2024, doi: 10.47065/josh.v5i2.4490.

C. Stephanidis dan G. Salvendy, Eds., Designing for Usability, Inclusion and Sustainability in Human-Computer Interaction. Boca Raton, FL, USA: CRC Press, 2025

S. V. B. H. Manurung et al, dalam Pengantar Desain UI/UX, K. N. Panilestari, Ed.,ke- 1. Yogyakarta, Indonesia: PT. Star Digital Publishing, 2025.

M. Schrepp, A. Hinderks, and J. Thomaschewski, “Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S),” Int. J. Interact. Multimed. Artif. Intell., vol. 4, no. 6, p. 103, September 2017, doi: 10.9781/ijimai.2017.09.001.

D. A. Rosandy et al, “Era transformasi digital dalam manajemen,” dalam Transformasi Digital dalam Manajemen, D. A. Rosandy, Ed., ke-1. Bandung, Indonesia: CV. Haura Utama, 2025.

E. M. Afiyah, M. Rizan, and O. Usman, “Analysis of Mobile Banking Usage in Increasing Customer Trust and Loyalty,” J. Econ. Manag. Trade, vol. 31, no. 1, pp. 51–64, January 2025, doi: 10.9734/jemt/2025/v31i11268.

A. Hinderks, A.-L. Meiners, F. Mayo, and J. Thomaschewski, “Interpreting the Results From the User Experience Questionnaire (UEQ) Using Importance-Performance Analysis (IPA),” in Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST), pp. 388–395, 2019, doi: 10.5220/0008366500002366.

A. A. I. I. Paramitha, G. R. Dantes, and G. Indrawan, “The evaluation of web based academic progress information system using heuristic evaluation and user experience questionnaire (UEQ),” Proc. 3rd Int. Conf. Informatics Comput. ICIC 2018, pp. 1–6, 2018, doi: 10.1109/IAC.2018.8780430.

P. T. S. Tri Anindia Putra I Nyoman, Sepdyana Kartini Ketut, “Analisis Sistem Informasi Eksekutif Experience Questionnaire ( UEQ ) Menggunakan User,” Int. J. Nat. Sci. Eng., vol. 5, no, pp. 25–29, April 2021.

B. Laugwitz, T. Held, and M. Schrepp, “Construction and evaluation of a user experience questionnaire,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5298 LNCS, pp. 63–76, 2008, doi: 10.1007/978-3-540-89350-9_6.

Made Yuda Sadewa, Made Agus Panji Sujaya, I Made Agus Oka Gunawan, and Gede Indrawan, “Evaluasi Pengalaman Pengguna Website PPID Undiksha Dengan Metode User Experience Questionnaire (UEQ),” Inser. Inf. Syst. Emerg. Technol. J., vol. 5, no. 1, pp. 1–11, Juni 2024, doi: 10.23887/insert.v5i1.70383.

M. Schrepp, A. Hinderks, and J. Thomaschewski, “Construction of a Benchmark for the User Experience Questionnaire (UEQ),” Int. J. Interact. Multimed. Artif. Intell., vol. 4, no. 4, p. 40, September 2017, doi: 10.9781/ijimai.2017.445.

R. J. Pratama, A. Junaidi, and S. Sulistiyah, “Analisis User Experience Aplikasi Mobile Banking LivinbyMandiri Menggunakan Metode User Experience Questionnaire,” J. Nas. Komputasi Dan Teknol. Inf., vol. 7, no. 6, Desember 2024, doi: 10.32672/jnkti.v7i6.8457.

P. W. S. Giridharma and I. N. T. A. Putra, “Evaluasi Pengalaman Pengguna Aplikasi Wondr By Bni Menggunakan User Experience Questionnaire (Ueq),” J. Inform. dan Tek. Elektro Terap., vol. 13, no. 2, April 2025, doi: 10.23960/jitet.v13i2.6428.

M. I. Ismail, dalam Evaluasi Pembelajaran Konsep Dasar, Prinsip, Teknik, dan Prosedur, P. Vita, Ed., ke-1. Depok, Indonesia: PT RajaGrafindo Persada, 2020.

G. Indrawan, Sariyasa, L. J. E. Dewi, M. S. Gitakarma, I. M. A. O. Gunawan, and P. A. Pranata, “Usability evaluation of ToAksara as Balinese script learning mobile application,” Int. J. Adv. Appl. Sci., vol. 14, no. 2, pp. 490–501, June 2025, doi: 10.11591/ijaas.v14.i2.pp490-501.

P. W. S. Dewi, G. R. Dantes, and G. Indrawan, “User experience evaluation of e-report application using cognitive walkthrough (cw), heuristic evaluation (he) and user experience questionnaire (ueq),” J. Phys. Conf. Ser., vol. 1516, no. 1, 2020, doi: 10.1088/1742-6596/1516/1/012024.

M. Ariansidi, I. M. Candiasa, I. Made, and G. Sunarya, “Analisis Usability Pada Sistem Informasi LAPORBUP Menggunakan Performance Measurement, Retrospective Think Aloud dan User Experience Questionnaire,” KLIK Kaji. Ilm. Inform. dan Komput., vol. 3, no. 6, pp. 754–764, Juni 2023, doi: 10.30865/klik.v3i6.807.

CNBC Indonesia, “Daftar 10 Bank Terbesar di Indonesia, OCBC Naik - BDMN Terpental,” CNBC Indonesia, 2024. [Online]. Tersedia: https://www.cnbcindonesia.com/market/20240825071058-17-566132/daftar-10-bank-terbesar-di-indonesia-ocbc-naik--bdmn-terpental. [Diakses: 21 Nov. 2025].

D. G. A. Mahendra, I. G. N. S. Adiputra, G. Indrawan, and I. M. A. O. Gunawan, “Evaluasi Pengalaman Pengguna Menggunakan Metode User Experience Questionnaire (UEQ) pada Automated Weather Observing System (AWOS) di Stasiun Meteorologi I Gusti Ngurah Rai,” Explor. J. Sist. Inf. dan Telemat., vol. 15, no. 2, p. 256, Desember 2024, doi: 10.36448/jsit.v15i2.4069.

M. Schrepp, "User Experience Questionnaire Handbook," 2019. [Online]. Available: https://www.ueq-online.org. [Accessed: Feb. 02, 2017].

M. G. Juniastra, I. G. M. Artha, I. M. Agus, O. Gunawan, and G. Indrawan, “KOMPUTA : Jurnal Ilmiah Komputer dan Informatika Evaluation of the BBMKG Region III Website Using the User Experience Questionnaire ( UEQ ) KOMPUTA : Jurnal Ilmiah Komput,” vol. 14, no. 1, pp. 9–12, April 2025, doi: 10.34010/komputa.v14i1.

I. M. A. O. Gunawan, G. Indrawan, and Sariyasa, “User experience evaluation of academic progress information systems using retrospective think aloud and user experience questionnaire,” J. Phys. Conf. Ser., vol. 1810, no. 1, 2021, doi: 10.1088/1742-6596/1810/1/012015.

M. Tavakol and R. Dennick, “Making sense of Cronbach’s alpha,” Int. J. Med. Educ., vol. 2, pp. 53–55, 2011, doi: 10.5116/ijme.4dfb.8dfd.

P. Mishra, C. Pandey, U. Singh, A. Keshri, and M. Sabaretnam, “Selection of appropriate statistical methods for data analysis,” Ann. Card. Anaesth., vol. 22, no. 3, pp. 297–301, 2019, doi: 10.4103/aca.ACA_248_18.

A. Banerjee, U. Chitnis, S. Jadhav, J. Bhawalkar, and S. Chaudhury, “Hypothesis testing, type I and type II errors,” Ind. Psychiatry J., vol. 18, no. 2, p. 127, 2009, doi: 10.4103/0972-6748.62274.