Evaluating Computer-Based Testing System Success Using the DeLone and McLean Model among Cadets at an Indonesian Aviation Higher Education Institution
DOI:
https://doi.org/10.17977/um065.v7.i2.2027.4Keywords:
Computer-based testing, DeLone and McLean model, Information system success, PLS-SEM, User satisfactionAbstract
The rapid adoption of Computer-Based Testing (CBT) in higher education requires an evaluation framework to determine whether the implemented system successfully provides benefits for users. This study aims to evaluate the success of CBT implementation among cadets of a higher education institution in Indonesia using the DeLone and McLean Information Systems Success Model. A quantitative explanatory approach was employed involving 108 cadets who had experience using the CBT system. Data were collected through a structured questionnaire and analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM) with SmartPLS 3.2.9. The evaluation consisted of measurement model assessment and structural model testing using bootstrapping with 5,000 resamples. The results indicate that information quality significantly influences user satisfaction (β = 0.399, p = 0.001), and service quality significantly influences user satisfaction (β = 0.298, p = 0.011). However, system quality does not significantly influence user satisfaction (β = 0.225, p = 0.055). Furthermore, user satisfaction has a strong positive influence on implementation impact (β = 0.846, p < 0.001). These findings indicate that successful CBT implementation is determined not only by technical system performance but also by the quality of information and service support provided to users. This study contributes practical implications for higher education institutions in improving digital assessment systems by prioritizing information clarity, technical assistance, and user satisfaction.References
Al-Banna, N. A. (2025). Undergraduate students’ attitudes toward computer-based and traditional paper-based testing: A comparative study. Social Sciences & Humanities Open, 12, 101988. https://doi.org/10.1016/j.ssaho.2025.101988
Alyoussef, I. Y. (2023). Acceptance of e-learning in higher education: The role of task-technology fit with the information systems success model. Heliyon, 9(3), e13751. https://doi.org/10.1016/j.heliyon.2023.e13751
Berkhout, E., Pradhan, M., Rahmawati, Suryadarma, D., & Swarnata, A. (2024). Using technology to prevent fraud in high stakes national school examinations: Evidence from Indonesia. Journal of Development Economics, 170, 103307. https://doi.org/10.1016/j.jdeveco.2024.103307
Bervell, B., Mireku, D. O., & Agyapong, D. (2024). Modelling the antecedents of students’ satisfaction and continuous use intentions of an electronic appraisal portal system in higher education. Computers in Human Behavior Reports, 15, 100431. https://doi.org/10.1016/j.chbr.2024.100431
Cardenas-Pauckar, D., Del-Aguila-Arcentales, S., & Puente-Custodio, E. (2026). Trust in the digital age: Chatbot service quality and its influence on e-WOM and loyalty among consumers. Telematics and Informatics Reports, 22, 100328. https://doi.org/10.1016/j.teler.2026.100328
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30. https://doi.org/10.1080/07421222.2003.11045748
Fragulis, G. F., Papatsimouli, M., Lazaridis, L., & Skordas, I. A. (2021). An online dynamic examination system (ODES) based on open source software tools. Software Impacts, 7, 100046. https://doi.org/10.1016/j.simpa.2020.100046
Hair, J., & Alamer, A. (2022). Partial least squares structural equation modeling (PLS-SEM) in second language and education research: Guidelines using an applied example. Research Methods in Applied Linguistics, 1(3), 100027. https://doi.org/10.1016/j.rmal.2022.100027
Hameed, A.-R. A. G., & Sumari, P. B. (2024). Adoption and continued usage of mobile learning of virtual platforms in Iraqi higher education an unstable environment. International Journal of Information Management Data Insights, 4(2), 100242. https://doi.org/10.1016/j.jjimei.2024.100242
Karattuthodi, M. S., C, S. C., Thorakkattil, S. A., Chandrasekhar, D., Punnoth Poonkuzhi, N., Mohammed Ahmed Ageeli, M., & Madathil, H. (2022). Pharmacy student’s challenges in virtual learning system during the second COVID-19 wave in southern India. Social Sciences & Humanities Open, 5(1), 100241. https://doi.org/10.1016/j.ssaho.2021.100241
Liu, C., & Li, J. (2012). Exploration and reflection of the design models of computerized testing. AASRI Procedia, 3, 101–106. https://doi.org/10.1016/j.aasri.2012.11.018
Lutfi, A. (2023). Factors affecting the success of accounting information system from the lens of DeLone and McLean IS model. International Journal of Information Management Data Insights, 3(2), 100202. https://doi.org/10.1016/j.jjimei.2023.100202
M Ayyoub, A. A., Abu Eidah, B. A., Khlaif, Z. N., Ahmad EL-Shamali, M., & Sulaiman, M. R. (2023). Understanding online assessment continuance intention and individual performance by integrating task technology fit and expectancy confirmation theory. Heliyon, 9(11), e22068. https://doi.org/10.1016/j.heliyon.2023.e22068
Nadella, G. S., Meduri, K., Satish, S., Maturi, M. H., & Gonaygunta, H. (2024). Examining e-learning tools impact using IS-impact model: A comparative PLS-SEM and IPMA case study. Journal of Open Innovation: Technology, Market, and Complexity, 10(3), 100351. https://doi.org/10.1016/j.joitmc.2024.100351
Roky, H., & Meriouh, Y. Al. (2015). Evaluation by users of an industrial information system (XPPS) based on the DeLone and McLean model for IS success. Procedia Economics and Finance, 26, 903–913. https://doi.org/10.1016/S2212-5671(15)00903-X
Sabeh, H. N., Husin, M. H., Kee, D. M. H., Baharudin, A. S., & Abdullah, R. (2021). A systematic review of the DeLone and McLean model of information systems success in an e-learning context (2010–2020). IEEE Access, 9, 81210–81235. https://doi.org/10.1109/ACCESS.2021.3084815
Sattar, M. R. I., Efty, M. T. B. H., Rafa, T. S., Das, T., Samad, M. S., Pathak, A., … Ullah, M. H. (2023). An advanced and secure framework for conducting online examination using blockchain method. Cyber Security and Applications, 1, 100005. https://doi.org/10.1016/j.csa.2022.100005
Varajão, J., Lourenço, J. C., & Gomes, J. (2022). Models and methods for information systems project success evaluation—A review and directions for research. Heliyon, 8(12), e11977. https://doi.org/10.1016/j.heliyon.2022.e11977
Viontita, S. C., & ER, M. (2024). Evaluation of Surabaya population administration & civil registration systems using DeLone & McLean information system success model. Procedia Computer Science, 234, 1154–1163. https://doi.org/10.1016/j.procs.2024.03.111
Wider, W., Tan, F. P., Tan, Y. P., Lin, J., Fauzi, M. A., Wong, L. S., … Hossain, S. F. A. (2024). Service quality (SERVQUAL) model in private higher education institutions: A bibliometric analysis of past, present, and future prospects. Social Sciences & Humanities Open, 9, 100805. https://doi.org/10.1016/j.ssaho.2024.100805
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Faoyan Agus Furyanto, Lady Silk Moonlight, Maulana Anifa Silvia, Anton Budiarto, Ahmad Musadek, Abdul Mu’ti Sazali
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
