Assessing computational thinking skills of science and mathematics upper-secondary school students

Assessing computational thinking skills of science and mathematics upper-secondary school students

Publication type: Original Article
Year: 8 October 2025
Author: Nur Huda, Eli Rohaeti
Journal name: European Journal of Science and Mathematics Education
Volume/Issue: 13(4)
DOI: 10.30935/scimath/17248

Abtract

Computational thinking (CT) is a thinking skill developed and integrated into curricula worldwide in recent years. However, limited assessment is one of the challenges in integrating CT skills into the educational curriculum of developing countries such as Indonesia. This study aimed to develop and validate a CT assessment instrument tailored for upper-secondary school students majoring in science and mathematics in Indonesia. The cross-cultural assessment adaptation method was adopted, comprising six stages: translation, synthesis, back-translation, expert committee review, pretesting, and research audit. Twelve experts were involved in the content validation stage to assess the feasibility of the instrument adapted in Indonesia. The validation process was followed by a pilot test with 501 upper-secondary students majoring in science and mathematics (220 female and 281 male). The data collected were analyzed using the Rasch model measurement. The findings showed that all adapted items met the fit based on the Rasch model measurement, except one spatial question item. The instrument demonstrated high item reliability, although person reliability was relatively low, indicating variation in student responses. The average upper-secondary school students majoring in science have good CT skills. Based on the differential item function value, there are two gender-biased items and four age-biased items. This study hopes to contribute to the literature on CT assessment by providing references and alternative tests for researchers and teachers to use in assessing CT in upper-secondary school students.

Keywords: assessment, computational thinking, Rasch model, science and mathematics student, test adaptation

Abstrak

Computational Thinking (CT) adalah keterampilan berpikir yang dikembangkan dan diintegrasikan ke dalam kurikulum di seluruh dunia dalam beberapa tahun terakhir. Namun, keterbatasan penilaian merupakan salah satu tantangan dalam mengintegrasikan keterampilan CT ke dalam kurikulum pendidikan di negara-negara berkembang seperti Indonesia. Studi ini bertujuan untuk mengembangkan dan memvalidasi alat penilaian CT yang disesuaikan untuk siswa sekolah menengah atas yang mengambil jurusan sains dan matematika di Indonesia. Metode adaptasi penilaian lintas budaya diterapkan, yang meliputi enam tahap: terjemahan, sintesis, terjemahan balik, tinjauan komite ahli, uji coba awal, dan audit penelitian. Dua belas ahli terlibat dalam tahap validasi konten untuk menilai kelayakan instrumen yang diadaptasi di Indonesia. Proses validasi dilanjutkan dengan uji coba dengan 501 siswa sekolah menengah atas yang mengambil jurusan sains dan matematika (220 perempuan dan 281 laki-laki). Data yang dikumpulkan dianalisis menggunakan model pengukuran Rasch. Hasil penelitian menunjukkan bahwa semua item yang diadaptasi memenuhi kriteria kesesuaian berdasarkan pengukuran model Rasch, kecuali satu item soal spasial. Instrumen tersebut menunjukkan reliabilitas item yang tinggi, meskipun reliabilitas individu relatif rendah, menunjukkan variasi dalam respons siswa. Siswa SMA jurusan sains memiliki keterampilan CT yang baik. Berdasarkan nilai fungsi item diferensial, terdapat dua item yang bias gender dan empat item yang bias usia. Penelitian ini bertujuan untuk berkontribusi pada literatur tentang penilaian CT dengan menyediakan referensi dan tes alternatif bagi peneliti dan guru untuk digunakan dalam menilai CT pada siswa sekolah menengah atas.

Kata kunci: adaptasi tes, Rasch model, pemikiran komputasional, penilaian, siswa Matematika dan IPA

Cara Sitasi

Rohaeti, E., & Huda, N. (2025). Assessing computational thinking skills of science and mathematics upper-secondary school students. European Journal of Science and Mathematics Education, 13(4), 289-303. https://doi.org/10.30935/scimath/17248
SCImago Journal & Country Rank

Download

Referensi

  1. Adams, C., Cutumisu, M., Yuen, C., Hackman, Lu, C., & Samuel, M. (2019). Callysto computational thinking test (CCTt) teacher version. Callysto. https://callysto.ca/computational-thinking-tests/callysto-computational-thinking-test-cctt-teacher-version/Resource_Callysto-CTt_Instrument_Teacher_version-1.pdf
  2. Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(4), 955–959. https://doi.org/10.1177/001316448004000419
  3. Andrich, D. (1988). Rasch models for measurement. SAGE. https://doi.org/10.4135/9781412985598
  4. Andrich, D., & Styles, I. (2004). Final report on the psychometric analysis of the early development instrument (EDI) using the Rasch model. AEDI. https://api.semanticscholar.org/CorpusID:141656000
  5. Angeli, C., & Giannakos, M. (2020). Computational thinking education: Issues and challenges. Computers in Human Behavior, 105, Article 106185. https://doi.org/10.1016/j.chb.2019.106185
  6. Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
  7. Bellettini, C., Lonati, V., Malchiodi, D., Monga, M., Morpurgo, A., & Torelli, M. (2015). How challenging are Bebras tasks?: An IRT analysis based on the performance of Italian students. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education (pp. 27–32). ACM. https://doi.org/10.1145/2729094.2742603
  8. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. Lawrence Erlbaum Associates Publishers.
  9. Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), Article rm4. https://doi.org/10.1187/cbe.16-04-0148
  10. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
  11. Cansu, F. K., & Cansu, S. K. (2019). An overview of computational thinking. International Journal of Computer Science Education in Schools, 3(1), 17–30. https://doi.org/10.21585/ijcses.v3i1.53
  12. Cantlon, J. F., Becker, K. T., & DeLong, C. M. (2024). Computational thinking during a short, authentic, interdisciplinary STEM experience for elementary students. Journal for STEM Education Research, 7, 425–443. https://doi.org/10.1007/s41979-024-00117-0
  13. Chagas, D., & Furtado, E. (2019). Computational thinking in basic education in a developing country perspective. In A. Visvizi, & M. D. Lytras (Eds.), Research & innovation forum 2019 (pp. 135–150). Springer. https://doi.org/10.1007/978-3-030-30809-4_14
  14. Chytas, C., Van Borkulo, S. P., Drijvers, P., Barendsen, E., & Tolboom, J. L. J. (2024). Computational thinking in secondary mathematics education with GeoGebra: Insights from an intervention in calculus lessons. Digital Experiences in Mathematics Education, 10, 228–259. https://doi.org/10.1007/s40751-024-00141-0
  15. Cutumisu, M., Adams, C., Glanfield, F., Yuen, C., & Lu, C. (2022). Using structural equation modeling to examine the relationship between preservice teachers’ computational thinking attitudes and skills. IEEE Transactions on Education, 65(2), 177–183. https://doi.org/10.1109/TE.2021.3105938
  16. Cutumisu, M., Adams, C., Yuen, C., Hackman, L., Lu, C., & Samuel, M. (2019a). Callysto computational thinking test (CCTt) student version. Callysto. https://callysto.ca/computational-thinking-tests/callysto-computational-thinking-test-cctt-student-version/Resource_Callysto-CTt_Instrument_Student_version-1.pdf
  17. Cutumisu, M., Adams, C., Yuen, C., Hackman, M., Lu, C., & Samuel, M. (2019b). Computational thinking test. Callysto. https://callysto.ca/computational-thinking-tests/
  18. Dagiene, V., & Stupuriene, G. (2016). Informatics concepts and computational thinking in K-12 education: A Lithuanian perspective. Journal of Information Processing, 24(4), 732–739. https://doi.org/10.2197/ipsjjip.24.732
  19. De Jong, I., & Jeuring, J. (2022). Developing a self-efficacy scale for computational thinking (CT-SES). In Proceedings of the 22nd Koli Calling International Conference on Computing Education Research (pp. 1–2). https://doi.org/10.1145/3564721.3565954
  20. El-Hamamsy, L., Zapata-Cáceres, M., Martín-Barroso, E., Mondada, F., Zufferey, J. D., Bruno, B., & Román-González, M. (2025). The competent computational thinking test (cCTt): A valid, reliable and gender-fair test for longitudinal CT studies in grades 3–6. Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-024-09777-8
  21. Ertugrul-Akyol, B. (2019). Development of computational thinking scale: Validity and reliability study. International Journal of Educational Methodology, 5(3), 421–432. https://doi.org/10.12973/ijem.5.3.421
  22. ESTE, & CSTA. (2011). Operational definition of computational thinking for K-12 education. ISTE. https://cdn.iste.org/www-root/Computational_Thinking_Operational_Definition_ISTE.pdf
  23. Ezeamuzie, N. O., & Leung, J. S. C. (2022). Computational thinking through an empirical lens: A systematic review of literature. Journal of Educational Computing Research, 60(2), 481–511. https://doi.org/10.1177/07356331211033158
  24. Güven, I., & Gulbahar, Y. (2020). Integrating computational thinking into social studies. The Social Studies, 111(5), 234–248. https://doi.org/10.1080/00377996.2020.1749017
  25. Harangus, K., & Kátai, Z. (2020). Computational thinking in secondary and higher education. Procedia Manufacturing, 46, 615–622. https://doi.org/10.1016/j.promfg.2020.03.088
  26. Herrmann-Abell, C. F., Hardcastle, J., & DeBoer, G. E. (2018). Comparability of computer-based and paper-based science assessments. In Proceedings of the NARST Annual International Conference.
  27. Hsu, T.-C., & Liang, Y.-S. (2021). Simultaneously improving computational thinking and foreign language learning: Interdisciplinary media with plugged and unplugged approaches. Journal of Educational Computing Research, 59(6), 1184–1207. https://doi.org/10.1177/0735633121992480
  28. Huda, N., & Rohaeti, E. (2024). Computational thinking skill level of senior high school students majoring in natural science. International Journal of Learning, Teaching and Educational Research, 23(1), 339–359. https://doi.org/10.26803/ijlter.23.1.17
  29. Hurt, T., Greenwald, E., Allan, S., Cannady, M. A., Krakowski, A., Brodsky, L., Collins, M. A., Montgomery, R., & Dorph, R. (2023). The computational thinking for science (CT-S) framework: Operationalizing CT-S for K-12 science education researchers and educators. International Journal of STEM Education, 10(1), Article 1. https://doi.org/10.1186/s40594-022-00391-7
  30. Jin, H.-Y., & Cutumisu, M. (2023). Predicting pre-service teachers’ computational thinking skills using machine learning classifiers. Education and Information Technologies 28, 11447–11467. https://doi.org/10.1007/s10639-023-11642-7
  31. Kakavas, P., & Ugolini, F. C. (2019). Computational thinking in primary education: A systematic literature review. Research on Education and Media, 11(2), 64–94. https://doi.org/10.2478/rem-2019-0023
  32. Kamak, L. P., & Mago, V. (2023). Assessing the impact of using Python to teach computational thinking for remote schools in a blended learning environment. In P. Zaphiris, & A. Ioannou (Eds.), Learning and collaboration technologies (vol. 14041, pp. 482–500). Springer. https://doi.org/10.1007/978-3-031-34550-0_35
  33. Kelderman, H. (1989). Item bias detection using loglinear IRT. Psychometrika, 54(4), 681–697. https://doi.org/10.1007/BF02296403
  34. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005
  35. Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607–610. https://doi.org/10.1177/001316447003000308
  36. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
  37. Li, X., Sang, G., Valcke, M., & Van Braak, J. (2024). The development of an assessment scale for computational thinking competence of in-service primary school teachers. Journal of Educational Computing Research, 62(6), 1318–1347. https://doi.org/10.1177/07356331241254575
  38. Li, Y., Schoenfeld, A. H., diSessa, A. A., Graesser, A. C., Benson, L. C., English, L. D., & Duschl, R. A. (2020). Computational thinking is more about thinking than computing. Journal for STEM Education Research, 3(1), 1–18. https://doi.org/10.1007/s41979-020-00030-2
  39. Mohaghegh, M., & McCauley, M. (2016). Computational thinking: The skill set of the 21st century. International Journal of Computer Science and Information Technologies, 7(3), 1524–1530.
  40. Moreno-Leon, J., Roman-Gonzalez, M., & Robles, G. (2018). On computational thinking as a universal skill: A review of the latest research on this ability. In Proceedings of the 2018 IEEE Global Engineering Education Conference (pp. 1684–1689). IEEE. https://doi.org/10.1109/EDUCON.2018.8363437
  41. Niousha, R., Saito, D., Washizaki, H., & Fukazawa, Y. (2023). Investigating the effect of binary gender preferences on computational thinking skills. Education Sciences, 13(5), Article 433. https://doi.org/10.3390/educsci13050433
  42. Noyes, J. M., & Garland, K. J. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51(9), 1352–1375. https://doi.org/10.1080/00140130802170387
  43. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
  44. Ocampo, L. M., Corrales-Álvarez, M., Cardona-Torres, S. A., & Zapata-Cáceres, M. (2024). Systematic review of instruments to assess computational thinking in early years of schooling. Education Sciences, 14(10), Article 1124. https://doi.org/10.3390/educsci14101124
  45. Ogegbo, A. A., & Ramnarain, U. (2022). A systematic review of computational thinking in science classrooms. Studies in Science Education, 58(2), 203–230. https://doi.org/10.1080/03057267.2021.1963580
  46. Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books.
  47. Popat, S., & Starkey, L. (2019). Learning to code or coding to learn? A systematic review. Computers & Education, 128, 365–376. https://doi.org/10.1016/j.compedu.2018.10.005
  48. Putra, Z. H., Ramiati, Zufriady, Hidayat, R., Jismulatif, Hermita, N., & Sulistiyo, U. (2022). Development of computational thinking tasks based on Riau Malay culture: A study of fifth-grade public school students in Pekanbaru, Indonesia. Education 3–13, 52(8), 1387–1397. https://doi.org/10.1080/03004279.2022.2150063
  49. Rahimi, A. R., & Sevilla-Pavón, A. (2025). Scaling up computational thinking skills in computer-assisted language learning (CTsCALL) and its fitness with language learners’ intentions to use virtual exchange: A bi-symmetric approach. Computers in Human Behavior Reports, 17, Article 100607. https://doi.org/10.1016/j.chbr.2025.100607
  50. Revana, G., Kavita, K., & Madhavi, V. (2021). Exploring the concept of computational thinking in STEM education. In Proceedings of the 2021 World Engineering Education Forum/Global Engineering Deans Council (pp. 375–380). https://doi.org/10.1109/WEEF/GEDC53299.2021.9657230
  51. Romero, M., Lepage, A., & Lille, B. (2017). Computational thinking development through creative programming in higher education. International Journal of Educational Technology in Higher Education, 14(1), Article 42. https://doi.org/10.1186/s41239-017-0080-z
  52. Rottenhofer, M., Kuka, L., Leitner, S., & Sabitzer, S. (2022). Using computational thinking to facilitate language learning: A survey of students’ strategy use in Austrian secondary schools. IAFOR Journal of Education, 10(2), 51–70. https://doi.org/10.22492/ije.10.2.03
  53. Saidin, N. D., Khalid, F., Martin, R., Kuppusamy, Y., & Munusamy, N. A. (2021). Benefits and challenges of applying computational thinking in education. International Journal of Information and Education Technology, 11(5), 248–254. https://doi.org/10.18178/ijiet.2021.11.5.1519
  54. Sari, U., Ulusoy, A., & Pektaş, H. M. (2025). Computational thinking in science laboratories based on the flipped classroom model: Computational thinking, laboratory entrepreneurial and attitude. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-024-10192-y
  55. Snyder, S., & Sheehan, R. (1992). The Rasch measurement model: An introduction. Journal of Early Intervention, 16(1), 87–95. https://doi.org/10.1177/105381519201600108
  56. So, H.-J., Jong, M. S.-Y., & Liu, C.-C. (2020). Computational thinking education in the Asian Pacific Region. The Asia-Pacific Education Researcher, 29(1), 1–8. https://doi.org/10.1007/s40299-019-00494-w
  57. Sovey, S., Osman, K., & Mohd Matore, M. E. E. (2022). Rasch analysis for disposition levels of computational thinking instrument among secondary school students. Eurasia Journal of Mathematics, Science and Technology Education, 18(3), Article em2088. https://doi.org/10.29333/ejmste/11794
  58. Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan [Application of Rasch modelling to educational assessment]. Trim Komunikata Publishing House.
  59. Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
  60. Tabesh, Y. (2017). Computational thinking: A 21st century skill. Olympiads in Informatics, 11(2), 65–70. https://doi.org/10.15388/ioi.2017.special.10
  61. Taherdoost, H. (2017). Determining sample size: How to calculate survey sample size. International Journal of Economics and Management Systems, 2, 237–239.
  62. Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, Article 103798. https://doi.org/10.1016/j.compedu.2019.103798
  63. Tariq, R., Aponte Babines, B. M., Ramirez, J., Alvarez-Icaza, I., & Naseer, F. (2025). Computational thinking in STEM education: Current state-of-the-art and future research directions. Frontiers in Computer Science, 6. https://doi.org/10.3389/fcomp.2024.1480404
  64. Tripon, C. (2022). Supporting future teachers to promote computational thinking skills in teaching STEM–A case study. Sustainability, 14(19), Article 12663. https://doi.org/10.3390/su141912663
  65. Tsai, M.-J., Liang, J.-C., & Hsu, C.-Y. (2021). The computational thinking scale for computer literacy education. Journal of Educational Computing Research, 59(4), 579–602. https://doi.org/10.1177/0735633120972356
  66. Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6
  67. Voskoglou, M. G., & Buckley, S. (2012). Problem solving and computational thinking in a learning environment. Egyptian Computer Science Journal, 36(4), 28–46.
  68. Waterman, K. P., Goldsmith, L., & Pasquale, M. (2020). Integrating computational thinking into elementary science curriculum: An examination of activities that support students’ computational thinking in the service of disciplinary learning. Journal of Science Education and Technology, 29(1), 53–64. https://doi.org/10.1007/s10956-019-09801-y
  69. Weintrop, D., Morehouse, S., & Subramaniam, M. (2021). Assessing computational thinking in libraries. Computer Science Education, 31(2), 290–311. https://doi.org/10.1080/08993408.2021.1874229
  70. Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
  71. Wu, M., & Adams, R. J. (2007). Applying the Rasch model to psycho-social measurement: A practical approach. Educational Measurement Solutions.
  72. Wu, T.-T., Asmara, A., Huang, Y.-M., & Permata Hapsari, I. (2024). Identification of problem-solving techniques in computational thinking studies: Systematic literature review. SAGE Open, 14(2). https://doi.org/10.1177/21582440241249897
  73. Xu, Z., & Zhang, J. (2021). Computational thinking: A perspective on computer science (1st ed.). Springer. https://doi.org/10.1007/978-981-16-3848-0
  74. Yamashita, T. (2022). Analyzing Likert scale surveys with Rasch models. Research Methods in Applied Linguistics, 1(3), Article 100022. https://doi.org/10.1016/j.rmal.2022.100022
  75. Yang, D., Snelson, C., & Feng, S. (2023). Identifying computational thinking in students through project-based problem-solving activities. Information Discovery and Delivery, 51(3), 293–305. https://doi.org/10.1108/IDD-09-2022-0091
  76. Yang, T.-C., & Lin, Z.-S. (2024). Enhancing elementary school students’ computational thinking and programming learning with graphic organizers. Computers & Education, 209, Article 104962. https://doi.org/10.1016/j.compedu.2023.104962
  77. Zamzami, E. M., Tarigan, J. T., Zendrato, N., Muis, A., Yoga, A. P., & Faisal, M. (2020). Exercising the students computational thinking ability using Bebras challenge. Journal of Physics: Conference Series, 1566(1), Article 012113. https://doi.org/10.1088/1742-6596/1566/1/012113
  78. Zapata, J. H., Gutiérrez Posada, J. E., & Diago, P. D. (2024). Design and validation of a computational thinking test for children in the first grades of elementary education. Multimodal Technologies and Interaction, 8(5), Article 39. https://doi.org/10.3390/mti8050039
Reaksi Anda?
+1
2
+1
0
+1
0
+1
0
+1
0
Bagikan:
HUDA
HUDA

Sekarang masih nganggur, besok nggak tahu. Kadang suka nulis, kadang suka tidur, kadang suka ngayal, ya gitu aja sih. Moto hidup, "Jangan kebanyakan bermoto"😅

Articles: 26

Leave a Reply

Alamat email Anda tidak akan dipublikasikan. Ruas yang wajib ditandai *

Baca lainnya

AI dan Masa Depan Penelitian Pendidikan
AI dan Masa Depan Penelitian Pendidikan
Cara Instal Ministep dan Winsteps
Cara Instal Ministep dan Winsteps: Aplikasi Pemodelan Rasch
computational thinking
Computational Thinking Skill Level of Senior High School Students Majoring in Natural Science
Dasar Teori Hasil ChatGPT dan AI
Dasar Teori Hasil ChatGPT dan AI ~ #2 SeriesKPP