A Meta-Analytic Reliability Generalization Study of the Computational Thinking Scale
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
This study aims to analyze the reliability generalization of the computational thinking scale. There are five dimensions of computational thinking: creativity, algorithmic thinking, coopera-tivity, critical thinking, and problem-solving. A Bonett transformation was used to standardize the reliability coefficient of Cronbach’s alpha. A random-effects meta-analysis was conducted since the heterogeneity among the studies was high. Results supported the RG of the computational thinking scale and its sub-dimensions, which were calculated as 0.843 for general, 0.799 for creativity, 0.848 for algorithmic thinking, 0.863 for cooperativity, 0.799 for critical thinking, and 0.817 for problem-solving. Besides that, the moderator analysis was conducted for the sample type, test length, country, and language of the study. According to the findings, there were no significant moderator effects on the reliability estimation.
Downloads
##plugins.themes.bootstrap3.article.details##
Reliability Generalization, Meta-Analysis, Computational Thinking
Akman, E., & Bircan, M. A. (2021). Öğrencilerin Teknolojiyle Kendi Kendine Öğrenme ve Bilgisayarca Düşünme Becerilerinin İncelenmes i. Ondokuz Mayıs Üniversitesi Eğitim Fakültesi Dergisi, 40(1):12-22. DOI: https://doi.org/10.7822/omuefd.860638
Anastasi, A. (1976). Psychological Testing. In Group Analysis (Fourth Edi). Macmillan Publishing. DOI: https://doi.org/10.1177/053331647600900215
APA. (2020). Publication manual of the american psychological association. in psychological review (7th ed., Vol. 126, Issue 1). The Official Guide to APA Style. DOI: https://doi.org/10.1037/rev0000126
Avcu, Y. E., & Ayverdi, L. (2020). Examination of the Computer Programming Self-Efficacy’s Prediction towards the Computational Thinking Skills of the Gifted and Talented Students. International Journal of Educational Methodology, 6(2):259-270. DOI: https://doi.org/10.12973/ijem.6.2.259
Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4):335-340. DOI: https://doi.org/10.3102/10769986027004335
Çakır, E., & Yaman, S. (2018). Ters Yüz Sınıf Modelinin Öğrencilerin Fen Başarısı ve Bilgisayarca Düşünme Becerileri Üzerine Etkisi. Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 38(1):75-99.
Çevik, M., Barış, N., Şirin, M., Ortak, Ö. K., Kaplan, Y., Özdemir, B. A., Yalçın, H., Şeref, G., Topal, S., & Delice, T. (2021). The effect of digital activities on the technology awareness and computational thinking skills of gifted students (eTwinning project example). In International Journal of Modern Education Studies (Vol. 5, Issue 1). Available at: http://www.ijonmes.net
Çiftçi, S., Cengel, M., & Paf, M. (2018). Reflective thinking skills on computational thinking and problem solving as a predictor of self-efficacy of informatics teacher candidates on programming. Kirşehir Eği̇ti̇m Fakültesi Dergi̇si̇, 19(1):335-348. DOI: https://doi.org/10.29299/kefad.2018.19.009
Clark, D. I. (2015). Computational and algorithmic thinking 2011-2015. AMT Publishing.
Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory- Procedures for Estimating Reliability. Cengage Learning.
Curzon, P., & McOwan, P. W. (2018). Computational thinking. In Springer. Available at: http://www.earticle.net/article.aspx?sn=249447
Denning, P. J. (2017). Computational thinking in science. American Scientist, 205(1):67-77. DOI: https://doi.org/10.1511/2017.124.13
Eser, M. T., & Aksu, G. (2021). Beck depression inventory-II: A study for Meta analytical reliability generalization. Pegem Egitim ve Ogretim Dergisi, 11(3):88-101. DOI: https://doi.org/10.14527/pegegog.2021.00
Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1):38-43. DOI: https://doi.org/10.3102/0013189X12463051
Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41(2):219-231. DOI: https://doi.org/10.1007/BF02291840
Hess, T. J., McNab, A. L., & Basoglu, K. A. (2014). Reliability generalization of perceived ease of use, perceived usefulness, and behavioral intentions. MIS Quarterly, 38(1):1-28. DOI: https://www.jstor.org/stable/26554866
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414):557-560. DOI: https://doi.org/10.1136/bmj.327.7414.557
International Society for Technology in Education [ISTE] (2019). International society for technology in education standards. Available at: https://www.iste.org
Karaçaltı, C., & Korkmaz, Ö. (2018). Öğrencilerin Programlama Başarılarının Bilgisayarca-Eleştirel Düşünme ile Problem Çözme Becerileri Çerçevesinde İncelenmesi. Amasya Üniversitesi Eğitim Fakültesi Dergisi, 7(2):343-370. DOI: https://dergipark.org.tr/tr/pub/amauefd/issue/41157/413487
Kıyıcı, G., & Yamak, H. (2021). Meta-analysis of computational thinking studies in science education: Descriptive statistics. Sakarya University Journal of Education, 11(3):526-544. DOI: https://doi.org/10.19126/suje.975108
Korkmaz, Ö., & Bai, X. (2019). Adapting computational thinking scale (CTS) for chinese high school students and their thinking scale skills level. Participatory Educational Research, 6(1):10-26. DOI: https://doi.org/10.17275/per.19.2.6.1
Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72:558-569. DOI: https://doi.org/10.1016/j.chb.2017.01.005
Korkmaz, Ozden, & Özden, M. Y. (2015). Bilgisayarca Düşünme Beceri Düzeyleri Ölçeğinin ( BDBD ) Ortaokul Düzeyine Uyarlanması. Gazi Eğitim Bilimleri Dergisi, 1:67-86. DOI: https://www.researchgate.net/publication/291338987
Korkmaz, Ozgen, Cakır, R., & Ozden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72:558-569. DOI: https://doi.org/10.1016/j.chb.2017.01.005
Lei, H., Ming, M., Li, F., Wang, X., & Geng, Y. (2020). Children and Youth Services Review Computational thinking and academic achievement: A meta-analysis among students. Children and Youth Services Review, 118:105439. DOI: https://doi.org/10.1016/j.childyouth.2020.105439
Liang, J., Shou, Y., Wang, M. C., Deng, J., & Luo, J. (2021). Alabama Parenting Questionnaire-9: A reliability generalization meta-analysis. Psychological Assessment, 33(10):940-951. DOI: https://doi.org/10.1037/pas0001031
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6(7):e1000100. DOI: https://doi.org/10.1371/journal.pmed.1000100
McDonald, R. P. (1999). Test theory: A unified treatment. Journal of the American Statistical Association, 95(451):1012-1013. DOI: https://doi.org/10.2307/2669496
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7):e1000097. DOI: https://doi.org/10.1371/journal.pmed.1000097
Negrín, F., Hernández-Fernaud, E., Hess, S., & Hernández, B. (2017). Discrimination of urban spaces with different level of restorativeness based on the original and on a shorter version of hartig et al.’s perceived restorativeness scale. Frontiers in psychology, 8:1735. DOI: https://doi.org/10.3389/fpsyg.2017.01735
Niemi, R. G., Carmines, E. G., & McIver, J. P. (1986). The impact of scale length on reliability and validity. Quality and Quantity, 20(4):371-376. DOI: https://doi.org/10.1007/BF00123086
Olderbak, S., Riggenmann, O., Wilhelm, O., & Doebler, P. (2021). Reliability generalization of tasks and recommendations for assessing the ability to perceive facial expressions of emotion. Psychological Assessment, 33(10):911-926. DOI: https://doi.org/10.1037/pas0001030
Oluk, A., Korkmaz, Ö., & Oluk, H. A. (2018). Effect of scratch on 5th graders’ algorithm development and computational thinking skills. Turkish Journal of Computer and Mathematics Education, 9(1):54-71. DOI: https://doi.org/10.16949/turkbilmat.399588
Opitz, M. C., Newman, E., Alvarado Vázquez Mellado, A. S., Robertson, M. D. A., & Sharpe, H. (2020). The psychometric properties of Orthorexia Nervosa assessment scales: A systematic review and reliability generalization. Appetite, 155:104797. DOI: https://doi.org/10.1016/j.appet.2020.104797
Özgür, H. (2020). Relationships between computational thinking skills, ways of thinking and demographic variables: A structural equation modeling. International Journal of Research in Education and Science, 6(2):299-314. DOI: https://doi.org/10.46328/ijres.v6i2.862
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., Mcdonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A.C., Welch, V. A., Whiting, P., Mckenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. The BMJ, 372:n160. DOI: https://doi.org/10.1136/bmj.n160
Papert, S. (1996). An exploration in the sapace of mathematics educations. International Journal of Computers for Mathematical Learning, 1(1):95-123. DOI: https://doi.org/10.1007/bf00191473
Román-González, M., Pérez-González, J. C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior, 72:678-691. DOI: https://doi.org/10.1016/j.chb.2016.08.047
Sen, S. (2021). A reliability generalization Meta-analysis of Runco ideational behavior scale. Creativity Research Journal, 33:1-17. DOI: https://doi.org/10.1080/10400419.2021.1960719
Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1):99-103.
Şencan, H. (2005). Sosyal ve Davranışsal Ölçümlerde Güvenilirlik ve Geçerlilik (1st ed.).
Tsai, M. J., Liang, J. C., & Hsu, C. Y. (2021). The computational thinking scale for computer literacy education. Journal of Educational Computing Research, 59(4):579-602. DOI: https://doi.org/10.1177/0735633120972356
Vacha-Haase, T. (1998). Reliability generalization: exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1):6. DOI: https://doi.org/10.1177/0013164498058001002
Vacha-Haase, T., Henson, R. K., & Caruso, J. C. (2002). Reliability generalization: Moving toward improved understanding and use of score reliability. Educational and Psychological Measurement, 62(4):562-569. DOI: https://doi.org/10.1177/0013164402062004002
Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60(4):509-522. DOI: https://doi.org/10.1177/00131640021970682
Vaidyanathan, S. (2016). What’s the Difference Between Coding and Computational Thinking? EdSurge. Available at: https://www.edsurge.com/news/2016-08-06-what-s-the-difference-between-coding-and-computational-thinking
Wilkinson, L. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8):594-604. DOI: https://doi.org/10.1037//0003-066x.54.8.594
Wing, J. M. (2006). Computational Thinking. Communications of The Acm, 49(3):33-35. DOI: https://doi.org/10.14742/ajet.3521
Yalçın, N., & İkinci, A. (2020). Meslek Liseleri Bilişim Teknolojileri Alan Öğrencilerinin Bilgisayarca Düşnme Beceri Düzeylerinin Eğitim Program Türüne Göre İncelenmesi. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 20(4):1639-1656. DOI: https://doi.org/2020.20.58249-650813
Yoon, S., Yang, Y., Ro, E., Ahn, W.-Y., Kim, J., Shin, S.-H., Chey, J., & Choi, K.-H. (2021). Reliability, and convergent and discriminant validity of gaming disorder scales: A meta-analysis. Frontiers in Psychology, 12:1-15. DOI: https://doi.org/10.3389/fpsyg.2021.764209
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.