The Education Compass: Navigating Beyond the North of PISA, TIMSS, and PIRLS

The Education Compass: Navigating Beyond the North of PISA, TIMSS, and PIRLS

Anh-Duc Hoang ducha@skylineschool.edu.vn Sky-Line School (Vietnam), and RMIT University (Australia)
Summary: 
Global learning measurement systems such as Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), and Programme for International Student Assessment (PISA) have been widely used to compare the educational performance of different countries. However, these systems have some limitations that need to be considered. After analyzing 1,265 research articles on such measurement agendas, this article highlights blind spots and blackspots surrounding largescale metrics’ development, measurement, and implications. This commentary article emphasizes that essential factors about culture and diversity have yet to be included in such metrics, leading to monochrome paintings of the global education landscape rather than mosaics. Overall, we advocate for a strategic shift in developing and adopting such global measurement agendas.
Keywords: 
Global Education Measurement; PISA; TIMSS; PIRLS; K-12 Education
Refers: 

[1] Baroutsis, A., & Lingard, B. (2022). A methodological approach to the analysis of PISA microblogs: social media during the release of the PISA 2015 results. Journal of Education Policy, 37(6), 904–924. https:// doi.org/10.1080/02680939.2021.1937706

[2] Bracey, G. W. (2003). PIRLS before the Press. Phi Delta Kappan, 84(10), 795–795. https://doi. org/10.1177/003172170308401017

[3] Burdett, N. (2013). The misuse of international studies in UK education. SecED.

[4] Checchi, D., & Verzillo, S. (2017). The Role of PISA in Regional and Private/Public Debates in Italy. In L. Volante (Ed.), Routledge Research in Education Policy and Politics, (pp. 127–148). Routledge.

[5] Chen, X. (2022). The effects of individual- and classlevel achievement on attitudes towards mathematics: An analysis of Hong Kong students using TIMSS 2019. Studies in Educational Evaluation, 72, 101113. https://doi.org/10.1016/j.stueduc.2021.101113

[6] Dohn, N. B. (2007). Knowledge and Skills for PISA?Assessing the Assessment. Journal of Philosophy of Education, 41(1), 1–16. https://doi.org/10.1111/j.1467-9752.2007.00542.x

[7] Elstad, E. (2023). The Evolution of the Extended Comprehensive School Model and the Modern Profession-Oriented Teacher Education After World War II (pp. 35–72). https://doi.org/10.1007/978-3- 031-26051-3_3

[8] Engel, L. C. (2015). Steering the National: Exploring the Education Policy Uses of PISA in Spain. European Education, 47(2), 100–116. https://doi.org/10.1080/1 0564934.2015.1033913

[9] Feniger, Y., & Lefstein, A. (2014). How not to reason with PISA data: an ironic investigation. Journal of Education Policy, 29(6), 845–855. https://doi.org/10 .1080/02680939.2014.892156

[10] Fredriksson, U., Holzer, T., McCluskey-Cavin, H., & Taube, K. (2009). Strengths and Weaknesses in the Swedish and Swiss Education Systems: A Comparative Analysis Based on PISA Data. European Educational Research Journal, 8(1), 54–68. https:// doi.org/10.2304/eerj.2009.8.1.54

[11] Freitas, P., Nunes, L. C., Balcão Reis, A., Seabra, C., & Ferro, A. (2016). Correcting for sample problems in PISA and the improvement in Portuguese students’ performance. Assessment in Education: Principles, Policy & Practice, 23(4), 456–472. https://doi.org/1 0.1080/0969594X.2015.1105784

[12] Gaber, S., Cankar, G., Umek, L. M., & Tašner, V. (2012). The danger of inadequate conceptualisation in PISA for education policy. Compare: A Journal of Comparative and International Education, 42(4), 647–663. https://doi.org/10.1080/03057925.2012.65 8275

[13] Glassow, L. N., Rolfe, V., & Hansen, K. Y. (2021). Assessing the comparability of teacher-related constructs in TIMSS 2015 across 46 education systems: an alignment optimization approach. Educational Assessment, Evaluation and Accountability, 33(1), 105–137. https://doi.org/10.1007/s11092-020-09348- 2

[14] Gravel, N., & Robin, L. (2022). Evaluating Education Systems Through Inequalities Between Families. Dialogues Economiques. https://www. dialogueseconomiques.fr/en/article/evaluatingeducation-systems-through-inequalities-betweenfamilies

[15] Hoang, A.-D. (2024). Studies on TIMSS, PIRLS and PISA from 1995 to 2023. Harvard Dataverse. https://doi. org/https://doi.org/10.7910/DVN/YXKVUR

[16] Hoang, A.-D., Pham, H.-H., Nguyen, Y.-C., Nguyen, L.-K.-N., Vuong, Q.-H., Dam, M. Q., Tran, T., & Nguyen, T.-T. (2020). Introducing a tool to gauge curriculum quality under Sustainable Development Goal 4: The case of primary schools in Vietnam. International Review of Education, 66(4), 457–485. https://doi.org/10.1007/s11159-020-09850-1

[17] Hwang, J., Runnalls, C., Bhansali, S., Navaandamba, K., & Choi, K. M. (2017). “Can I do well in mathematics reasoning?” Comparing US and Finnish students’ attitude and reasoning via TIMSS 2011. Educational Research and Evaluation, 23(7–8), 328–348. https:// doi.org/10.1080/13803611.2018.1500293

[18] Jerrim, J. (2011). England’s “plummeting” PISA test scores between 2000 and 2009: Is the performance of our secondary school pupils really in relative decline (DoQSS Working Papers).

[19] Kell, M., & Kell, P. (2014). Global Testing: PISA, TIMSS and PIRLS. In Literacy and Language in East Asia (pp. 33–49). Springer Singapore. https://doi. org/10.1007/978-981-4451-30-7_5

[20] Labaree, D. F. (2014). Let’s Measure What No One Teaches: PISA, NCLB, and the Shrinking Aims of Education. Teachers College Record: The Voice of Scholarship in Education, 116(9), 1–14. https://doi. org/10.1177/016146811411600905

[21] Martens, K., Knodel, P., & Windzio, M. (Eds.). (2014). Internationalization of Education Policy. Palgrave Macmillan UK. https://doi. org/10.1057/9781137401694

[22] Moher, D., Altman, D. G., Liberati, A., & Tetzlaff, J. (2011). PRISMA Statement. Epidemiology, 22(1), 128. https://doi.org/10.1097/EDE.0b013e3181fe7825

[23] Muench, R., Wieczorek, O., & Dressler, J. (2023). Equity lost: Sweden and Finland in the struggle for PISA scores. European Educational Research Journal, 22(3), 413– 432. https://doi.org/10.1177/14749041211069240

[24] OECD. (2022). PISA Creative Thinking. https://www. oecd.org/pisa/innovation/creative-thinking/OECD. (2023). PISA Learning Digital World

[25] Østbø, I. U., & Zachrisson, H. D. (2022). Student Motivation and Parental Attitude as Mediators for SES Effects on Mathematics Achievement: Evidence from Norway in TIMSS 2015. Scandinavian Journal of Educational Research, 66(5), 808–823. https://doi. org/10.1080/00313831.2021.1939138

[26] Oz, E. (2021). Comparability of teachers’ educational background items in TIMSS: a case from Turkey. Large-Scale Assessments in Education, 9(1), 4. https://doi.org/10.1186/s40536-021-00097-2

[27] Pham, H.-H., Hoang, A.-D., Lai, S.-L., Dong, T.-K.-T., Nghia, T. L. H., Ho, M.-T., & Vuong, Q.-H. (2022). International education as an export sector: an investigation of 49 Vietnamese universities and colleges using Bayesian analysis. Globalisation, Societies and Education, 1–19. https://doi.org/10.10 80/14767724.2022.2081536

[28] Prais, S. J. (2003). Cautions on OECD’S Recent Educational Survey (PISA). Oxford Review of Education, 29(2), 139–163. https://doi. org/10.1080/0305498032000080657

[29] Raitano, M., & Vona, F. (2013). Peer heterogeneity, school tracking and students’ performances: evidence from PISA 2006. Applied Economics, 45(32), 4516–4532. https://doi.org/10.1080/00036846.2013.791020

[30] Rochex, J.-Y. (2006). Chapter 5: Social, Methodological, and Theoretical Issues Regarding Assessment: Lessons From a Secondary Analysis of PISA 2000 Literacy Tests. Review of Research in Education, 30(1), 163– 212. https://doi.org/10.3102/0091732X030001163

[31] Rowley, K. J., McNeill, S. M., Dufur, M. J., Edmunds, C., & Jarvis, J. A. (2019). Trends in International PISA Scores over Time: Which Countries Are Actually Improving? Social Sciences, 8(8), 231. https://doi. org/10.3390/socsci8080231

[32] Sato, H. (2017). The Structure of PISA Penetration into Education Policy in Japan and Norway (pp. 209–230). https://doi.org/10.1108/S1479-367920160000031011

[33] Schuelka, M. J. (2013). Excluding students with disabilities from the culture of achievement: the case of the TIMSS, PIRLS, and PISA. Journal of Education Policy, 28(2), 216–230. https://doi.org/10.1080/0268 0939.2012.708789

[34] Stack, M. (2006). Testing, Testing, Read All about It: Canadian Press Coverage of the PISA Results. Canadian Journal of Education / Revue Canadienne de l’éducation, 29(1), 49. https://doi. org/10.2307/20054146

[35] Takayama, K. (2008). The politics of international league tables: PISA in Japan’s achievement crisis debate. Comparative Education, 44(4), 387–407. https://doi. org/10.1080/03050060802481413

[36] UNESCO. (2024). Over half a billion children missing from education data. UNESCO Conference on Education Data and Statistics. https://www.linkedin. com/pulse/over-half-billion-children-missing-fromeducation-data-unesco-rwmhe/

[37] Vasalampi, K., Tolvanen, A., Torppa, M., Poikkeus, A.- M., Hankimaa, H., & Aunola, K. (2023). PISA reading achievement, literacy motivation, and school burnout predicting Adolescents’ educational track and educational attainment. Learning and Individual Differences, 108, 102377. https://doi.org/10.1016/j. lindif.2023.102377

[38] Vik, F. N., Nilsen, T., & Øverby, N. C. (2022). Aspects of nutritional deficits and cognitive outcomes – Triangulation across time and subject domains among students and teachers in TIMSS. International Journal of Educational Development, 89, 102553. https://doi. org/10.1016/j.ijedudev.2022.102553

[39] Vuong, Q.-H. (2018). The (ir)rational consideration of the cost of science in transition economies. Nature Human Behaviour, 2(1), 5–5. https://doi.org/10.1038/ s41562-017-0281-4

[40] Webster, B. J., & Fisher, D. L. (2000). Accounting for Variation in Science and Mathematics Achievement: A Multilevel Analysis of Australian Data Third International Mathematics and Science Study (Timss). School Effectiveness and School Improvement, 11(3), 339–360. https://doi.org/10.1076/0924-3453 (200009)11:3;1-G;FT339

Articles in Issue