Comparability and equity in mathematics assessment in PISA 2022 and TIMSS 2023
Keywords:
Comparative education, Educational assessment, Comparability, Equity, MathematicsAbstract
International assessments provide information for educational research, but their results depend on choices concerning populations, knowledge and context. This article examines the conditions for comparability between the 2022 Programme for International Student Assessment and the 2023 Trends in International Mathematics and Science Study, focusing on mathematics. It presents a qualitative documentary study based on a directed reading of official frameworks and a contrast across six analytical categories: purpose, population, construct, knowledge organization, contextual information and scope of inference. The discussion draws on comparative education, the validity of score interpretations and educational equity. The documentary contrast identifies shared attention to reasoning and problem solving alongside differences in student eligibility and domain organization. These differences require researchers to distinguish comparisons of assessment designs from equivalence between results. The article proposes a set of questions to make the conditions and limitations of research using international assessments explicit. It concludes that equity should inform the construction of the comparative object, particularly regarding the population represented and the learning opportunities considered. The proposed framework is interpretive and has not been empirically validated; it does not provide a method for converting scores across programmes.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/uploads/7/6/6/4/76643089/standards_2014edition.pdf
Bartlett, L., & Vavrus, F. (2017). Comparative case studies: An innovative approach. Nordic Journal of Comparative and International Education, 1(1). https://doi.org/10.7577/njcie.1929
Bray, M., & Thomas, R. M. (1995). Levels of comparison in educational studies: Different insights from different literatures and the value of multilevel analyses. Harvard Educational Review, 65(3), 472–491. https://doi.org/10.17763/haer.65.3.g3228437224v4877
Mullis, I. V. S., Martin, M. O., & von Davier, M. (2021). Introduction. In I. V. S. Mullis, M. O. Martin, & M. von Davier (Eds.), TIMSS 2023 assessment frameworks (pp. 1–4). TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2023/frameworks/pdf/T23_Frameworks_Introduction.pdf
OECD. (2023a). PISA 2022 assessment and analytical framework. OECD Publishing. https://doi.org/10.1787/dfe0bf9c-en
OECD. (2023b). PISA 2022 results (Volume I): The state of learning and equity in education. OECD Publishing. https://doi.org/10.1787/53f23881-en
Philpot, R., Lindquist, M., Mullis, I. V. S., & Aldrich, C. E. A. (2021). TIMSS 2023 mathematics framework. In I. V. S. Mullis, M. O. Martin, & M. von Davier (Eds.), TIMSS 2023 assessment frameworks (pp. 5–18). TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2023/frameworks/pdf/T23_Frameworks_Ch1_Mathematics.pdf
Reynolds, K. A., Mullis, I. V. S., & Martin, M. O. (2021). TIMSS 2023 context questionnaire framework. In I. V. S. Mullis, M. O. Martin, & M. von Davier (Eds.), TIMSS 2023 assessment frameworks (pp. 46–70). TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2023/frameworks/pdf/T23_Frameworks_Ch3_Context-Questionnaires.pdf
Silveira, C. L. A. da, & Esquinsani, R. S. S. (2024). A educação comparada como evidência para as políticas de avaliação externa em educação. Cuadernos de Educación y Desarrollo, 16(4), 1–26. https://doi.org/10.55905/cuadv16n4-005
UNESCO. (2017). A guide for ensuring inclusion and equity in education. https://unesdoc.unesco.org/ark:/48223/pf0000248254
