Pensémoslo de nuevo: : ¿Podemos comparar las escalas de antecedentes socioeconómicos?

Andrés Sandoval-Hernández, Rutkowski David, Tyler Mata, Daniel Miranda

Research output: Contribution to journalArticle

Abstract

Using data from international large-scale assessments (ILSA), we evaluate the issue of country-level model-data consistency of background socio-economic scales, as well as the invariance across countries. To that end, we use data from PISA, TERCE, and TIMSS, as they operationalize socio-economic status somewhat differently. As part of our analysis, we examine whether TERCE, a Latin American study – with measures that are regionally developed – exhibits better psychometric properties than measures that are designed to function across a larger and more diverse number of educational systems. We also examine TIMSS, a trends focused study – that has historically emphasized consistency and comparison. Finally, we include PISA which has the largest number of participants and has changed and conceptualized a great deal of its background questionnaire depending on the study’s major domain and focus. Our findings suggest that none of the socioeconomic background scales we analyzed are fully invariant in any of the three studies, and therefore comparisons across countries should be done with caution. The different levels of equivalence reached by each scale in each study and the type of comparisons that can be made given these results (e.g., comparison of average scale scores, comparison of relationships between the tested scales and other variables) are discussed in the full paper.
Translated title of the contribution Pensémoslo de nuevo: : ¿Podemos comparar las escalas de antecedentes socioeconómicos?
LanguageSpanish
Pages37-61
Number of pages25
JournalRevista de Educacion
Issue number383
DOIs
StatusPublished - 31 Jan 2019

Keywords

  • measurement invariance
  • measurement equivalence
  • TERCE
  • TIMSS
  • PISA
  • multi-group confirmatory factor analysis
  • socio-economic scales

Cite this

Back to the drawing board: Can we compare background scales? / Sandoval-Hernández, Andrés; David, Rutkowski; Mata, Tyler; Miranda, Daniel.

In: Revista de Educacion, No. 383, 31.01.2019, p. 37-61.

Research output: Contribution to journalArticle

Sandoval-Hernández, Andrés ; David, Rutkowski ; Mata, Tyler ; Miranda, Daniel. / Back to the drawing board: Can we compare background scales?. In: Revista de Educacion. 2019 ; No. 383. pp. 37-61.
@article{2829aa705f48427392dd6d86eef60f27,
title = "Back to the drawing board:: Can we compare background scales?",
abstract = "Using data from international large-scale assessments (ILSA), we evaluate the issue of country-level model-data consistency of background socio-economic scales, as well as the invariance across countries. To that end, we use data from PISA, TERCE, and TIMSS, as they operationalize socio-economic status somewhat differently. As part of our analysis, we examine whether TERCE, a Latin American study – with measures that are regionally developed – exhibits better psychometric properties than measures that are designed to function across a larger and more diverse number of educational systems. We also examine TIMSS, a trends focused study – that has historically emphasized consistency and comparison. Finally, we include PISA which has the largest number of participants and has changed and conceptualized a great deal of its background questionnaire depending on the study’s major domain and focus. Our findings suggest that none of the socioeconomic background scales we analyzed are fully invariant in any of the three studies, and therefore comparisons across countries should be done with caution. The different levels of equivalence reached by each scale in each study and the type of comparisons that can be made given these results (e.g., comparison of average scale scores, comparison of relationships between the tested scales and other variables) are discussed in the full paper.",
keywords = "measurement invariance, measurement equivalence, TERCE, TIMSS, PISA, multi-group confirmatory factor analysis, socio-economic scales",
author = "Andr{\'e}s Sandoval-Hern{\'a}ndez and Rutkowski David and Tyler Mata and Daniel Miranda",
year = "2019",
month = "1",
day = "31",
doi = "10.4438/1988-592X-RE-2019-383-400",
language = "Spanish",
pages = "37--61",
journal = "Revista de Educacion",
issn = "1853-1318",
number = "383",

}

TY - JOUR

T1 - Back to the drawing board:

T2 - Revista de Educacion

AU - Sandoval-Hernández, Andrés

AU - David, Rutkowski

AU - Mata, Tyler

AU - Miranda, Daniel

PY - 2019/1/31

Y1 - 2019/1/31

N2 - Using data from international large-scale assessments (ILSA), we evaluate the issue of country-level model-data consistency of background socio-economic scales, as well as the invariance across countries. To that end, we use data from PISA, TERCE, and TIMSS, as they operationalize socio-economic status somewhat differently. As part of our analysis, we examine whether TERCE, a Latin American study – with measures that are regionally developed – exhibits better psychometric properties than measures that are designed to function across a larger and more diverse number of educational systems. We also examine TIMSS, a trends focused study – that has historically emphasized consistency and comparison. Finally, we include PISA which has the largest number of participants and has changed and conceptualized a great deal of its background questionnaire depending on the study’s major domain and focus. Our findings suggest that none of the socioeconomic background scales we analyzed are fully invariant in any of the three studies, and therefore comparisons across countries should be done with caution. The different levels of equivalence reached by each scale in each study and the type of comparisons that can be made given these results (e.g., comparison of average scale scores, comparison of relationships between the tested scales and other variables) are discussed in the full paper.

AB - Using data from international large-scale assessments (ILSA), we evaluate the issue of country-level model-data consistency of background socio-economic scales, as well as the invariance across countries. To that end, we use data from PISA, TERCE, and TIMSS, as they operationalize socio-economic status somewhat differently. As part of our analysis, we examine whether TERCE, a Latin American study – with measures that are regionally developed – exhibits better psychometric properties than measures that are designed to function across a larger and more diverse number of educational systems. We also examine TIMSS, a trends focused study – that has historically emphasized consistency and comparison. Finally, we include PISA which has the largest number of participants and has changed and conceptualized a great deal of its background questionnaire depending on the study’s major domain and focus. Our findings suggest that none of the socioeconomic background scales we analyzed are fully invariant in any of the three studies, and therefore comparisons across countries should be done with caution. The different levels of equivalence reached by each scale in each study and the type of comparisons that can be made given these results (e.g., comparison of average scale scores, comparison of relationships between the tested scales and other variables) are discussed in the full paper.

KW - measurement invariance

KW - measurement equivalence

KW - TERCE

KW - TIMSS

KW - PISA

KW - multi-group confirmatory factor analysis

KW - socio-economic scales

U2 - 10.4438/1988-592X-RE-2019-383-400

DO - 10.4438/1988-592X-RE-2019-383-400

M3 - Article

SP - 37

EP - 61

JO - Revista de Educacion

JF - Revista de Educacion

SN - 1853-1318

IS - 383

ER -