How reliable are the socioeconomic measures used in PISA data?

Socioeconomic deprivation has been shown to be associated with poor educational outcomes. Using deprivation indicators generated from surveys and administrative datasets, we can analyse how students from diverse socioeconomic backgrounds engage in education on a local, national, and international level. In recent decades, attempts have been made to develop national and cross-national indices that reflect the impact of student’s socioeconomic background characteristics on a variety of aspects of teaching, learning, and educational attainment. However, efforts to establish a useful and reliable global instrument have failed to produce a widely accepted standard index for use across demographically similar countries, let alone states with dramatically different economic and social characteristics. The OECD has historically released reports on inequality and poverty using some deprivation measures. One of these is the PISA survey which uses a specially created technique based on self-reporting of household possessions to ascertain participants' socioeconomic status. This data informs the PISA reports' conclusions and contributes to the creation of policy recommendations aimed at closing achievement disparities for socially disadvantaged children in schools. The purpose of this article is to examine the reliability and cross-national comparability of socioeconomic disadvantage metrics included in PISA. We argue that the measures used to assign social class on PISA are imprecise, excessively simplistic, and prone to reporting bias, and that the conclusions obtained from them are also dubious. While making recommendations for cross-national SES measures is outside the scope of this paper, we conclude by examining the eligibility for free school meals in England which, despite being a proxy SES indicator, is more trustworthy than some of the deprivation measures used in PISA.