[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/qss/dqsswp/1704.html
   My bibliography  Save this paper

What Happens When Econometrics and Psychometrics Collide? An Example Using PISA Data

Author

Listed:
  • John Jerrim

    (Department of Social Science, UCL Institute of Education, University College London)

  • Luis Alejandro Lopez-Agudo

    (Departamento de Economía Aplicada (Estadística y Econometría). Facultad de Ciencias Económicas y Empresariales. Universidad de Málaga)

  • Oscar D. Marcenaro-Gutierrez

    (Departamento de Economía Aplicada (Estadística y Econometría). Facultad de Ciencias Económicas y Empresariales. Universidad de Málaga)

  • Nikki Shure

    (Department of Social Science, UCL Institute of Education and Institute of Labor Economics)

Abstract
International large-scale assessments such as PISA are increasingly being used to benchmark the academic performance of young people across the world. Yet many of the technicalities underpinning these datasets are miss-understood by applied researchers, who sometimes fail to take into account their complex survey and test designs. The aim of this paper is to generate a better understanding amongst economists about how such databases are created, and what this implies for the empirical methodologies one should or should not apply. In doing so, we explain how some of the modelling strategies preferred by economists is at odds with the design of these studies. In doing so, we hope to generate a better understanding of international large-scale education datasets, and promote better practice in their use.

Suggested Citation

  • John Jerrim & Luis Alejandro Lopez-Agudo & Oscar D. Marcenaro-Gutierrez & Nikki Shure, 2017. "What Happens When Econometrics and Psychometrics Collide? An Example Using PISA Data," DoQSS Working Papers 17-04, Quantitative Social Science - UCL Social Research Institute, University College London.
    • Handle: RePEc:qss:dqsswp:1704
    as

    Download full text from publisher

    File URL: https://repec.ucl.ac.uk/REPEc/pdf/qsswp1704.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Andreas Ammermueller, 2007. "Poor Background or Low Returns? Why Immigrant Students in Germany Perform so Poorly in the Programme for International Student Assessment," Education Economics, Taylor & Francis Journals, vol. 15(2), pages 215-230.
    2. Barbara Mueller & Stefan Wolter, 2014. "The role of hard-to-obtain information on ability for the school-to-work transition," Empirical Economics, Springer, vol. 46(4), pages 1447-1471, June.
    3. Eric A. Hanushek & Ludger Wössmann, 2006. "Does Educational Tracking Affect Performance and Inequality? Differences- in-Differences Evidence Across Countries," Economic Journal, Royal Economic Society, vol. 116(510), pages 63-76, March.
    4. Herrero, Carmen & Mendez, Ildefonso & Villar, Antonio, 2014. "Analysis of group performance with categorical data when agents are heterogeneous: The evaluation of scholastic performance in the OECD through PISA," Economics of Education Review, Elsevier, vol. 40(C), pages 140-151.
    5. David Kiss, 2013. "Are immigrants and girls graded worse? Results of a matching approach," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 447-463, December.
    6. S. Mahuteau & K. Mavromaras, 2014. "An analysis of the impact of socio-economic disadvantage and school quality on the probability of school dropout," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 389-411, August.
    7. Asako Ohinata & Jan C. van Ours, 2013. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," Economic Journal, Royal Economic Society, vol. 0, pages 308-331, August.
    8. Maciej Jakubowski & Harry Anthony Patrinos & Emilio Ernesto Porta & Jerzy Wiśniewski, 2016. "The effects of delaying tracking in secondary school: evidence from the 1999 education reform in Poland," Education Economics, Taylor & Francis Journals, vol. 24(6), pages 557-572, November.
    9. Nonoyama-Tarumi, Yuko & Willms, J. Douglas, 2010. "The relative and absolute risks of disadvantaged family background and low levels of school resources on student literacy," Economics of Education Review, Elsevier, vol. 29(2), pages 214-224, April.
    10. Lant Pritchett & Martina Viarengo, 2015. "Does public sector control reduce variance in school quality?," Education Economics, Taylor & Francis Journals, vol. 23(5), pages 557-576, October.
    11. Lounkaew, Kiatanantha, 2013. "Explaining urban–rural differences in educational achievement in Thailand: Evidence from PISA literacy data," Economics of Education Review, Elsevier, vol. 37(C), pages 213-225.
    12. Marina Murat & Patrizio Frederic, 2015. "Institutions, culture and background: the school performance of immigrant students," Education Economics, Taylor & Francis Journals, vol. 23(5), pages 612-630, October.
    13. Stefanie Hof, 2014. "Does private tutoring work? The effectiveness of private tutoring: a nonparametric bounds analysis," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 347-366, August.
    14. Harry Anthony Patrinos, 2013. "Private education provision and public finance: the Netherlands," Education Economics, Taylor & Francis Journals, vol. 21(4), pages 392-414, September.
    15. Tommaso Agasisti, 2013. "The efficiency of Italian secondary schools and the potential role of competition: a data envelopment analysis using OECD-PISA2006 data," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 520-544, December.
    16. Cain Polidano & Barbara Hanel & Hielke Buddelmeyer, 2013. "Explaining the socio-economic status school completion gap," Education Economics, Taylor & Francis Journals, vol. 21(3), pages 230-247, July.
    17. Andreas Ammermueller, 2007. "PISA: What makes the difference?," Empirical Economics, Springer, vol. 33(2), pages 263-287, September.
    18. Ohinata, Asako & van Ours, Jan C., 2011. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," IZA Discussion Papers 6212, Institute of Labor Economics (IZA).
    19. Veruska Oppedisano & Gilberto Turati, 2015. "What are the causes of educational inequality and of its evolution over time in Europe? Evidence from PISA," Education Economics, Taylor & Francis Journals, vol. 23(1), pages 3-24, February.
    20. Jensen, Peter & Rasmussen, Astrid Würtz, 2011. "The effect of immigrant concentration in schools on native and immigrant children's reading and math skills," Economics of Education Review, Elsevier, vol. 30(6), pages 1503-1515.
    21. Micklewright, John & Schnepf, Sylke V. & Silva, Pedro N., 2012. "Peer effects and measurement error: The impact of sampling variation in school survey data (evidence from PISA)," Economics of Education Review, Elsevier, vol. 31(6), pages 1136-1142.
    22. John Micklewright & Sylke V. Schnepf & Pedro N. Silva, 2010. "Peer effects and measurement error: the impact of sampling variation in school survey data," DoQSS Working Papers 10-13, Quantitative Social Science - UCL Social Research Institute, University College London.
    23. Polidano, Cain & Tabasso, Domenico, 2014. "Making it real: The benefits of workplace learning in upper-secondary vocational education and training courses," Economics of Education Review, Elsevier, vol. 42(C), pages 130-146.
    24. Maciej Jakubowski, 2015. "Latent variables and propensity score matching: a simulation study with application to data from the Programme for International Student Assessment in Poland," Empirical Economics, Springer, vol. 48(3), pages 1287-1325, May.
    25. Jan van Ours, 2008. "When do children read books?," Education Economics, Taylor & Francis Journals, vol. 16(4), pages 313-328.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giannelli, Gianna Claudia & Rapallini, Chiara, 2016. "Immigrant student performance in Math: Does it matter where you come from?," Economics of Education Review, Elsevier, vol. 52(C), pages 291-304.
    2. Maria Cattaneo & Stefan Wolter, 2015. "Better migrants, better PISA results: Findings from a natural experiment," IZA Journal of Migration and Development, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-19, December.
    3. Bonin, Holger, 2017. "The Potential Economic Benefits of Education of Migrants in the EU," IZA Research Reports 75, Institute of Labor Economics (IZA).
    4. Jens Ruhose, 2015. "Microeconometric Analyses on Economic Consequences of Selective Migration," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 61.
    5. José M. Cordero & Víctor Cristóbal & Daniel Santín, 2018. "Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls," Journal of Economic Surveys, Wiley Blackwell, vol. 32(3), pages 878-915, July.
    6. De Paola, Maria & Brunello, Giorgio, 2016. "Education as a Tool for the Economic Integration of Migrants," IZA Discussion Papers 9836, Institute of Labor Economics (IZA).
    7. Clifton-Sprigg, Joanna, 2014. "Educational spillovers and parental migration," 2007 Annual Meeting, July 29-August 1, 2007, Portland, Oregon TN 2015-46, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    8. Cattaneo, Maria Alejandra & Wolter, Stefan C., 2012. "Migration Policy Can Boost PISA Results: Findings from a Natural Experiment," IZA Discussion Papers 6300, Institute of Labor Economics (IZA).
    9. Julia Bredtmann & Sebastian Otten & Christina Vonnahme, 2021. "Linguistic diversity in the classroom, student achievement, and social integration," Education Economics, Taylor & Francis Journals, vol. 29(2), pages 121-142, March.
    10. Murat Marina, 2012. "Do Immigrant Students Succeed? Evidence from Italy and France," Global Economy Journal, De Gruyter, vol. 12(3), pages 1-22, September.
    11. Seah, Kelvin, 2016. "The Impact of Immigrant Peers on Native Students' Academic Achievement in Countries Where Parents of Immigrants Are Relatively Skilled," IZA Discussion Papers 10065, Institute of Labor Economics (IZA).
    12. Ohinata, A. & van Ours, J.C., 2013. "Spillover Effects of Studying with Immigrant Students : A Quantile Regression Approach," Discussion Paper 2013-058, Tilburg University, Center for Economic Research.
    13. Björn NILSSON, 2019. "Education and migration: insights for policymakers," Working Paper 23ca9c54-061a-4d60-967c-f, Agence française de développement.
    14. David Figlio & Umut Özek, 2019. "Unwelcome Guests? The Effects of Refugees on the Educational Outcomes of Incumbent Students," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1061-1096.
    15. Timothy M. Diette & Ruth Uwaifo Oyelere, 2017. "Do limited English students jeopardize the education of other students? Lessons from the North Carolina public school system," Education Economics, Taylor & Francis Journals, vol. 25(5), pages 446-461, September.
    16. Ong C. & Witte K. de, 2013. "The influence of ethnic segregation and school mobility in primary education on high school dropout : evidence from regression discontinuity at a contextual tipping point," MERIT Working Papers 2013-064, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    17. Frattini, Tommaso & Meschi, Elena, 2019. "The effect of immigrant peers in vocational schools," European Economic Review, Elsevier, vol. 113(C), pages 1-22.
    18. Sweetman, A. & van Ours, J.C., 2014. "Immigration : What About the Children and Grandchildren?," Discussion Paper 2014-009, Tilburg University, Center for Economic Research.
    19. Facundo Albornoz & Antonio Cabrales & Esther Hauk, 2018. "Immigration and the school system," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(4), pages 855-890, June.
    20. Brunello, Giorgio & Rocco, Lorenzo, 2013. "The effect of immigration on the school performance of natives: Cross country evidence using PISA test scores," Economics of Education Review, Elsevier, vol. 32(C), pages 234-246.

    More about this item

    Keywords

    Survey design; Test design; PISA; Weights; Replicate weights; Plausible values;
    All these keywords.

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qss:dqsswp:1704. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dr Neus Bover Fonts (email available below). General contact details of provider: https://edirc.repec.org/data/dqioeuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.