[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/7612.html
   My bibliography  Save this paper

Is random forest a superior methodology for predicting poverty ? an empirical assessment

Author

Listed:
  • Pave Sohnesen,Thomas
  • Stender,Niels
Abstract
Random forest is in many fields of research a common method for data driven predictions. Within economics and prediction of poverty, random forest is rarely used. Comparing out-of-sample predictions in surveys for same year in six countries shows that random forest is often more accurate than current common practice (multiple imputations with variables selected by stepwise and Lasso), suggesting that this method could contribute to better poverty predictions. However, none of the methods consistently provides accurate predictions of poverty over time, highlighting that technical model fitting by any method within a single year is not always, by itself, sufficient for accurate predictions of poverty over time.

Suggested Citation

  • Pave Sohnesen,Thomas & Stender,Niels, 2016. "Is random forest a superior methodology for predicting poverty ? an empirical assessment," Policy Research Working Paper Series 7612, The World Bank.
    • Handle: RePEc:wbk:wbrwps:7612
    as

    Download full text from publisher

    File URL: http://documents.worldbank.org/curated/en/777401467987858907/pdf/WPS7612.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    2. Kasirye, Ibrahim & Ssewanyana, Sarah N., 2010. "Impacts and determinants of panel survey attrition: The case of Northern Uganda survey 2004-2008," Research Series 127536, Economic Policy Research Centre (EPRC).
    3. Sudarno Sumarto & Daniel Suryadarma & Asep Suryahadi, 2007. "Predicting Consumption Poverty using Non-Consumption Indicators: Experiments using Indonesian Data," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 81(3), pages 543-578, May.
    4. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    5. Chris Elbers & Jean O. Lanjouw & Peter Lanjouw, 2003. "Micro--Level Estimation of Poverty and Inequality," Econometrica, Econometric Society, vol. 71(1), pages 355-364, January.
    6. Astrid Mathiassen, 2013. "Testing Prediction Performance of Poverty Models: Empirical Evidence from U ganda," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 59(1), pages 91-112, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gianni Betti & Ruzhdie Bici & Laura Neri & Thomas Pave Sohnesen & Ledia Thomo, 2018. "Local Poverty and Inequality in Albania," Eastern European Economics, Taylor & Francis Journals, vol. 56(3), pages 223-245, May.
    2. Ratzanyel Rincón, 2023. "Quarterly multidimensional poverty estimates in Mexico using machine learning algorithms/Estimaciones trimestrales de pobreza multidimensional en México mediante algoritmos de aprendizaje de máquina," Estudios Económicos, El Colegio de México, Centro de Estudios Económicos, vol. 38(1), pages 3-68.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Pave Sohnesen & Niels Stender, 2017. "Is Random Forest a Superior Methodology for Predicting Poverty? An Empirical Assessment," Poverty & Public Policy, John Wiley & Sons, vol. 9(1), pages 118-133, March.
    2. Dang,Hai-Anh H. & Kilic,Talip & Carletto,Calogero & Abanokova,Kseniya, 2021. "Poverty Imputation in Contexts without Consumption Data : A Revisit with Further Refinements," Policy Research Working Paper Series 9838, The World Bank.
    3. Brown, Caitlin & Ravallion, Martin & van de Walle, Dominique, 2018. "A poor means test? Econometric targeting in Africa," Journal of Development Economics, Elsevier, vol. 134(C), pages 109-124.
    4. Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    5. Ligon, Ethan & Christiaensen, Luc & Sohnesen, Thomas P, 2020. "Should Consumption Sub-Aggregates be Used to Measure Poverty?," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt9b9929jh, Department of Agricultural & Resource Economics, UC Berkeley.
    6. Cuesta, Jose & Chagalj, Cristian, 2019. "Measuring poverty with administrative data in data deprived contexts: The case of Nicaragua," Economics Letters, Elsevier, vol. 183(C), pages 1-1.
    7. Dang, Hai-Anh H & Kilic, Talip & Hlasny, Vladimir & Abanokova, Kseniya & Carletto, Calogero, 2024. "Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment," IZA Discussion Papers 16792, Institute of Labor Economics (IZA).
    8. Paolo Brunori & Pedro Salas-Rojo & Paolo Verme, 2022. "Estimating Inequality with Missing Incomes," Working Papers 616, ECINEQ, Society for the Study of Economic Inequality.
    9. Betti, Gianni & Molini, Vasco & Mori, Lorenzo, 2024. "An attempt to correct the underestimation of inequality measures in cross-survey imputation through generalized additive models for location, scale and shape," Socio-Economic Planning Sciences, Elsevier, vol. 91(C).
    10. Paolo Verme, 2020. "Which Model for Poverty Predictions?," Working Papers 521, ECINEQ, Society for the Study of Economic Inequality.
    11. Adama BAH, 2013. "Finding the Best Indicators to Identify the Poor," Working Papers 201324, CERDI.
    12. Astrid Mathiassen & Bjørn K. Wold, 2019. "Challenges in predicting poverty trends using survey to survey imputation. Experiences from Malawi," Discussion Papers 900, Statistics Norway, Research Department.
    13. Dang, Hai-Anh H & Lanjouw, Peter F., 2021. "Data Scarcity and Poverty Measurement," IZA Discussion Papers 14631, Institute of Labor Economics (IZA).
    14. Betti,Gianni & Molini,Vasco & Mori,Lorenzo, 2022. "New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation : An Attemptto Correct the Underestimation of Extreme Values," Policy Research Working Paper Series 10013, The World Bank.
    15. Hai-Anh H. Dang & Peter F. Lanjouw & Umar Serajuddin, 2017. "Updating poverty estimates in the absence of regular and comparable consumption data: methods and illustration with reference to a middle-income country," Oxford Economic Papers, Oxford University Press, vol. 69(4), pages 939-962.
    16. Sophie-Charlotte Klose & Johannes Lederer, 2020. "A Pipeline for Variable Selection and False Discovery Rate Control With an Application in Labor Economics," Papers 2006.12296, arXiv.org, revised Jun 2020.
    17. Bijlsma Ineke & van den Brakel Jan & van der Velden Rolf & Allen Jim, 2020. "Estimating Literacy Levels at a Detailed Regional Level: an Application Using Dutch Data," Journal of Official Statistics, Sciendo, vol. 36(2), pages 251-274, June.
    18. John A. Maluccio, 2009. "Household targeting in practice: The Nicaraguan Red de Protección Social," Journal of International Development, John Wiley & Sons, Ltd., vol. 21(1), pages 1-23.
    19. Claudio A. Agostini & Philip H. Brown, 2010. "Local Distributional Effects Of Government Cash Transfers In Chile," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 56(2), pages 366-388, June.
    20. Patrick Bajari & Victor Chernozhukov & Ali Hortaçsu & Junichi Suzuki, 2019. "The Impact of Big Data on Firm Performance: An Empirical Investigation," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 33-37, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:7612. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.