[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20202408.html
   My bibliography  Save this paper

Random forest versus logit models: which offers better early warning of fiscal stress?

Author

Listed:
  • Jarmulska, Barbara
Abstract
This study seeks to answer whether it is possible to design an early warning system framework that can signal the risk of fiscal stress in the near future, and what shape such a system should take. To do so, multiple models based on econometric logit and the random forest models are designed and compared. Using a dataset of 20 annual frequency variables pertaining to 43 advanced and emerging countries during 1992-2018, the results confirm the possibility of obtaining an effective model, which correctly predicts 70-80% of fiscal stress events and tranquil periods. The random forest-based early warning model outperforms logit models. While the random forest model is commonly understood to provide lower interpretability than logit models do, this study employs tools that can be used to provide useful information for understanding what is behind the black-box. These tools can provide information on the most important explanatory variables and on the shape of the relationship between these variables and the outcome classification. Thus, the study contributes to the discussion on the usefulness of machine learning methods in economics. JEL Classification: C40, C53, H63, G01

Suggested Citation

  • Jarmulska, Barbara, 2020. "Random forest versus logit models: which offers better early warning of fiscal stress?," Working Paper Series 2408, European Central Bank.
    • Handle: RePEc:ecb:ecbwps:20202408
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp2408~aa6b05aed7.en.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    2. Ciarlone, Alessio & Trebeschi, Giorgio, 2005. "Designing an early warning system for debt crises," Emerging Markets Review, Elsevier, vol. 6(4), pages 376-395, December.
    3. repec:zbw:bofitp:2011_002 is not listed on IDEAS
    4. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    5. Manasse, Paolo & Roubini, Nouriel, 2009. ""Rules of thumb" for sovereign debt crises," Journal of International Economics, Elsevier, vol. 78(2), pages 192-205, July.
    6. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    7. Rupa Duttagupta & Mr. Paul Cashin, 2008. "The Anatomy of Banking Crises," IMF Working Papers 2008/093, International Monetary Fund.
    8. Benedict S. Jimenez, 2017. "Institutional Constraints, Rule-Following, and Circumvention: Tax and Expenditure Limits and the Choice of Fiscal Tools During a Budget Crisis," Public Budgeting & Finance, Wiley Blackwell, vol. 37(2), pages 5-34, June.
    9. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    10. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    11. Mr. Axel Schimmelpfennig & Nouriel Roubini & Paolo Manasse, 2003. "Predicting Sovereign Debt Crises," IMF Working Papers 2003/221, International Monetary Fund.
    12. Demirgüç-Kunt, Asli & Detragiache, Enrica, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, National Institute of Economic and Social Research, vol. 192, pages 68-83, April.
    13. Bluwstein, Kristina & Buckmann, Marcus & Joseph, Andreas & Kapadia, Sujit & Şimşek, Özgür, 2023. "Credit growth, the yield curve and financial crisis prediction: Evidence from a machine learning approach," Journal of International Economics, Elsevier, vol. 145(C).
    14. Fuertes, Ana-Maria & Kalotychou, Elena, 2006. "Early warning systems for sovereign debt crises: The role of heterogeneity," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1420-1441, November.
    15. Juan J. Cruces & Christoph Trebesch, 2013. "Sovereign Defaults: The Price of Haircuts," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(3), pages 85-117, July.
    16. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    17. David Beers & Jean-Sébastien Nadeau, 2014. "Database of Sovereign Defaults, 2015 (Revised May 2015)," Technical Reports 101, Bank of Canada.
    18. Miss Gabriela Dobrescu & Iva Petrova & Nazim Belhocine & Mr. Emanuele Baldacci, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    19. Barrell, Ray & Davis, E. Philip & Karim, Dilruba & Liadze, Iana, 2010. "Bank regulation, property prices and early warning systems for banking crises in OECD countries," Journal of Banking & Finance, Elsevier, vol. 34(9), pages 2255-2264, September.
    20. Pablo Hernández de Cos & Enrique Moral-Benito & Gerrit B. Koester & Christiane Nickel, 2014. "Signalling fiscal stress in the euro area: A country-specific early warning system," Working Papers 1418, Banco de España.
    21. Moritz Schularick & Alan M. Taylor, 2012. "Credit Booms Gone Bust: Monetary Policy, Leverage Cycles, and Financial Crises, 1870-2008," American Economic Review, American Economic Association, vol. 102(2), pages 1029-1061, April.
    22. Marco Lo Duca & Tuomas Peltonen, 2011. "Macrofinancial vulnerabilities and future financial stress: assessing systemic risks and predicting systemic events," BIS Papers chapters, in: Bank for International Settlements (ed.), Macroprudential regulation and policy, volume 60, pages 82-88, Bank for International Settlements.
    23. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Is the 2007 US Sub-Prime Financial Crisis So Different?: An International Historical Comparison," Panoeconomicus, Savez ekonomista Vojvodine, Novi Sad, Serbia, vol. 56(3), pages 291-299.
    24. Kar‐yiu Wong & Richard Y. K. Ho, 2002. "The Asian Crisis, 1997," Review of International Economics, Wiley Blackwell, vol. 10(1), pages 1-1, February.
    25. Kumar, Mohan & Moorthy, Uma & Perraudin, William, 2003. "Predicting emerging market currency crashes," Journal of Empirical Finance, Elsevier, vol. 10(4), pages 427-454, September.
    26. Fioramanti, Marco, 2008. "Predicting sovereign debt crises using artificial neural networks: A comparative approach," Journal of Financial Stability, Elsevier, vol. 4(2), pages 149-164, June.
    27. Carmen M. Reinhart & Kenneth S. Rogoff, 2011. "From Financial Crash to Debt Crisis," American Economic Review, American Economic Association, vol. 101(5), pages 1676-1706, August.
    28. Demyanyk, Yuliya & Hasan, Iftekhar, 2010. "Financial crises and bank failures: A review of prediction methods," Omega, Elsevier, vol. 38(5), pages 315-324, October.
    29. Fuertes, Ana-Maria & Kalotychou, Elena, 2007. "Optimal design of early warning systems for sovereign debt crises," International Journal of Forecasting, Elsevier, vol. 23(1), pages 85-100.
    30. Markus Holopainen & Peter Sarlin, 2017. "Toward robust early-warning models: a horse race, ensembles and model uncertainty," Quantitative Finance, Taylor & Francis Journals, vol. 17(12), pages 1933-1963, December.
    31. David T. Mitchell & Dean Stansel, 2016. "The Determinants of the Severity of State Fiscal Crises," Public Budgeting & Finance, Wiley Blackwell, vol. 36(4), pages 50-67, December.
    32. E. Philip Davis & Dilruba Karim, 2008. "Could Early Warning Systems Have Helped To Predict the Sub-Prime Crisis?," National Institute Economic Review, National Institute of Economic and Social Research, vol. 206(1), pages 35-47, October.
    33. Katia Berti & Matteo Salto & Matthieu Lequien, 2012. "An early-detection index of fiscal stress for EU countries," European Economy - Economic Papers 2008 - 2015 475, Directorate General Economic and Financial Affairs (DG ECFIN), European Commission.
    34. Christoph Trebesch & Mr. Michael G. Papaioannou & Mr. Udaibir S Das, 2012. "Sovereign Debt Restructurings 1950-2010: Literature Survey, Data, and Stylized Facts," IMF Working Papers 2012/203, International Monetary Fund.
    35. Graciela Laura Kaminsky, 1999. "Currency and Banking Crises: The Early Warnings of Distress," IMF Working Papers 1999/178, International Monetary Fund.
    36. repec:zbw:bofrdp:2009_035 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Donato Ceci & Andrea Silvestrini, 2023. "Nowcasting the state of the Italian economy: The role of financial markets," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1569-1593, November.
    2. Valencia, Oscar & Parra, Diego A. & Díaz, Juan Camilo, 2022. "Assessing Macro-Fiscal Risk for Latin American and Caribbean Countries," IDB Publications (Working Papers) 12482, Inter-American Development Bank.
    3. Raffaele Marchi & Alessandro Moro, 2024. "Forecasting Fiscal Crises in Emerging Markets and Low-Income Countries with Machine Learning Models," Open Economies Review, Springer, vol. 35(1), pages 189-213, February.
    4. Jiaming Liu & Chengzhang Li & Peng Ouyang & Jiajia Liu & Chong Wu, 2023. "Interpreting the prediction results of the tree‐based gradient boosting models for financial distress prediction with an explainable machine learning approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(5), pages 1112-1137, August.
    5. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    6. Moreno Badia, Marialuz & Medas, Paulo & Gupta, Pranav & Xiang, Yuan, 2022. "Debt is not free," Journal of International Money and Finance, Elsevier, vol. 127(C).
    7. Jorge M. Uribe, 2023. ""Fiscal crises and climate change"," IREA Working Papers 202303, University of Barcelona, Research Institute of Applied Economics, revised Feb 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    2. Moreno Badia, Marialuz & Medas, Paulo & Gupta, Pranav & Xiang, Yuan, 2022. "Debt is not free," Journal of International Money and Finance, Elsevier, vol. 127(C).
    3. Tamás Kristóf, 2021. "Sovereign Default Forecasting in the Era of the COVID-19 Crisis," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    4. Casabianca, Elizabeth Jane & Catalano, Michele & Forni, Lorenzo & Giarda, Elena & Passeri, Simone, 2022. "A machine learning approach to rank the determinants of banking crises over time and across countries," Journal of International Money and Finance, Elsevier, vol. 129(C).
    5. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    6. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    7. Stijn Claessens & M. Ayhan Kose, 2013. "Financial Crises: Explanations, Types and Implications," CAMA Working Papers 2013-06, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    8. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    9. Betz, Frank & Oprică, Silviu & Peltonen, Tuomas A. & Sarlin, Peter, 2014. "Predicting distress in European banks," Journal of Banking & Finance, Elsevier, vol. 45(C), pages 225-241.
    10. Bitetto, Alessandro & Cerchiello, Paola & Mertzanis, Charilaos, 2023. "Measuring financial soundness around the world: A machine learning approach," International Review of Financial Analysis, Elsevier, vol. 85(C).
    11. Mathonnat, Clément & Minea, Alexandru, 2018. "Financial development and the occurrence of banking crises," Journal of Banking & Finance, Elsevier, vol. 96(C), pages 344-354.
    12. Huynh, Tran & Uebelmesser, Silke, 2024. "Early warning models for systemic banking crises: Can political indicators improve prediction?," European Journal of Political Economy, Elsevier, vol. 81(C).
    13. Fendel Ralf & Stremmel Hanno, 2016. "Characteristics of Banking Crises: A Comparative Study with Geographical Contagion," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(3), pages 349-388, May.
    14. Honda, Jiro & Tapsoba, René & Issifou, Ismael, 2022. "When do we repair the roof? Insights from responses to fiscal crisis early warning signals," International Economics, Elsevier, vol. 172(C), pages 349-367.
    15. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    16. Boysen-Hogrefe, Jens & Gern, Klaus-Jürgen & Groll, Dominik & Jannsen, Nils & Kooths, Stefan & Plödt, Martin & van Roye, Björn & Scheide, Joachim & Schwarzmüller, Tim, 2015. "Das europäische Verfahren zur Vermeidung und Korrektur makroökonomischer Ungleichgewichte: Auswertung der bisherigen Erfahrung und mögliche Reformansätze," Kieler Beiträge zur Wirtschaftspolitik 7, Kiel Institute for the World Economy (IfW Kiel).
    17. Rho, Caterina & Saenz, Manrique, 2021. "Financial stress and the probability of sovereign default," Journal of International Money and Finance, Elsevier, vol. 110(C).
    18. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    19. Hamdaoui, Mekki, 2016. "Are systemic banking crises in developed and developing countries predictable?," Journal of Multinational Financial Management, Elsevier, vol. 37, pages 114-138.
    20. Fu, Junhui & Zhou, Qingling & Liu, Yufang & Wu, Xiang, 2020. "Predicting stock market crises using daily stock market valuation and investor sentiment indicators," The North American Journal of Economics and Finance, Elsevier, vol. 51(C).

    More about this item

    Keywords

    early warning system; interpretability of machine learning; predictive performance;
    All these keywords.

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • H63 - Public Economics - - National Budget, Deficit, and Debt - - - Debt; Debt Management; Sovereign Debt
    • G01 - Financial Economics - - General - - - Financial Crises

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20202408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.