[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/wly/japmet/v29y2014i3p415-430.html
   My bibliography  Save this article

Tests Of Equal Forecast Accuracy For Overlapping Models

Author

Listed:
  • Todd E. Clark
  • Michael W. Mccracken
Abstract
SUMMARY This paper examines the asymptotic and finite‐sample properties of tests of equal forecast accuracy when the models being compared are overlapping in the sense of Vuong (Econometrica 1989; 57 : 307–333). Two models are overlapping when the true model contains just a subset of variables common to the larger sets of variables included in the competing forecasting models. We consider an out‐of‐sample version of the two‐step testing procedure recommended by Vuong but also show that an exact one‐step procedure is sometimes applicable. When the models are overlapping, we provide a simple‐to‐use fixed‐regressor wild bootstrap that can be used to conduct valid inference. Monte Carlo simulations generally support the theoretical results: the two‐step procedure is conservative, while the one‐step procedure can be accurately sized when appropriate. We conclude with an empirical application comparing the predictive content of credit spreads to growth in real stock prices for forecasting US real gross domestic product growth. Copyright © 2013 John Wiley & Sons, Ltd.

Suggested Citation

  • Todd E. Clark & Michael W. Mccracken, 2014. "Tests Of Equal Forecast Accuracy For Overlapping Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(3), pages 415-430, April.
    • Handle: RePEc:wly:japmet:v:29:y:2014:i:3:p:415-430
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    2. Todd Clark & Michael McCracken, 2012. "Reality Checks and Comparisons of Nested Predictive Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 53-66.
    3. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    4. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 8(4), pages 489-500, December.
    5. Faust, Jon & Rogers, John H & Wright, Jonathan H, 2005. "News and Noise in G-7 GDP Announcements," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 403-419, June.
    6. Naes, Randi & Skjeltorp, Johannes & Odegaard, Bernt Arne, 2008. "Liquidity and the Business Cycle," UiS Working Papers in Economics and Finance 2009/1, University of Stavanger.
    7. Fabio Fornari & Antonio Mele, 2013. "Financial Volatility and Economic Activity," Journal of Financial Management, Markets and Institutions, Società editrice il Mulino, issue 2, pages 155-198, December.
    8. Yongmiao Hong & Tae-Hwy Lee, 2003. "Inference on Predictability of Foreign Exchange Rates via Generalized Spectrum and Nonlinear Time Series Models," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 1048-1062, November.
    9. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    10. Randi Næs & Johannes A. Skjeltorp & Bernt Arne Ødegaard, 2011. "Stock Market Liquidity and the Business Cycle," Journal of Finance, American Finance Association, vol. 66(1), pages 139-176, February.
    11. Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, vol. 104(2), pages 315-358, September.
    12. Robert Rich & Jason Bram & Andrew Haughwout & James Orr & Rae Rosen & Rebecca Sela, 2005. "Using Regional Economic Indexes to Forecast Tax Bases: Evidence from New York," The Review of Economics and Statistics, MIT Press, vol. 87(4), pages 627-634, November.
    13. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    14. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    15. Wright, Jonathan H. & Zhou, Hao, 2009. "Bond risk premia and realized jump risk," Journal of Banking & Finance, Elsevier, vol. 33(12), pages 2333-2345, December.
    16. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    17. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    18. Wegener, Christian & von Nitzsch, Rüdiger & Cengiz, Cetin, 2010. "An advanced perspective on the predictability in hedge fund returns," Journal of Banking & Finance, Elsevier, vol. 34(11), pages 2694-2708, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Firmin Doko Tchatoka & Qazi Haque, 2023. "On bootstrapping tests of equal forecast accuracy for nested models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1844-1864, November.
    2. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    3. Brent Meyer & Saeed Zaman, 2013. "It’s not just for inflation: The usefulness of the median CPI in BVAR forecasting," Working Papers (Old Series) 1303, Federal Reserve Bank of Cleveland.
    4. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2023. "Out-of-sample tests for conditional quantile coverage an application to Growth-at-Risk," Journal of Econometrics, Elsevier, vol. 236(2).
    5. T. S. McElroy, 2016. "Nonnested model comparisons for time series," Biometrika, Biometrika Trust, vol. 103(4), pages 905-914.
    6. Isao Ishida & Virmantas Kvedaras, 2015. "Modeling Autoregressive Processes with Moving-Quantiles-Implied Nonlinearity," Econometrics, MDPI, vol. 3(1), pages 1-53, January.
    7. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2024. "Predictive ability tests with possibly overlapping models," Journal of Econometrics, Elsevier, vol. 241(1).
    8. Mayer, Walter J. & Liu, Feng & Dang, Xin, 2017. "Improving the power of the Diebold–Mariano–West test for least squares predictions," International Journal of Forecasting, Elsevier, vol. 33(3), pages 618-626.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mayer, Walter J. & Liu, Feng & Dang, Xin, 2017. "Improving the power of the Diebold–Mariano–West test for least squares predictions," International Journal of Forecasting, Elsevier, vol. 33(3), pages 618-626.
    2. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    3. Clark, Todd E. & McCracken, Michael W., 2009. "Tests of Equal Predictive Ability With Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 441-454.
    4. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.
    5. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
    6. Florackis, Chris & Giorgioni, Gianluigi & Kostakis, Alexandros & Milas, Costas, 2014. "On stock market illiquidity and real-time GDP growth," Journal of International Money and Finance, Elsevier, vol. 44(C), pages 210-229.
    7. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    8. Firmin Doko Tchatoka & Qazi Haque, 2023. "On bootstrapping tests of equal forecast accuracy for nested models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1844-1864, November.
    9. Granziera, Eleonora & Hubrich, Kirstin & Moon, Hyungsik Roger, 2014. "A predictability test for a small number of nested models," Journal of Econometrics, Elsevier, vol. 182(1), pages 174-185.
    10. Jean-Yves Pitarakis, 2020. "A Novel Approach to Predictive Accuracy Testing in Nested Environments," Papers 2008.08387, arXiv.org, revised Oct 2023.
    11. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    12. Kirstin Hubrich & Kenneth D. West, 2010. "Forecast evaluation of small nested model sets," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 574-594.
    13. Todd E. Clark & Michael W. McCracken, 2001. "Evaluating long-horizon forecasts," Research Working Paper RWP 01-14, Federal Reserve Bank of Kansas City.
    14. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2024. "Predictive ability tests with possibly overlapping models," Journal of Econometrics, Elsevier, vol. 241(1).
    15. Gonçalves, Sílvia & McCracken, Michael W. & Perron, Benoit, 2017. "Tests of equal accuracy for nested models with estimated factors," Journal of Econometrics, Elsevier, vol. 198(2), pages 231-252.
    16. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    17. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    18. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    19. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    20. Pablo Pincheira Brown & Nicolás Hardy, 2024. "Correlation‐based tests of predictability," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(6), pages 1835-1858, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:japmet:v:29:y:2014:i:3:p:415-430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.interscience.wiley.com/jpages/0883-7252/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.