[go: up one dir, main page]

IDEAS home Printed from https://ideas.repec.org/p/cir/cirwor/2009s-45.html
   My bibliography  Save this paper

On Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models

Author

Listed:
  • Sébastien Laurent
  • Jeroen Rombouts
  • Francesco Violente
Abstract
A large number of parameterizations have been proposed to model conditional variance dynamics in a multivariate framework. However, little is known about the ranking of multivariate volatility models in terms of their forecasting ability. The ranking of multivariate volatility models is inherently problematic because it requires the use of a proxy for the unobservable volatility matrix and this substitution may severely affect the ranking. We address this issue by investigating the properties of the ranking with respect to alternative statistical loss functions used to evaluate model performances. We provide conditions on the functional form of the loss function that ensure the proxy-based ranking to be consistent for the true one - i.e., the ranking that would be obtained if the true variance matrix was observable. We identify a large set of loss functions that yield a consistent ranking. In a simulation study, we sample data from a continuous time multivariate diffusion process and compare the ordering delivered by both consistent and inconsistent loss functions. We further discuss the sensitivity of the ranking to the quality of the proxy and the degree of similarity between models. An application to three foreign exchange rates, where we compare the forecasting performance of 16 multivariate GARCH specifications, is provided. Un grand nombre de méthodes de paramétrage ont été proposées dans le but de modéliser la dynamique de la variance conditionnelle dans un cadre multivarié. Toutefois, on connaît peu de choses sur le classement des modèles de volatilité multivariés, du point de vue de leur capacité à permettre de faire des prédictions. Le classement des modèles de volatilité multivariés est forcément problématique du fait qu'il requiert l'utilisation d'une valeur substitutive pour la matrice de la volatilité non observable et cette substitution peut influencer sérieusement le classement. Nous abordons ce problème en examinant les propriétés du classement en relation avec les fonctions de perte statistiques alternatives utilisées pour évaluer la performance des modèles. Nous présentons des conditions liées à la forme fonctionnelle de la fonction de perte qui garantissent que le classement fondé sur une valeur de substitution est constant par rapport au classement réel, c'est-à-dire à celui qui serait obtenu si la matrice de variance réelle était observable. Nous établissons un vaste ensemble de fonctions de perte qui produisent un classement constant. Dans le cadre d'une étude par simulation, nous fournissons un échantillon de données à partir d'un processus de diffusion multivarié en temps continu et comparons l'ordre généré par les fonctions de perte constantes et inconstantes. Nous approfondissons la question de la sensibilité du classement à la qualité de la substitution et le degré de ressemblance entre les modèles. Une application à trois taux de change est proposée et, dans ce contexte, nous comparons l'efficacité de prédiction de 16 paramètres du modèle GARCH multivarié (approche d'hétéroscédasticité conditionnelle autorégressive généralisée).

Suggested Citation

  • Sébastien Laurent & Jeroen Rombouts & Francesco Violente, 2009. "On Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models," CIRANO Working Papers 2009s-45, CIRANO.
  • Handle: RePEc:cir:cirwor:2009s-45
    as

    Download full text from publisher

    File URL: https://cirano.qc.ca/files/publications/2009s-45.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Engle, Robert F. & Kroner, Kenneth F., 1995. "Multivariate Simultaneous Generalized ARCH," Econometric Theory, Cambridge University Press, vol. 11(1), pages 122-150, February.
    2. Weide, R. van der, 2002. "Generalized Orthogonal GARCH. A Multivariate GARCH model," CeNDEF Working Papers 02-02, Universiteit van Amsterdam, Center for Nonlinear Dynamics in Economics and Finance.
    3. Nour Meddahi, 2002. "A theoretical comparison between integrated and realized volatility," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 17(5), pages 479-508.
    4. Diebold, Francis X & Nerlove, Marc, 1989. "The Dynamics of Exchange Rate Volatility: A Multivariate Latent Factor Arch Model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 4(1), pages 1-21, Jan.-Mar..
    5. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    6. Torben G. Andersen & Tim Bollerslev & Nour Meddahi, 2005. "Correcting the Errors: Volatility Forecast Evaluation Using High-Frequency Data and Realized Volatilities," Econometrica, Econometric Society, vol. 73(1), pages 279-296, January.
    7. Hansen, Peter Reinhard & Lunde, Asger, 2006. "Consistent ranking of volatility models," Journal of Econometrics, Elsevier, vol. 131(1-2), pages 97-121.
    8. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    9. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    10. Ole E. Barndorff-Nielsen & Peter Reinhard Hansen & Asger Lunde & Neil Shephard, 2008. "Designing Realized Kernels to Measure the ex post Variation of Equity Prices in the Presence of Noise," Econometrica, Econometric Society, vol. 76(6), pages 1481-1536, November.
    11. Bollerslev, Tim & Russell, Jeffrey & Watson, Mark (ed.), 2010. "Volatility and Time Series Econometrics: Essays in Honor of Robert Engle," OUP Catalogue, Oxford University Press, number 9780199549498.
    12. Andersen, Torben G & Bollerslev, Tim, 1998. "Answering the Skeptics: Yes, Standard Volatility Models Do Provide Accurate Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 885-905, November.
    13. Barndorff-Nielsen, Ole E. & Hansen, Peter Reinhard & Lunde, Asger & Shephard, Neil, 2011. "Multivariate realised kernels: Consistent positive semi-definite estimators of the covariation of equity prices with noise and non-synchronous trading," Journal of Econometrics, Elsevier, vol. 162(2), pages 149-169, June.
    14. Bauwens, Luc & Lubrano, Michel & Richard, Jean-Francois, 2000. "Bayesian Inference in Dynamic Econometric Models," OUP Catalogue, Oxford University Press, number 9780198773139.
    15. Michael W. Brandt & Francis X. Diebold, 2006. "A No-Arbitrage Approach to Range-Based Estimation of Return Covariances and Correlations," The Journal of Business, University of Chicago Press, vol. 79(1), pages 61-74, January.
    16. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    17. Gourieroux, Christian & Monfort, Alain & Trognon, Alain, 1984. "Pseudo Maximum Likelihood Methods: Theory," Econometrica, Econometric Society, vol. 52(3), pages 681-700, May.
    18. Ole E. Barndorff-Nielsen & Neil Shephard, 2001. "Econometric Analysis of Realised Covariation: High Frequency Covariance, Regression and Correlation in Financial Economics," Economics Papers 2002-W13, Economics Group, Nuffield College, University of Oxford, revised 18 Mar 2002.
    19. Torben G. Andersen & Tim Bollerslev & Francis X. Diebold & Paul Labys, 2003. "Modeling and Forecasting Realized Volatility," Econometrica, Econometric Society, vol. 71(2), pages 579-625, March.
    20. Carol Alexander, 2000. "Orthogonal Methods for Generating Large Positive Semi-Definite Covariance Matrices," ICMA Centre Discussion Papers in Finance icma-dp2000-06, Henley Business School, University of Reading.
    21. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    22. repec:oxf:wpaper:264 is not listed on IDEAS
    23. Asger Lunde & Peter R. Hansen, 2005. "A forecast comparison of volatility models: does anything beat a GARCH(1,1)?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(7), pages 873-889.
    24. Colacito, Riccardo & Engle, Robert F. & Ghysels, Eric, 2011. "A component model for dynamic correlations," Journal of Econometrics, Elsevier, vol. 164(1), pages 45-59, September.
    25. Valeri Voev & Asger Lunde, 2007. "Integrated Covariance Estimation using High-frequency Data in the Presence of Noise," Journal of Financial Econometrics, Oxford University Press, vol. 5(1), pages 68-104.
    26. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    27. Davidson, James, 2004. "Moment and Memory Properties of Linear Conditional Heteroscedasticity Models, and a New Model," Journal of Business & Economic Statistics, American Statistical Association, vol. 22(1), pages 16-29, January.
    28. Ledoit, Olivier & Wolf, Michael, 2003. "Improved estimation of the covariance matrix of stock returns with an application to portfolio selection," Journal of Empirical Finance, Elsevier, vol. 10(5), pages 603-621, December.
    29. Bollerslev, Tim, 1990. "Modelling the Coherence in Short-run Nominal Exchange Rates: A Multivariate Generalized ARCH Model," The Review of Economics and Statistics, MIT Press, vol. 72(3), pages 498-505, August.
    30. Gourieroux,Christian & Monfort,Alain, 1995. "Statistics and Econometric Models," Cambridge Books, Cambridge University Press, number 9780521405515, September.
    31. Hansen, Peter R. & Lunde, Asger, 2006. "Realized Variance and Market Microstructure Noise," Journal of Business & Economic Statistics, American Statistical Association, vol. 24, pages 127-161, April.
    32. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    33. Barone-Adesi, Giovanni & Rasmussen, Henrik & Ravanelli, Claudia, 2005. "An option pricing formula for the GARCH diffusion model," Computational Statistics & Data Analysis, Elsevier, vol. 49(2), pages 287-310, April.
    34. Andersen T. G & Bollerslev T. & Diebold F. X & Labys P., 2001. "The Distribution of Realized Exchange Rate Volatility," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 42-55, March.
    35. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    36. Nelson, Daniel B, 1991. "Conditional Heteroskedasticity in Asset Returns: A New Approach," Econometrica, Econometric Society, vol. 59(2), pages 347-370, March.
    37. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    38. Ding, Zhuanxin & Granger, Clive W. J. & Engle, Robert F., 1993. "A long memory property of stock market returns and a new model," Journal of Empirical Finance, Elsevier, vol. 1(1), pages 83-106, June.
    39. Luc Bauwens & Sébastien Laurent & Jeroen V. K. Rombouts, 2006. "Multivariate GARCH models: a survey," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(1), pages 79-109, January.
    40. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    41. Zhou, Bin, 1996. "High-Frequency Data and Volatility in Foreign-Exchange Rates," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 45-52, January.
    42. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-350, July.
    43. Torben G. Andersen & Tim Bollerslev & Francis X. Diebold & Paul Labys, 1999. "(Understanding, Optimizing, Using and Forecasting) Realized Volatility and Correlation," New York University, Leonard N. Stern School Finance Department Working Paper Seires 99-061, New York University, Leonard N. Stern School of Business-.
    44. Meddahi, N., 2001. "A Theoretical Comparison Between Integrated and Realized Volatilies," Cahiers de recherche 2001-26, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
    45. Roy van der Weide, 2002. "GO-GARCH: a multivariate generalized orthogonal GARCH model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 17(5), pages 549-564.
    46. Jeffrey R. Russell & Federico M. Bandi, 2004. "Microstructure noise, realized volatility, and optimal sampling," Econometric Society 2004 Latin American Meetings 220, Econometric Society.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sébastien Laurent & Jeroen V. K. Rombouts & Francesco Violante, 2012. "On the forecasting accuracy of multivariate GARCH models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 934-955, September.
    2. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    3. Torben G. Andersen & Tim Bollerslev & Peter F. Christoffersen & Francis X. Diebold, 2005. "Volatility Forecasting," PIER Working Paper Archive 05-011, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    4. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2013. "Financial Risk Measurement for Financial Risk Management," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1127-1220, Elsevier.
    5. Caporin, Massimiliano & McAleer, Michael, 2014. "Robust ranking of multivariate GARCH models by problem dimension," Computational Statistics & Data Analysis, Elsevier, vol. 76(C), pages 172-185.
    6. Patton, Andrew J., 2011. "Data-based ranking of realised volatility estimators," Journal of Econometrics, Elsevier, vol. 161(2), pages 284-303, April.
    7. Liu, Lily Y. & Patton, Andrew J. & Sheppard, Kevin, 2015. "Does anything beat 5-minute RV? A comparison of realized measures across multiple asset classes," Journal of Econometrics, Elsevier, vol. 187(1), pages 293-311.
    8. Bu, Ruijun & Hizmeri, Rodrigo & Izzeldin, Marwan & Murphy, Anthony & Tsionas, Mike, 2023. "The contribution of jump signs and activity to forecasting stock price volatility," Journal of Empirical Finance, Elsevier, vol. 70(C), pages 144-164.
    9. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    10. Li, Jia & Patton, Andrew J., 2018. "Asymptotic inference about predictive accuracy using high frequency data," Journal of Econometrics, Elsevier, vol. 203(2), pages 223-240.
    11. LAURENT, Sébastien & VIOLANTE, Francesco, 2012. "Volatility forecasts evaluation and comparison," LIDAM Reprints CORE 2414, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    12. Massimiliano Caporin & Michael McAleer, 2011. "Ranking Multivariate GARCH Models by Problem Dimension: An Empirical Evaluation," Working Papers in Economics 11/23, University of Canterbury, Department of Economics and Finance.
    13. Andrea BUCCI, 2017. "Forecasting Realized Volatility A Review," Journal of Advanced Studies in Finance, ASERS Publishing, vol. 8(2), pages 94-138.
    14. Hansen, Peter Reinhard & Lunde, Asger, 2006. "Consistent ranking of volatility models," Journal of Econometrics, Elsevier, vol. 131(1-2), pages 97-121.
    15. Patton, Andrew J. & Sheppard, Kevin, 2009. "Optimal combinations of realised volatility estimators," International Journal of Forecasting, Elsevier, vol. 25(2), pages 218-238.
    16. Roxana Halbleib & Valeri Voev, 2016. "Forecasting Covariance Matrices: A Mixed Approach," Journal of Financial Econometrics, Oxford University Press, vol. 14(2), pages 383-417.
    17. Sucarrat, Genaro, 2009. "Forecast Evaluation of Explanatory Models of Financial Variability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 3, pages 1-33.
    18. Roxana Halbleib & Valeri Voev, 2011. "Forecasting Covariance Matrices: A Mixed Frequency Approach," CREATES Research Papers 2011-03, Department of Economics and Business Economics, Aarhus University.
    19. McAleer, Michael & Medeiros, Marcelo C., 2008. "A multiple regime smooth transition Heterogeneous Autoregressive model for long memory and asymmetries," Journal of Econometrics, Elsevier, vol. 147(1), pages 104-119, November.
    20. Wang Pu & Yixiang Chen & Feng Ma, 2016. "Forecasting the realized volatility in the Chinese stock market: further evidence," Applied Economics, Taylor & Francis Journals, vol. 48(33), pages 3116-3130, July.

    More about this item

    Keywords

    Volatility; multivariate GARCH; matrix norm; loss function; model confidence set; Volatilité; modèle GARCH multivarié; norme matricielle; fonction de perte; ensemble de modèles de confiance.;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cir:cirwor:2009s-45. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Webmaster (email available below). General contact details of provider: https://edirc.repec.org/data/ciranca.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.