[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/oup/jfinec/v21y2023i5p1680-1727..html
   My bibliography  Save this article

A Machine Learning Approach to Volatility Forecasting

Author

Listed:
  • Kim Christensen
  • Mathias Siggaard
  • Bezirgen Veliyev
Abstract
We inspect how accurate machine learning (ML) is at forecasting realized variance of the Dow Jones Industrial Average index constituents. We compare several ML algorithms, including regularization, regression trees, and neural networks, to multiple heterogeneous autoregressive (HAR) models. ML is implemented with minimal hyperparameter tuning. In spite of this, ML is competitive and beats the HAR lineage, even when the only predictors are the daily, weekly, and monthly lags of realized variance. The forecast gains are more pronounced at longer horizons. We attribute this to higher persistence in the ML models, which helps to approximate the long memory of realized variance. ML also excels at locating incremental information about future volatility from additional predictors. Lastly, we propose an ML measure of variable importance based on accumulated local effects. This shows that while there is agreement about the most important predictors, there is disagreement on their ranking, helping to reconcile our results.

Suggested Citation

  • Kim Christensen & Mathias Siggaard & Bezirgen Veliyev, 2023. "A Machine Learning Approach to Volatility Forecasting," Journal of Financial Econometrics, Oxford University Press, vol. 21(5), pages 1680-1727.
    • Handle: RePEc:oup:jfinec:v:21:y:2023:i:5:p:1680-1727.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/jjfinec/nbac020
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Asger Lunde & Peter R. Hansen, 2005. "A forecast comparison of volatility models: does anything beat a GARCH(1,1)?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(7), pages 873-889.
    2. Fernandes, Marcelo & Medeiros, Marcelo C. & Scharth, Marcel, 2014. "Modeling and predicting the CBOE market volatility index," Journal of Banking & Finance, Elsevier, vol. 40(C), pages 1-10.
    3. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    4. Donaldson, R. Glen & Kamstra, Mark, 1997. "An artificial neural network-GARCH model for international stock return volatility," Journal of Empirical Finance, Elsevier, vol. 4(1), pages 17-46, January.
    5. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    6. Christensen, Kim & Thyrsgaard, Martin & Veliyev, Bezirgen, 2019. "The realized empirical distribution function of stochastic variance with application to goodness-of-fit testing," Journal of Econometrics, Elsevier, vol. 212(2), pages 556-583.
    7. Andrew J. Patton & Kevin Sheppard, 2015. "Good Volatility, Bad Volatility: Signed Jumps and The Persistence of Volatility," The Review of Economics and Statistics, MIT Press, vol. 97(3), pages 683-697, July.
    8. Torben G. Andersen & Tim Bollerslev & Francis X. Diebold, 2007. "Roughing It Up: Including Jump Components in the Measurement, Modeling, and Forecasting of Return Volatility," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 701-720, November.
    9. Fulvio Corsi & Roberto Renò, 2012. "Discrete-Time Volatility Forecasting With Persistent Leverage Effect and the Link With Continuous-Time Volatility Modeling," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 368-380, January.
    10. Audrino, Francesco & Corsi, Fulvio, 2010. "Modeling tick-by-tick realized correlations," Computational Statistics & Data Analysis, Elsevier, vol. 54(11), pages 2372-2382, November.
    11. Audrino, Francesco & Sigrist, Fabio & Ballinari, Daniele, 2020. "The impact of sentiment and attention measures on stock market volatility," International Journal of Forecasting, Elsevier, vol. 36(2), pages 334-357.
    12. Daniel W. Apley & Jingyu Zhu, 2020. "Visualizing the effects of predictor variables in black box supervised learning models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(4), pages 1059-1086, September.
    13. Peter Carr & Liuren Wu & Zhibai Zhang, 2019. "Using Machine Learning to Predict Realized Variance," Papers 1909.10035, arXiv.org.
    14. Francesco Audrino & Simon D. Knaus, 2016. "Lassoing the HAR Model: A Model Selection Perspective on Realized Volatility Dynamics," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1485-1521, December.
    15. Massimiliano Caporin & Francesco Poli, 2017. "Building News Measures from Textual Data and an Application to Volatility Forecasting," Econometrics, MDPI, vol. 5(3), pages 1-46, August.
    16. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    17. Fulvio Corsi, 2009. "A Simple Approximate Long-Memory Model of Realized Volatility," Journal of Financial Econometrics, Oxford University Press, vol. 7(2), pages 174-196, Spring.
    18. Scott R. Baker & Nicholas Bloom & Steven J. Davis, 2016. "Measuring Economic Policy Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1593-1636.
    19. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    20. Aruoba, S. BoraÄŸan & Diebold, Francis X. & Scotti, Chiara, 2009. "Real-Time Measurement of Business Conditions," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 417-427.
    21. Busch, Thomas & Christensen, Bent Jesper & Nielsen, Morten Ørregaard, 2011. "The role of implied volatility in forecasting future realized volatility and jumps in foreign exchange, stock, and bond markets," Journal of Econometrics, Elsevier, vol. 160(1), pages 48-57, January.
    22. Robert F. Engle & Eric Ghysels & Bumjean Sohn, 2013. "Stock Market Volatility and Macroeconomic Fundamentals," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 776-797, July.
    23. Chuong Luong & Nikolai Dokuchaev, 2018. "Forecasting of Realised Volatility with the Random Forests Algorithm," JRFM, MDPI, vol. 11(4), pages 1-15, October.
    24. Mittnik, Stefan & Robinzonov, Nikolay & Spindler, Martin, 2015. "Stock market volatility: Identifying major drivers and the nature of their impact," Journal of Banking & Finance, Elsevier, vol. 58(C), pages 1-14.
    25. Bollerslev, Tim & Patton, Andrew J. & Quaedvlieg, Rogier, 2016. "Exploiting the errors: A simple approach for improved volatility forecasting," Journal of Econometrics, Elsevier, vol. 192(1), pages 1-18.
    26. Tim Bollerslev & Jia Li & Yuan Xue, 2018. "Volume, Volatility, and Public News Announcements," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(4), pages 2005-2041.
    27. Andersen, Torben G. & Bollerslev, Tim & Diebold, Francis X. & Ebens, Heiko, 2001. "The distribution of realized stock return volatility," Journal of Financial Economics, Elsevier, vol. 61(1), pages 43-76, July.
    28. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    29. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    30. Eric Hillebrand & Marcelo Medeiros, 2010. "The Benefits of Bagging for Forecast Models of Realized Volatility," Econometric Reviews, Taylor & Francis Journals, vol. 29(5-6), pages 571-593.
    31. Christensen, B. J. & Prabhala, N. R., 1998. "The relation between implied and realized volatility," Journal of Financial Economics, Elsevier, vol. 50(2), pages 125-150, November.
    32. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    33. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Machine Learning for Realized Volatility Forecasting
      by Francis Diebold in No Hesitations on 2021-02-01 12:16:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. M. Shabani & M. Magris & George Tzagkarakis & J. Kanniainen & A. Iosifidis, 2023. "Predicting the state of synchronization of financial time series using cross recurrence plots," Post-Print hal-04415269, HAL.
    2. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    3. Reisenhofer, Rafael & Bayer, Xandro & Hautsch, Nikolaus, 2022. "HARNet: A convolutional neural network for realized volatility forecasting," CFS Working Paper Series 680, Center for Financial Studies (CFS).
    4. Chao Zhang & Yihuang Zhang & Mihai Cucuringu & Zhongmin Qian, 2022. "Volatility forecasting with machine learning and intraday commonality," Papers 2202.08962, arXiv.org, revised Feb 2023.
    5. Francesco Audrino & Jonathan Chassot, 2024. "HARd to Beat: The Overlooked Impact of Rolling Windows in the Era of Machine Learning," Papers 2406.08041, arXiv.org.
    6. Zhang, Hongwei & Zhao, Xinyi & Gao, Wang & Niu, Zibo, 2023. "The role of higher moments in predicting China's oil futures volatility: Evidence from machine learning models," Journal of Commodity Markets, Elsevier, vol. 32(C).
    7. Robert Stok & Paul Bilokon, 2023. "From Deep Filtering to Deep Econometrics," Papers 2311.06256, arXiv.org.
    8. Liao, Cunfei & Ma, Tian, 2024. "From fundamental signals to stock volatility: A machine learning approach," Pacific-Basin Finance Journal, Elsevier, vol. 84(C).
    9. Kaczmarek, Tomasz & Będowska-Sójka, Barbara & Grobelny, Przemysław & Perez, Katarzyna, 2022. "False Safe Haven Assets: Evidence From the Target Volatility Strategy Based on Recurrent Neural Network," Research in International Business and Finance, Elsevier, vol. 60(C).
    10. Guangying Liu & Ziyan Zhuang & Min Wang, 2024. "Forecasting the high‐frequency volatility based on the LSTM‐HIT model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(5), pages 1356-1373, August.
    11. Anubha Goel & Puneet Pasricha & Juho Kanniainen, 2024. "Time-Series Foundation Model for Value-at-Risk," Papers 2410.11773, arXiv.org, revised Dec 2024.
    12. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    13. Timothé Gronier & William Maréchal & Christophe Geissler & Stéphane Gibout, 2022. "Usage of GAMS-Based Digital Twins and Clustering to Improve Energetic Systems Control," Energies, MDPI, vol. 16(1), pages 1-17, December.
    14. Lyócsa, Štefan & Todorova, Neda, 2024. "Forecasting of clean energy market volatility: The role of oil and the technology sector," Energy Economics, Elsevier, vol. 132(C).
    15. Jiawen Luo & Oguzhan Cepni & Riza Demirer & Rangan Gupta, 2022. "Forecasting Multivariate Volatilities with Exogenous Predictors: An Application to Industry Diversification Strategies," Working Papers 202258, University of Pretoria, Department of Economics.
    16. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    17. Borup, Daniel & Rapach, David E. & Schütte, Erik Christian Montes, 2023. "Mixed-frequency machine learning: Nowcasting and backcasting weekly initial claims with daily internet search volume data," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1122-1144.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    2. Chun, Dohyun & Cho, Hoon & Ryu, Doojin, 2023. "Discovering the drivers of stock market volatility in a data-rich world," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 82(C).
    3. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    4. Li, Zhao-Chen & Xie, Chi & Wang, Gang-Jin & Zhu, You & Zeng, Zhi-Jian & Gong, Jue, 2024. "Forecasting global stock market volatilities: A shrinkage heterogeneous autoregressive (HAR) model with a large cross-market predictor set," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 673-711.
    5. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. Chao Zhang & Yihuang Zhang & Mihai Cucuringu & Zhongmin Qian, 2022. "Volatility forecasting with machine learning and intraday commonality," Papers 2202.08962, arXiv.org, revised Feb 2023.
    8. Xue Gong & Weiguo Zhang & Yuan Zhao & Xin Ye, 2023. "Forecasting stock volatility with a large set of predictors: A new forecast combination method," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1622-1647, November.
    9. Cheng, Mingmian & Swanson, Norman R. & Yang, Xiye, 2021. "Forecasting volatility using double shrinkage methods," Journal of Empirical Finance, Elsevier, vol. 62(C), pages 46-61.
    10. Hassanniakalager, Arman & Baker, Paul L. & Platanakis, Emmanouil, 2024. "A False Discovery Rate approach to optimal volatility forecasting model selection," International Journal of Forecasting, Elsevier, vol. 40(3), pages 881-902.
    11. Byounghyun Jeon & Sung Won Seo & Jun Sik Kim, 2020. "Uncertainty and the volatility forecasting power of option‐implied volatility," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 40(7), pages 1109-1126, July.
    12. Audrino, Francesco & Sigrist, Fabio & Ballinari, Daniele, 2020. "The impact of sentiment and attention measures on stock market volatility," International Journal of Forecasting, Elsevier, vol. 36(2), pages 334-357.
    13. Wilms, Ines & Rombouts, Jeroen & Croux, Christophe, 2021. "Multivariate volatility forecasts for stock market indices," International Journal of Forecasting, Elsevier, vol. 37(2), pages 484-499.
    14. Tomáš Plíhal, 2021. "Scheduled macroeconomic news announcements and Forex volatility forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(8), pages 1379-1397, December.
    15. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    16. Zhang, Hongwei & Zhao, Xinyi & Gao, Wang & Niu, Zibo, 2023. "The role of higher moments in predicting China's oil futures volatility: Evidence from machine learning models," Journal of Commodity Markets, Elsevier, vol. 32(C).
    17. Kaminska, Iryna & Roberts-Sklar, Matt, 2018. "Volatility in equity markets and monetary policy rate uncertainty," Journal of Empirical Finance, Elsevier, vol. 45(C), pages 68-83.
    18. Degiannakis, Stavros & Filis, George & Hassani, Hossein, 2018. "Forecasting global stock market implied volatility indices," Journal of Empirical Finance, Elsevier, vol. 46(C), pages 111-129.
    19. Wang Pu & Yixiang Chen & Feng Ma, 2016. "Forecasting the realized volatility in the Chinese stock market: further evidence," Applied Economics, Taylor & Francis Journals, vol. 48(33), pages 3116-3130, July.
    20. Andrew J. Patton & Yasin Simsek, 2023. "Generalized Autoregressive Score Trees and Forests," Papers 2305.18991, arXiv.org.

    More about this item

    Keywords

    accumulated local effect; heterogeneous auto-regression; machine learning; realized variance; volatility forecasting;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:jfinec:v:21:y:2023:i:5:p:1680-1727.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://edirc.repec.org/data/sofieea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.