[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v210y2019i1p4-25.html
   My bibliography  Save this article

Sequentially adaptive Bayesian learning algorithms for inference and optimization

Author

Listed:
  • Geweke, John
  • Durham, Garland
Abstract
The sequentially adaptive Bayesian learning algorithm (SABL) builds on and ties together ideas from sequential Monte Carlo and simulated annealing. The algorithm can be used to simulate from Bayesian posterior distributions, using either data tempering or power tempering, or for optimization. A key feature of SABL is that the introduction of information is adaptive and controlled, ensuring that the algorithm performs reliably and efficiently in a wide variety of applications with off-the-shelf settings, minimizing the need for tedious tuning, tinkering, trial and error by users. The algorithm is pleasingly parallel, and a Matlab toolbox implementing the algorithm is able to make efficient use of massively parallel computing environments such as graphics processing units (GPUs) with minimal user effort. This paper describes the algorithm, provides theoretical foundations, applies the algorithm to Bayesian inference and optimization problems illustrating key properties of its operation, and briefly describes the open source software implementation.

Suggested Citation

  • Geweke, John & Durham, Garland, 2019. "Sequentially adaptive Bayesian learning algorithms for inference and optimization," Journal of Econometrics, Elsevier, vol. 210(1), pages 4-25.
    • Handle: RePEc:eee:econom:v:210:y:2019:i:1:p:4-25
    DOI: 10.1016/j.jeconom.2018.11.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407618302021
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2018.11.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pierre Del Moral & Arnaud Doucet & Ajay Jasra, 2006. "Sequential Monte Carlo samplers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 411-436, June.
    2. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    3. Ardia, David & Hoogerheide, Lennart F. & van Dijk, Herman K., 2009. "Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i03).
    4. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
    5. Asger Lunde & Peter R. Hansen, 2005. "A forecast comparison of volatility models: does anything beat a GARCH(1,1)?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(7), pages 873-889.
    6. John Geweke, "undated". "Posterior Simulators in Econometrics," Computing in Economics and Finance 1996 _019, Society for Computational Economics.
    7. Dreze, Jacques H, 1976. "Bayesian Limited Information Analysis of the Simultaneous Equations Model," Econometrica, Econometric Society, vol. 44(5), pages 1045-1075, September.
    8. Garland Durham & John Geweke, 2014. "Improving Asset Price Prediction When All Models are False," Journal of Financial Econometrics, Oxford University Press, vol. 12(2), pages 278-306.
    9. Jin-Chuan Duan & Andras Fulop, 2015. "Density-Tempered Marginalized Sequential Monte Carlo Samplers," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 192-202, April.
    10. Enlu Zhou & Xi Chen, 2013. "Sequential Monte Carlo simulated annealing," Journal of Global Optimization, Springer, vol. 55(1), pages 101-124, January.
    11. Nalan Baştürk & Stefano Grassi & Lennart Hoogerheide & Herman K. Van Dijk, 2016. "Parallelization Experience with Four Canonical Econometric Models Using ParMitISEM," Econometrics, MDPI, vol. 4(1), pages 1-20, March.
    12. Goffe, William L. & Ferrier, Gary D. & Rogers, John, 1994. "Global optimization of statistical functions with simulated annealing," Journal of Econometrics, Elsevier, vol. 60(1-2), pages 65-99.
    13. Fiorentini, Gabriele & Calzolari, Giorgio & Panattoni, Lorenzo, 1996. "Analytic Derivatives and the Computation of GARCH Estimates," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(4), pages 399-417, July-Aug..
    14. Jason R. Blevins, 2016. "Sequential Monte Carlo Methods for Estimating Dynamic Microeconomic Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(5), pages 773-804, August.
    15. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    16. Nicolas Chopin, 2002. "A sequential particle filter method for static models," Biometrika, Biometrika Trust, vol. 89(3), pages 539-552, August.
    17. Drew Creal, 2012. "A Survey of Sequential Monte Carlo Methods for Economics and Finance," Econometric Reviews, Taylor & Francis Journals, vol. 31(3), pages 245-296.
    18. Jasra, Ajay & Doucet, Arnaud & Stephens, David A. & Holmes, Christopher C., 2008. "Interacting sequential Monte Carlo samplers for trans-dimensional simulation," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 1765-1791, January.
    19. Daron Acemoglu & Simon Johnson & James A. Robinson, 2001. "The Colonial Origins of Comparative Development: An Empirical Investigation," American Economic Review, American Economic Association, vol. 91(5), pages 1369-1401, December.
    20. Baştürk, Nalan & Grassi, Stefano & Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2017. "The R Package MitISEM: Efficient and Robust Simulation Procedures for Bayesian Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 79(i01).
    21. Walter R. Gilks & Carlo Berzuini, 2001. "Following a moving target—Monte Carlo inference for dynamic Bayesian models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(1), pages 127-146.
    22. Kleibergen, Frank & van Dijk, Herman K., 1998. "Bayesian Simultaneous Equations Analysis Using Reduced Rank Structures," Econometric Theory, Cambridge University Press, vol. 14(6), pages 701-743, December.
    23. Garland Durham & John Geweke, 2014. "Adaptive Sequential Posterior Simulators for Massively Parallel Computing Environments," Advances in Econometrics, in: Bayesian Model Comparison, volume 34, pages 1-44, Emerald Group Publishing Limited.
    24. Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2012. "A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation," Journal of Econometrics, Elsevier, vol. 171(2), pages 101-120.
    25. Geweke, John, 1996. "Bayesian reduced rank regression in econometrics," Journal of Econometrics, Elsevier, vol. 75(1), pages 121-146, November.
    26. Ardia, David & Hoogerheide, Lennart F. & van Dijk, Herman K., 2009. "Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i03).
    27. Edward Herbst & Frank Schorfheide, 2014. "Sequential Monte Carlo Sampling For Dsge Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(7), pages 1073-1098, November.
    28. John Geweke, 2016. "Sequentially Adaptive Bayesian Learning for a Nonlinear Model of the Secular and Cyclical Behavior of US Real GDP," Econometrics, MDPI, vol. 4(1), pages 1-23, March.
    29. Arnold Zellner & Tomohiro Ando & Nalan Baştük & Lennart Hoogerheide & Herman K. van Dijk, 2014. "Bayesian Analysis of Instrumental Variable Models: Acceptance-Rejection within Direct Monte Carlo," Econometric Reviews, Taylor & Francis Journals, vol. 33(1-4), pages 3-35, June.
    30. Dreze, Jacques H., 1977. "Bayesian regression analysis using poly-t densities," Journal of Econometrics, Elsevier, vol. 6(3), pages 329-354, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barnett, William A. & Bella, Giovanni & Ghosh, Taniya & Mattana, Paolo & Venturi, Beatrice, 2022. "Shilnikov chaos, low interest rates, and New Keynesian macroeconomics," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    2. Li, Yong & Zhang, Mingzhi & Zhang, Yonghui, 2022. "Sequential Bayesian bandwidth selection for multivariate kernel regression with applications," Economic Modelling, Elsevier, vol. 112(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Baştürk, Nalan & Grassi, Stefano & Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2017. "The R Package MitISEM: Efficient and Robust Simulation Procedures for Bayesian Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 79(i01).
    2. Nalan Basturk & Cem Cakmakli & S. Pinar Ceyhan & Herman K. van Dijk, 2014. "On the Rise of Bayesian Econometrics after Cowles Foundation Monographs 10, 14," Tinbergen Institute Discussion Papers 14-085/III, Tinbergen Institute, revised 04 Sep 2014.
    3. Dellaportas, Petros & Tsionas, Mike G., 2019. "Importance sampling from posterior distributions using copula-like approximations," Journal of Econometrics, Elsevier, vol. 210(1), pages 45-57.
    4. Arnaud Dufays, 2016. "Evolutionary Sequential Monte Carlo Samplers for Change-Point Models," Econometrics, MDPI, vol. 4(1), pages 1-33, March.
    5. Nalan Baştürk & Stefano Grassi & Lennart Hoogerheide & Herman K. Van Dijk, 2016. "Parallelization Experience with Four Canonical Econometric Models Using ParMitISEM," Econometrics, MDPI, vol. 4(1), pages 1-20, March.
    6. Markku Lanne & Jani Luoto, 2015. "Estimation of DSGE Models under Diffuse Priors and Data-Driven Identification Constraints," CREATES Research Papers 2015-37, Department of Economics and Business Economics, Aarhus University.
    7. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    8. Herbst, Edward & Schorfheide, Frank, 2019. "Tempered particle filtering," Journal of Econometrics, Elsevier, vol. 210(1), pages 26-44.
    9. Martin Burda & Remi Daviet, 2023. "Hamiltonian sequential Monte Carlo with application to consumer choice behavior," Econometric Reviews, Taylor & Francis Journals, vol. 42(1), pages 54-77, January.
    10. Xiaohong Chen & Timothy M. Christensen & Elie Tamer, 2018. "Monte Carlo Confidence Sets for Identified Sets," Econometrica, Econometric Society, vol. 86(6), pages 1965-2018, November.
    11. Michael Cai & Marco Del Negro & Edward Herbst & Ethan Matlin & Reca Sarfati & Frank Schorfheide, 2021. "Online estimation of DSGE models," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 33-58.
    12. Markku Lanne & Jani Luoto, 2018. "Data†Driven Identification Constraints for DSGE Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 80(2), pages 236-258, April.
    13. Ardia, David & Hoogerheide, Lennart F., 2010. "Efficient Bayesian estimation and combination of GARCH-type models," MPRA Paper 22919, University Library of Munich, Germany.
    14. Mark Bognanni & Edward P. Herbst, 2014. "Estimating (Markov-Switching) VAR Models without Gibbs Sampling: A Sequential Monte Carlo Approach," Working Papers (Old Series) 1427, Federal Reserve Bank of Cleveland.
    15. Nalan Basturk & Cem Cakmakli & S. Pinar Ceyhan & Herman K. van Dijk, 2013. "Historical Developments in Bayesian Econometrics after Cowles Foundation Monographs 10, 14," Tinbergen Institute Discussion Papers 13-191/III, Tinbergen Institute.
    16. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    17. John Geweke, 2016. "Sequentially Adaptive Bayesian Learning for a Nonlinear Model of the Secular and Cyclical Behavior of US Real GDP," Econometrics, MDPI, vol. 4(1), pages 1-23, March.
    18. Nomen Nescio, 2013. "Nomen Nescio," Tinbergen Institute Discussion Papers 12-095 not issued, Tinbergen Institute.
    19. Bognanni, Mark & Zito, John, 2020. "Sequential Bayesian inference for vector autoregressions with stochastic volatility," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
    20. Casarin, Roberto & Grassi, Stefano & Ravazzolo, Francesco & van Dijk, Herman K., 2023. "A flexible predictive density combination for large financial data sets in regular and crisis periods," Journal of Econometrics, Elsevier, vol. 237(2).

    More about this item

    Keywords

    Sequential Monte Carlo; Simulated annealing; Posterior simulation; Bayesian learning; Parallel computing; Particle filtering;
    All these keywords.

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:210:y:2019:i:1:p:4-25. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.