[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2405.02161.html
   My bibliography  Save this paper

Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling

Author

Listed:
  • Simone Brusatin
  • Tommaso Padoan
  • Andrea Coletta
  • Domenico Delli Gatti
  • Aldo Glielmo
Abstract
Agent-based models (ABMs) are simulation models used in economics to overcome some of the limitations of traditional frameworks based on general equilibrium assumptions. However, agents within an ABM follow predetermined 'bounded rational' behavioural rules which can be cumbersome to design and difficult to justify. Here we leverage multi-agent reinforcement learning (RL) to expand the capabilities of ABMs with the introduction of 'fully rational' agents that learn their policy by interacting with the environment and maximising a reward function. Specifically, we propose a 'Rational macro ABM' (R-MABM) framework by extending a paradigmatic macro ABM from the economic literature. We show that gradually substituting ABM firms in the model with RL agents, trained to maximise profits, allows for studying the impact of rationality on the economy. We find that RL agents spontaneously learn three distinct strategies for maximising profits, with the optimal strategy depending on the level of market competition and rationality. We also find that RL agents with independent policies, and without the ability to communicate with each other, spontaneously learn to segregate into different strategic groups, thus increasing market power and overall profits. Finally, we find that a higher number of rational (RL) agents in the economy always improves the macroeconomic environment as measured by total output. Depending on the specific rational policy, this can come at the cost of higher instability. Our R-MABM framework allows for stable multi-agent learning, is available in open source, and represents a principled and robust direction to extend economic simulators.

Suggested Citation

  • Simone Brusatin & Tommaso Padoan & Andrea Coletta & Domenico Delli Gatti & Aldo Glielmo, 2024. "Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling," Papers 2405.02161, arXiv.org, revised Oct 2024.
    • Handle: RePEc:arx:papers:2405.02161
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2405.02161
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tohid Atashbar & Rui Aruhan Shi, 2022. "Deep Reinforcement Learning: Emerging Trends in Macroeconomics and Future Prospects," IMF Working Papers 2022/259, International Monetary Fund.
    2. Leigh Tesfatsion & Kenneth L. Judd (ed.), 2006. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 2, number 2.
    3. Hinterlang, Natascha & Tänzer, Alina, 2021. "Optimal monetary policy using reinforcement learning," Discussion Papers 51/2021, Deutsche Bundesbank.
    4. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    5. Blume, Lawrence & Easley, David & Kleinberg, Jon & Kleinberg, Robert & Tardos, Éva, 2015. "Introduction to computer science and economic theory," Journal of Economic Theory, Elsevier, vol. 156(C), pages 1-13.
    6. J. Doyne Farmer & Duncan Foley, 2009. "The economy needs agent-based modelling," Nature, Nature, vol. 460(7256), pages 685-686, August.
    7. Farmer, J. Doyne & Axtell, Robert L., 2022. "Agent-Based Modeling in Economics and Finance: Past, Present, and Future," INET Oxford Working Papers 2022-10, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    8. Assenza, Tiziana & Delli Gatti, Domenico & Grazzini, Jakob, 2015. "Emergent dynamics of a macroeconomic agent based model with capital and credit," Journal of Economic Dynamics and Control, Elsevier, vol. 50(C), pages 5-28.
    9. Delli Gatti, Domenico & Grazzini, Jakob, 2020. "Rising to the challenge: Bayesian estimation and forecasting techniques for macroeconomic Agent Based Models," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 875-902.
    10. Artem Kuriksha, 2021. "An Economy of Neural Networks:Learning from Heterogeneous Experiences," PIER Working Paper Archive 21-027, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    11. Maskin, Eric & Tirole, Jean, 2001. "Markov Perfect Equilibrium: I. Observable Actions," Journal of Economic Theory, Elsevier, vol. 100(2), pages 191-219, October.
    12. Edward Hill & Marco Bardoscia & Arthur Turrell, 2021. "Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning," Papers 2103.16977, arXiv.org.
    13. Artem Kuriksha, 2021. "An Economy of Neural Networks: Learning from Heterogeneous Experiences," Papers 2110.11582, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kshama Dwarakanath & Svitlana Vyetrenko & Tucker Balch, 2024. "Empirical Equilibria in Agent-based Economic systems with Learning agents," Papers 2408.12038, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tommaso Ciarli & André Lorentz & Marco Valente & Maria Savona, 2019. "Structural changes and growth regimes," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 119-176, March.
    2. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    3. Guerini, Mattia & Napoletano, Mauro & Roventini, Andrea, 2018. "No man is an Island: The impact of heterogeneity and local interactions on macroeconomic dynamics," Economic Modelling, Elsevier, vol. 68(C), pages 82-95.
    4. Aldo Glielmo & Marco Favorito & Debmallya Chanda & Domenico Delli Gatti, 2023. "Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs," Papers 2302.11835, arXiv.org, revised Dec 2023.
    5. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2018. "Faraway, So Close: Coupled Climate and Economic Dynamics in an Agent-based Integrated Assessment Model," Ecological Economics, Elsevier, vol. 150(C), pages 315-339.
    6. Popoyan, Lilit & Napoletano, Mauro & Roventini, Andrea, 2020. "Winter is possibly not coming: Mitigating financial instability in an agent-based model with interbank market," Journal of Economic Dynamics and Control, Elsevier, vol. 117(C).
    7. Kshama Dwarakanath & Svitlana Vyetrenko & Tucker Balch, 2024. "Empirical Equilibria in Agent-based Economic systems with Learning agents," Papers 2408.12038, arXiv.org.
    8. Ciola, Emanuele & Turco, Enrico & Gurgone, Andrea & Bazzana, Davide & Vergalli, Sergio & Menoncin, Francesco, 2023. "Enter the MATRIX model:a Multi-Agent model for Transition Risks with application to energy shocks," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    9. Delli Gatti,Domenico & Fagiolo,Giorgio & Gallegati,Mauro & Richiardi,Matteo & Russo,Alberto (ed.), 2018. "Agent-Based Models in Economics," Cambridge Books, Cambridge University Press, number 9781108400046, September.
    10. Kshama Dwarakanath & Svitlana Vyetrenko & Peyman Tavallali & Tucker Balch, 2024. "ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents," Papers 2402.09563, arXiv.org.
    11. Kshama Dwarakanath & Jialin Dong & Svitlana Vyetrenko, 2024. "Tax Credits and Household Behavior: The Roles of Myopic Decision-Making and Liquidity in a Simulated Economy," Papers 2408.10391, arXiv.org, revised Oct 2024.
    12. Kirill S. Glavatskiy & Mikhail Prokopenko & Adrian Carro & Paul Ormerod & Michael Harré, 2021. "Explaining herding and volatility in the cyclical price dynamics of urban housing markets using a large-scale agent-based model," SN Business & Economics, Springer, vol. 1(6), pages 1-21, June.
    13. Ciola, Emanuele & Turco, Enrico & Gurgone, Andrea & Bazzana, Davide & Vergalli, Sergio & Menoncin, Francesco, 2022. "Charging the macroeconomy with an energy sector: an agent-based model," FEEM Working Papers 319877, Fondazione Eni Enrico Mattei (FEEM).
    14. Conor B. Hamill & Raad Khraishi & Simona Gherghel & Jerrard Lawrence & Salvatore Mercuri & Ramin Okhrati & Greig A. Cowan, 2023. "Agent-based Modelling of Credit Card Promotions," Papers 2311.01901, arXiv.org, revised Nov 2023.
    15. J. Silvestre, & T. Araújo & M. St. Aubyn, 2016. "Economic growth and individual satisfaction in an agent-based economy," Working Papers Department of Economics 2016/19, ISEG - Lisbon School of Economics and Management, Department of Economics, Universidade de Lisboa.
    16. Balint, T. & Lamperti, F. & Mandel, A. & Napoletano, M. & Roventini, A. & Sapio, A., 2017. "Complexity and the Economics of Climate Change: A Survey and a Look Forward," Ecological Economics, Elsevier, vol. 138(C), pages 252-265.
    17. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Alessandro Sapio, 2018. "And then he wasn't a she : Climate change and green transitions in an agent-based integrated assessment model," Working Papers hal-03443464, HAL.
    18. Zhang, Hui & Cao, Libin & Zhang, Bing, 2017. "Emissions trading and technology adoption: An adaptive agent-based analysis of thermal power plants in China," Resources, Conservation & Recycling, Elsevier, vol. 121(C), pages 23-32.
    19. Poledna, Sebastian & Thurner, Stefan & Farmer, J. Doyne & Geanakoplos, John, 2014. "Leverage-induced systemic risk under Basle II and other credit risk policies," Journal of Banking & Finance, Elsevier, vol. 42(C), pages 199-212.
    20. Cristiano CODAGNONE & Giovanni LIVA & Egidijus BARCEVICIUS & Gianluca MISURACA & Luka KLIMAVICIUTE & Michele BENEDETTI & Irene VANINI & Giancarlo VECCHI & Emily RYEN GLOINSON & Katherine STEWART & Sti, 2020. "Assessing the impacts of digital government transformation in the EU: Conceptual framework and empirical case studies," JRC Research Reports JRC120865, Joint Research Centre.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2405.02161. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.