[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2304.00364.html
   My bibliography  Save this paper

Mastering Pair Trading with Risk-Aware Recurrent Reinforcement Learning

Author

Listed:
  • Weiguang Han
  • Jimin Huang
  • Qianqian Xie
  • Boyi Zhang
  • Yanzhao Lai
  • Min Peng
Abstract
Although pair trading is the simplest hedging strategy for an investor to eliminate market risk, it is still a great challenge for reinforcement learning (RL) methods to perform pair trading as human expertise. It requires RL methods to make thousands of correct actions that nevertheless have no obvious relations to the overall trading profit, and to reason over infinite states of the time-varying market most of which have never appeared in history. However, existing RL methods ignore the temporal connections between asset price movements and the risk of the performed trading. These lead to frequent tradings with high transaction costs and potential losses, which barely reach the human expertise level of trading. Therefore, we introduce CREDIT, a risk-aware agent capable of learning to exploit long-term trading opportunities in pair trading similar to a human expert. CREDIT is the first to apply bidirectional GRU along with the temporal attention mechanism to fully consider the temporal correlations embedded in the states, which allows CREDIT to capture long-term patterns of the price movements of two assets to earn higher profit. We also design the risk-aware reward inspired by the economic theory, that models both the profit and risk of the tradings during the trading period. It helps our agent to master pair trading with a robust trading preference that avoids risky trading with possible high returns and losses. Experiments show that it outperforms existing reinforcement learning methods in pair trading and achieves a significant profit over five years of U.S. stock data.

Suggested Citation

  • Weiguang Han & Jimin Huang & Qianqian Xie & Boyi Zhang & Yanzhao Lai & Min Peng, 2023. "Mastering Pair Trading with Risk-Aware Recurrent Reinforcement Learning," Papers 2304.00364, arXiv.org.
    • Handle: RePEc:arx:papers:2304.00364
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2304.00364
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Weiguang Han & Boyi Zhang & Qianqian Xie & Min Peng & Yanzhao Lai & Jimin Huang, 2023. "Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning," Papers 2301.10724, arXiv.org, revised Feb 2023.
    2. Engle, Robert & Granger, Clive, 2015. "Co-integration and error correction: Representation, estimation, and testing," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 39(3), pages 106-135.
    3. Markowitz, Harry, 2014. "Mean–variance approximations to expected utility," European Journal of Operational Research, Elsevier, vol. 234(2), pages 346-355.
    4. Kiyoshi Suzuki, 2018. "Optimal pair-trading strategy over long/short/square positions—empirical study," Quantitative Finance, Taylor & Francis Journals, vol. 18(1), pages 97-119, January.
    5. Christopher Krauss, 2017. "Statistical Arbitrage Pairs Trading Strategies: Review And Outlook," Journal of Economic Surveys, Wiley Blackwell, vol. 31(2), pages 513-545, April.
    6. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weiguang Han & Boyi Zhang & Qianqian Xie & Min Peng & Yanzhao Lai & Jimin Huang, 2023. "Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning," Papers 2301.10724, arXiv.org, revised Feb 2023.
    2. Anna Bykhovskaya & Vadim Gorin, 2020. "Cointegration in large VARs," Papers 2006.14179, arXiv.org, revised Dec 2021.
    3. Fenghui Yu & Wai-Ki Ching & Chufang Wu & Jia-Wen Gu, 2023. "Optimal Pairs Trading Strategies: A Stochastic Mean–Variance Approach," Journal of Optimization Theory and Applications, Springer, vol. 196(1), pages 36-55, January.
    4. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    5. Ziping Zhao & Rui Zhou & Zhongju Wang & Daniel P. Palomar, 2018. "Optimal Portfolio Design for Statistical Arbitrage in Finance," Papers 1803.02974, arXiv.org.
    6. Kasper Johansson & Thomas Schmelzer & Stephen Boyd, 2024. "Finding Moving-Band Statistical Arbitrages via Convex-Concave Optimization," Papers 2402.08108, arXiv.org.
    7. Marianna Brunetti & Roberta de Luca, 2022. "Sensitivity of profitability in cointegration-based pairs trading," Centro Studi di Banca e Finanza (CEFIN) (Center for Studies in Banking and Finance) 0090, Universita di Modena e Reggio Emilia, Dipartimento di Economia "Marco Biagi".
    8. Marianna Brunetti & Roberta De Luca, 2023. "Pairs trading in the index options market," Eurasian Economic Review, Springer;Eurasia Business and Economics Society, vol. 13(1), pages 145-173, March.
    9. Law, K.F. & Li, W.K. & Yu, Philip L.H., 2018. "A single-stage approach for cointegration-based pairs trading," Finance Research Letters, Elsevier, vol. 26(C), pages 177-184.
    10. Marianna Brunetti & Roberta De Luca, 2023. "Pre-selection in cointegration-based pairs trading," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(5), pages 1611-1640, December.
    11. Yu, Philip L.H. & Lu, Renjie, 2017. "Cointegrated market-neutral strategy for basket trading," International Review of Economics & Finance, Elsevier, vol. 49(C), pages 112-124.
    12. Danni Chen & Jing Cui & Yan Gao & Leilei Wu, 2017. "Pairs trading in Chinese commodity futures markets: an adaptive cointegration approach," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 57(5), pages 1237-1264, December.
    13. Miroslav Fil, 2020. "Gold Standard Pairs Trading Rules: Are They Valid?," Papers 2010.01157, arXiv.org.
    14. Tian-Shyr Dai & Yi-Jen Luo & Hao-Han Chang & Chu-Lan Kao & Kuan-Lun Wang & Liang-Chih Liu, 2024. "Asymptotic analyses for trend-stationary pairs trading strategy in high-frequency trading," Review of Quantitative Finance and Accounting, Springer, vol. 63(4), pages 1391-1411, November.
    15. Han, Chulwoo & He, Zhaodong & Toh, Alenson Jun Wei, 2023. "Pairs trading via unsupervised learning," European Journal of Operational Research, Elsevier, vol. 307(2), pages 929-947.
    16. Herwartz, Helmut & Reimers, Hans-Eggert, 2006. "Modelling the Fisher hypothesis: World wide evidence," Economics Working Papers 2006-04, Christian-Albrechts-University of Kiel, Department of Economics.
    17. Bilal Mehmood & Syed Hassan Raza & Mahwish Rana & Huma Sohaib & Muhammad Azhar Khan, 2014. "Triangular Relationship between Energy Consumption, Price Index and National Income in Asian Countries: A Pooled Mean Group Approach in Presence of Structural Breaks," International Journal of Energy Economics and Policy, Econjournals, vol. 4(4), pages 610-620.
    18. Tyrväinen, Timo, 1991. "Unions, wages and employment: evidence from Finland," Bank of Finland Research Discussion Papers 16/1991, Bank of Finland.
    19. Anatoly A. Peresetsky & Ruslan I. Yakubov, 2017. "Autocorrelation in an unobservable global trend: does it help to forecast market returns?," International Journal of Computational Economics and Econometrics, Inderscience Enterprises Ltd, vol. 7(1/2), pages 152-169.
    20. Santos, João & Domingos, Tiago & Sousa, Tânia & St. Aubyn, Miguel, 2016. "Does a small cost share reflect a negligible role for energy in economic production? Testing for aggregate production functions including capital, labor, and useful exergy through a cointegration-base," MPRA Paper 70850, University Library of Munich, Germany.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2304.00364. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.