[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2303.12751.html
   My bibliography  Save this paper

A Unified Framework for Fast Large-Scale Portfolio Optimization

Author

Listed:
  • Weichuan Deng
  • Pawel Polak
  • Abolfazl Safikhani
  • Ronakdilip Shah
Abstract
We introduce a unified framework for rapid, large-scale portfolio optimization that incorporates both shrinkage and regularization techniques. This framework addresses multiple objectives, including minimum variance, mean-variance, and the maximum Sharpe ratio, and also adapts to various portfolio weight constraints. For each optimization scenario, we detail the translation into the corresponding quadratic programming (QP) problem and then integrate these solutions into a new open-source Python library. Using 50 years of return data from US mid to large-sized companies, and 33 distinct firm-specific characteristics, we utilize our framework to assess the out-of-sample monthly rebalanced portfolio performance of widely-adopted covariance matrix estimators and factor models, examining both daily and monthly returns. These estimators include the sample covariance matrix, linear and nonlinear shrinkage estimators, and factor portfolios based on Asset Pricing (AP) Trees, Principal Component Analysis (PCA), Risk Premium PCA (RP-PCA), and Instrumented PCA (IPCA). Our findings emphasize that AP-Trees and PCA-based factor models consistently outperform all other approaches in out-of-sample portfolio performance. Finally, we develop new l1 and l2 regularizations of factor portfolio norms which not only elevate the portfolio performance of AP-Trees and PCA-based factor models but they have a potential to reduce an excessive turnover and transaction costs often associated with these models.

Suggested Citation

  • Weichuan Deng & Pawel Polak & Abolfazl Safikhani & Ronakdilip Shah, 2023. "A Unified Framework for Fast Large-Scale Portfolio Optimization," Papers 2303.12751, arXiv.org, revised Nov 2023.
    • Handle: RePEc:arx:papers:2303.12751
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2303.12751
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kelly, Bryan T. & Pruitt, Seth & Su, Yinan, 2019. "Characteristics are covariances: A unified model of risk and return," Journal of Financial Economics, Elsevier, vol. 134(3), pages 501-524.
    2. Holthausen, Robert W. & Larcker, David F., 1992. "The prediction of stock returns using financial statement information," Journal of Accounting and Economics, Elsevier, vol. 15(2-3), pages 373-411, August.
    3. Amit Goyal & Alessio Saretto, 2022. "Are Equity Option Returns Abnormal? IPCA Says No," Working Papers 2214, Federal Reserve Bank of Dallas.
    4. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    5. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    6. Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020. "Taming the Factor Zoo: A Test of New Factors," Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
    7. Bali, Turan G. & Cakici, Nusret & Whitelaw, Robert F., 2011. "Maxing out: Stocks as lotteries and the cross-section of expected returns," Journal of Financial Economics, Elsevier, vol. 99(2), pages 427-446, February.
    8. Heitor Almeida & Murillo Campello, 2007. "Financial Constraints, Asset Tangibility, and Corporate Investment," The Review of Financial Studies, Society for Financial Studies, vol. 20(5), pages 1429-1460, 2007 12.
    9. Richardson, Scott A. & Sloan, Richard G. & Soliman, Mark T. & Tuna, Irem, 2005. "Accrual reliability, earnings persistence and stock prices," Journal of Accounting and Economics, Elsevier, vol. 39(3), pages 437-485, September.
    10. Roncalli, Thierry, 2013. "Introduction to Risk Parity and Budgeting," MPRA Paper 47679, University Library of Munich, Germany.
    11. Victor DeMiguel & Lorenzo Garlappi & Francisco J. Nogales & Raman Uppal, 2009. "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms," Management Science, INFORMS, vol. 55(5), pages 798-812, May.
    12. Barth, ME & Elliott, JA & Finn, MW, 1999. "Market rewards associated with patterns of increasing earnings," Journal of Accounting Research, Wiley Blackwell, vol. 37(2), pages 387-413.
    13. Frederico Belo & Xiaoji Lin & Santiago Bazdresch, 2014. "Labor Hiring, Investment, and Stock Return Predictability in the Cross Section," Journal of Political Economy, University of Chicago Press, vol. 122(1), pages 129-177.
    14. Michael J. Cooper & Huseyin Gulen & Michael J. Schill, 2008. "Asset Growth and the Cross‐Section of Stock Returns," Journal of Finance, American Finance Association, vol. 63(4), pages 1609-1651, August.
    15. Bai, Jushan & Liao, Yuan, 2016. "Efficient estimation of approximate factor models via penalized maximum likelihood," Journal of Econometrics, Elsevier, vol. 191(1), pages 1-18.
    16. Tobias J. Moskowitz & Mark Grinblatt, 1999. "Do Industries Explain Momentum?," Journal of Finance, American Finance Association, vol. 54(4), pages 1249-1290, August.
    17. Re‐Jin Guo & Baruch Lev & Charles Shi, 2006. "Explaining the Short‐ and Long‐Term IPO Anomalies in the US by R&D," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 33(3‐4), pages 550-579, April.
    18. Palazzo, Berardino, 2012. "Cash holdings, risk, and expected returns," Journal of Financial Economics, Elsevier, vol. 104(1), pages 162-185.
    19. Banz, Rolf W., 1981. "The relationship between return and market value of common stocks," Journal of Financial Economics, Elsevier, vol. 9(1), pages 3-18, March.
    20. repec:bla:jfinan:v:44:y:1989:i:2:p:479-86 is not listed on IDEAS
    21. John Lintner, 1965. "Security Prices, Risk, And Maximal Gains From Diversification," Journal of Finance, American Finance Association, vol. 20(4), pages 587-615, December.
    22. Kewei Hou & Chen Xue & Lu Zhang, 2015. "Editor's Choice Digesting Anomalies: An Investment Approach," The Review of Financial Studies, Society for Financial Studies, vol. 28(3), pages 650-705.
    23. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    24. Amihud, Yakov, 2002. "Illiquidity and stock returns: cross-section and time-series effects," Journal of Financial Markets, Elsevier, vol. 5(1), pages 31-56, January.
    25. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    26. Andrew Ang & Robert J. Hodrick & Yuhang Xing & Xiaoyan Zhang, 2006. "The Cross‐Section of Volatility and Expected Returns," Journal of Finance, American Finance Association, vol. 61(1), pages 259-299, February.
    27. Itay Kama, 2009. "On the Market Reaction to Revenue and Earnings Surprises," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 36(1‐2), pages 31-50, January.
    28. Jeffrey Pontiff & Artemiza Woodgate, 2008. "Share Issuance and Cross‐sectional Returns," Journal of Finance, American Finance Association, vol. 63(2), pages 921-945, April.
    29. Basu, S, 1977. "Investment Performance of Common Stocks in Relation to Their Price-Earnings Ratios: A Test of the Efficient Market Hypothesis," Journal of Finance, American Finance Association, vol. 32(3), pages 663-682, June.
    30. Andrea L. Eisfeldt & Dimitris Papanikolaou, 2013. "Organization Capital and the Cross-Section of Expected Returns," Journal of Finance, American Finance Association, vol. 68(4), pages 1365-1406, August.
    31. Marc S. PAOLELLA & Pawel POLAK, 2015. "Portfolio Selection with Active Risk Monitoring," Swiss Finance Institute Research Paper Series 15-17, Swiss Finance Institute.
    32. Rui Ding & Stan Uryasev, 2022. "Drawdown beta and portfolio optimization," Quantitative Finance, Taylor & Francis Journals, vol. 22(7), pages 1265-1276, July.
    33. Michaely, Roni & Thaler, Richard H & Womack, Kent L, 1995. "Price Reactions to Dividend Initiations and Omissions: Overreaction or Drift?," Journal of Finance, American Finance Association, vol. 50(2), pages 573-608, June.
    34. Fama, Eugene F & MacBeth, James D, 1973. "Risk, Return, and Equilibrium: Empirical Tests," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 607-636, May-June.
    35. Hong, Harrison & Kacperczyk, Marcin, 2009. "The price of sin: The effects of social norms on markets," Journal of Financial Economics, Elsevier, vol. 93(1), pages 15-36, July.
    36. Paolella, Marc S. & Polak, Paweł & Walker, Patrick S., 2021. "A non-elliptical orthogonal GARCH model for portfolio selection under transaction costs," Journal of Banking & Finance, Elsevier, vol. 125(C).
    37. Lakonishok, Josef & Shleifer, Andrei & Vishny, Robert W, 1994. "Contrarian Investment, Extrapolation, and Risk," Journal of Finance, American Finance Association, vol. 49(5), pages 1541-1578, December.
    38. Jegadeesh, Narasimhan & Titman, Sheridan, 1993. "Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency," Journal of Finance, American Finance Association, vol. 48(1), pages 65-91, March.
    39. Asger Lunde & Neil Shephard & Kevin Sheppard, 2016. "Econometric Analysis of Vast Covariance Matrices Using Composite Realized Kernels and Their Application to Portfolio Choice," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 504-518, October.
    40. Novy-Marx, Robert, 2013. "The other side of value: The gross profitability premium," Journal of Financial Economics, Elsevier, vol. 108(1), pages 1-28.
    41. Chordia, Tarun & Subrahmanyam, Avanidhar & Anshuman, V. Ravi, 2001. "Trading activity and expected stock returns," Journal of Financial Economics, Elsevier, vol. 59(1), pages 3-32, January.
    42. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    43. Kewei Hou & Tobias J. Moskowitz, 2005. "Market Frictions, Price Delay, and the Cross-Section of Expected Returns," The Review of Financial Studies, Society for Financial Studies, vol. 18(3), pages 981-1020.
    44. Soros Chitsiripanich & Marc S. Paolella & Pawel Polak & Patrick S. Walker, 2022. "Momentum Without Crashes," Swiss Finance Institute Research Paper Series 22-87, Swiss Finance Institute.
    45. Balakrishnan, Karthik & Bartov, Eli & Faurel, Lucile, 2010. "Post loss/profit announcement drift," Journal of Accounting and Economics, Elsevier, vol. 50(1), pages 20-41, May.
    46. Jacob Thomas & Frank X. Zhang, 2011. "Tax Expense Momentum," Journal of Accounting Research, Wiley Blackwell, vol. 49(3), pages 791-821, June.
    47. Tsai, Henghsiu & Tsay, Ruey S., 2010. "Constrained Factor Models," Journal of the American Statistical Association, American Statistical Association, vol. 105(492), pages 1593-1605.
    48. Carhart, Mark M, 1997. "On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    49. repec:bla:jfinan:v:59:y:2004:i:2:p:623-650 is not listed on IDEAS
    50. Litzenberger, Robert H & Ramaswamy, Krishna, 1982. "The Effects of Dividends on Common Stock Prices: Tax Effects or Information Effects?," Journal of Finance, American Finance Association, vol. 37(2), pages 429-443, May.
    51. Kewei Hou & David T. Robinson, 2006. "Industry Concentration and Average Stock Returns," Journal of Finance, American Finance Association, vol. 61(4), pages 1927-1956, August.
    52. Valta, Philip, 2016. "Strategic Default, Debt Structure, and Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 51(1), pages 197-229, February.
    53. Huang, Alan Guoming, 2009. "The cross section of cashflow volatility and expected stock returns," Journal of Empirical Finance, Elsevier, vol. 16(3), pages 409-429, June.
    54. Simon Hediger & Jeffrey Näf & Marc S. Paolella & Pawel Polak, 2021. "Heterogeneous Tail Generalized Common Factor Modeling," Swiss Finance Institute Research Paper Series 21-73, Swiss Finance Institute.
    55. Bai, Jushan & Ng, Serena, 2013. "Principal components estimation and identification of static factors," Journal of Econometrics, Elsevier, vol. 176(1), pages 18-29.
    56. Piotroski, JD, 2000. "Value investing: The use of historical financial statement information to separate winners from losers," Journal of Accounting Research, Wiley Blackwell, vol. 38, pages 1-41.
    57. Christopher W. Anderson & Luis Garcia‐Feijóo, 2006. "Empirical Evidence on Capital Investment, Growth Options, and Security Returns," Journal of Finance, American Finance Association, vol. 61(1), pages 171-194, February.
    58. Ou, Jane A. & Penman, Stephen H., 1989. "Financial statement analysis and the prediction of stock returns," Journal of Accounting and Economics, Elsevier, vol. 11(4), pages 295-329, November.
    59. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    60. William F. Sharpe, 1964. "Capital Asset Prices: A Theory Of Market Equilibrium Under Conditions Of Risk," Journal of Finance, American Finance Association, vol. 19(3), pages 425-442, September.
    61. Titman, Sheridan & Wei, K. C. John & Xie, Feixue, 2004. "Capital Investments and Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 39(4), pages 677-700, December.
    62. Selale Tuzel, 2010. "Corporate Real Estate Holdings and the Cross-Section of Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 23(6), pages 2268-2302, June.
    63. Lasse Heje Pedersen, 2015. "Efficiently Inefficient: How Smart Money Invests and Market Prices Are Determined," Economics Books, Princeton University Press, edition 1, number 10441.
    64. Itay Kama, 2009. "On the Market Reaction to Revenue and Earnings Surprises," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 36(1-2), pages 31-50.
    65. Liu, Weimin, 2006. "A liquidity-augmented capital asset pricing model," Journal of Financial Economics, Elsevier, vol. 82(3), pages 631-671, December.
    66. Paolella, Marc S. & Polak, Paweł & Walker, Patrick S., 2019. "Regime switching dynamic correlations for asymmetric and fat-tailed conditional returns," Journal of Econometrics, Elsevier, vol. 213(2), pages 493-515.
    67. Datar, Vinay T. & Y. Naik, Narayan & Radcliffe, Robert, 1998. "Liquidity and stock returns: An alternative test," Journal of Financial Markets, Elsevier, vol. 1(2), pages 203-219, August.
    68. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    69. Re-Jin Guo & Baruch Lev & Charles Shi, 2006. "Explaining the Short- and Long-Term IPO Anomalies in the US by R&D," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 33(3-4), pages 550-579.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. De Nard, Gianluca & Zhao, Zhao, 2022. "A large-dimensional test for cross-sectional anomalies:Efficient sorting revisited," International Review of Economics & Finance, Elsevier, vol. 80(C), pages 654-676.
    2. Jiaju Miao & Pawel Polak, 2023. "Online Ensemble of Models for Optimal Predictive Performance with Applications to Sector Rotation Strategy," Papers 2304.09947, arXiv.org.
    3. Tran, Vu Le, 2023. "Sentiment and covariance characteristics," International Review of Financial Analysis, Elsevier, vol. 86(C).
    4. Hediger, Simon & Michel, Loris & Näf, Jeffrey, 2022. "On the use of random forest for two-sample testing," Computational Statistics & Data Analysis, Elsevier, vol. 170(C).
    5. Bui, Dien Giau & Kong, De-Rong & Lin, Chih-Yung & Lin, Tse-Chun, 2023. "Momentum in machine learning: Evidence from the Taiwan stock market," Pacific-Basin Finance Journal, Elsevier, vol. 82(C).
    6. Hoang, Khoa & Cannavan, Damien & Gaunt, Clive & Huang, Ronghong, 2019. "Is that factor just lucky? Australian evidence," Pacific-Basin Finance Journal, Elsevier, vol. 57(C).
    7. Gianluca De Nard & Simon Hediger & Markus Leippold, 2022. "Subsampled factor models for asset pricing: The rise of Vasa," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(6), pages 1217-1247, September.
    8. Andrew Y. Chen & Tom Zimmermann, 2022. "Open Source Cross-Sectional Asset Pricing," Critical Finance Review, now publishers, vol. 11(2), pages 207-264, May.
    9. Hoang, Khoa & Huang, Ronghong & Truong, Helen, 2023. "Resurrecting the market factor: A case of data mining across international markets," Pacific-Basin Finance Journal, Elsevier, vol. 82(C).
    10. Kewei Hou & Chen Xue & Lu Zhang, 2017. "Replicating Anomalies," NBER Working Papers 23394, National Bureau of Economic Research, Inc.
    11. Kristoffer Pons Bertelsen, 2022. "The Prior Adaptive Group Lasso and the Factor Zoo," CREATES Research Papers 2022-05, Department of Economics and Business Economics, Aarhus University.
    12. Geertsema, Paul & Lu, Helen, 2020. "The correlation structure of anomaly strategies," Journal of Banking & Finance, Elsevier, vol. 119(C).
    13. Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020. "Taming the Factor Zoo: A Test of New Factors," Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
    14. Cederburg, Scott & O’Doherty, Michael S. & Wang, Feifei & Yan, Xuemin (Sterling), 2020. "On the performance of volatility-managed portfolios," Journal of Financial Economics, Elsevier, vol. 138(1), pages 95-117.
    15. Tobek, Ondrej & Hronec, Martin, 2021. "Does it pay to follow anomalies research? Machine learning approach with international evidence," Journal of Financial Markets, Elsevier, vol. 56(C).
    16. Yu-Chin Hsu & Hsiou-Wei Lin & Kendro Vincent, 2017. "Do Cross-Sectional Stock Return Predictors Pass the Test without Data-Snooping Bias?," IEAS Working Paper : academic research 17-A003, Institute of Economics, Academia Sinica, Taipei, Taiwan.
    17. Wang, Feifei & Yan, Xuemin Sterling, 2021. "Downside risk and the performance of volatility-managed portfolios," Journal of Banking & Finance, Elsevier, vol. 131(C).
    18. Jozef Barunik & Martin Hronec & Ondrej Tobek, 2024. "Predicting the distributions of stock returns around the globe in the era of big data and learning," Papers 2408.07497, arXiv.org.
    19. Kewei Hou & Haitao Mo & Chen Xue & Lu Zhang, 2019. "Which Factors?," Review of Finance, European Finance Association, vol. 23(1), pages 1-35.
    20. Nettayanun, Sampan, 2023. "Asset pricing in bull and bear markets," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 83(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2303.12751. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.