[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/43862.html
   My bibliography  Save this paper

The best estimation for high-dimensional Markowitz mean-variance optimization

Author

Listed:
  • Bai, Zhidong
  • Li, Hua
  • Wong, Wing-Keung
Abstract
The traditional(plug-in) return for the Markowitz mean-variance (MV) optimization has been demonstrated to seriously overestimate the theoretical optimal return, especially when the dimension to sample size ratio $p/n$ is large. The newly developed bootstrap-corrected estimator corrects the overestimation, but it incurs the "under-prediction problem," it does not do well on the estimation of the corresponding allocation, and it has bigger risk. To circumvent these limitations and to improve the optimal return estimation further, this paper develops the theory of spectral-corrected estimation. We first establish a theorem to explain why the plug-in return greatly overestimates the theoretical optimal return. We prove that under some situations the plug-in return is $\sqrt{\gamma}\ $\ times bigger than the theoretical optimal return, while under other situations, the plug-in return is bigger than but may not be $\sqrt{\gamma}\ $\ times larger than its theoretic counterpart where $\gamma = \frac 1{1-y}$ with $y$ being the limit of the ratio $p/n$. Thereafter, we develop the spectral-corrected estimation for the Markowitz MV model which performs much better than both the plug-in estimation and the bootstrap-corrected estimation not only in terms of the return but also in terms of the allocation and the risk. We further develop properties for our proposed estimation and conduct a simulation to examine the performance of our proposed estimation. Our simulation shows that our proposed estimation not only overcomes the problem of "over-prediction," but also circumvents the "under-prediction," "allocation estimation," and "risk" problems. Our simulation also shows that our proposed spectral-corrected estimation is stable for different values of sample size $n$, dimension $p$, and their ratio $p/n$. In addition, we relax the normality assumption in our proposed estimation so that our proposed spectral-corrected estimators could be obtained when the returns of the assets being studied could follow any distribution under the condition of the existence of the fourth moments.

Suggested Citation

  • Bai, Zhidong & Li, Hua & Wong, Wing-Keung, 2013. "The best estimation for high-dimensional Markowitz mean-variance optimization," MPRA Paper 43862, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:43862
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/43862/1/MPRA_paper_43862.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clark, Ephraim & Jokung, Octave & Kassimatis, Konstantinos, 2011. "Making inefficient market indices efficient," European Journal of Operational Research, Elsevier, vol. 209(1), pages 83-93, February.
    2. Soyer, Refik & Tanyeri, Kadir, 2006. "Bayesian portfolio selection with multi-variate random variance models," European Journal of Operational Research, Elsevier, vol. 171(3), pages 977-990, June.
    3. Blattberg, Robert C & Gonedes, Nicholas J, 1974. "A Comparison of the Stable and Student Distributions as Statistical Models for Stock Prices," The Journal of Business, University of Chicago Press, vol. 47(2), pages 244-280, April.
    4. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
    5. Kan, Raymond & Zhou, Guofu, 2007. "Optimal Portfolio Choice with Parameter Uncertainty," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 42(3), pages 621-656, September.
    6. Stephen A. Ross, 2013. "The Arbitrage Theory of Capital Asset Pricing," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 1, pages 11-30, World Scientific Publishing Co. Pte. Ltd..
    7. Wai Fong & Wing Wong, 2006. "The modified mixture of distributions model: a revisit," Annals of Finance, Springer, vol. 2(2), pages 167-178, March.
    8. Bob Korkie & Harry J. Turtle, 2002. "A Mean-Variance Analysis of Self-Financing Portfolios," Management Science, INFORMS, vol. 48(3), pages 427-443, March.
    9. Eugene F. Fama, 1965. "Portfolio Analysis in a Stable Paretian Market," Management Science, INFORMS, vol. 11(3), pages 404-419, January.
    10. Egozcue, Martin & Wong, Wing-Keung, 2010. "Gains from diversification on convex combinations: A majorization and stochastic dominance approach," European Journal of Operational Research, Elsevier, vol. 200(3), pages 893-900, February.
    11. Frederick Wong, 2003. "Efficient estimation of covariance selection models," Biometrika, Biometrika Trust, vol. 90(4), pages 809-830, December.
    12. Dominic Gasbarro & Wing-Keung Wong & J. Kenton Zumwalt, 2007. "Stochastic Dominance Analysis of iShares," The European Journal of Finance, Taylor & Francis Journals, vol. 13(1), pages 89-101.
    13. Wong, Wing-Keung & Phoon, Kok Fai & Lean, Hooi Hooi, 2008. "Stochastic dominance analysis of Asian hedge funds," Pacific-Basin Finance Journal, Elsevier, vol. 16(3), pages 204-223, June.
    14. Yusif Simaan, 1997. "Estimation Risk in Portfolio Selection: The Mean Variance Model Versus the Mean Absolute Deviation Model," Management Science, INFORMS, vol. 43(10), pages 1437-1446, October.
    15. Wing-Keung Wong & Chenghu Ma, 2008. "Preferences over location-scale family," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 37(1), pages 119-146, October.
    16. Wong, Wing-Keung, 2007. "Stochastic dominance and mean-variance measures of profit and loss for business planning and investment," European Journal of Operational Research, Elsevier, vol. 182(2), pages 829-843, October.
    17. Fu, Chenpeng & Lari-Lavassani, Ali & Li, Xun, 2010. "Dynamic mean-variance portfolio selection with borrowing constraint," European Journal of Operational Research, Elsevier, vol. 200(1), pages 312-319, January.
    18. Andrew W. Lo, A. Craig MacKinlay, 1988. "Stock Market Prices do not Follow Random Walks: Evidence from a Simple Specification Test," The Review of Financial Studies, Society for Financial Studies, vol. 1(1), pages 41-66.
    19. Elton, Edwin J & Gruber, Martin J & Padberg, Manfred W, 1976. "Simple Criteria for Optimal Portfolio Selection," Journal of Finance, American Finance Association, vol. 31(5), pages 1341-1357, December.
    20. Zymler, Steve & Rustem, Berç & Kuhn, Daniel, 2011. "Robust portfolio optimization with derivative insurance guarantees," European Journal of Operational Research, Elsevier, vol. 210(2), pages 410-424, April.
    21. Bruce I. Jacobs & Kenneth N. Levy & Harry M. Markowitz, 2005. "Portfolio Optimization with Factors, Scenarios, and Realistic Short Positions," Operations Research, INFORMS, vol. 53(4), pages 586-599, August.
    22. Simone Manganelli, 2004. "Asset Allocation by Variance Sensitivity Analysis," Journal of Financial Econometrics, Oxford University Press, vol. 2(3), pages 370-389.
    23. Zhidong Bai & Hua Li & Huixia Liu & Wing‐Keung Wong, 2011. "Test statistics for prospect and Markowitz stochastic dominances with applications," Econometrics Journal, Royal Economic Society, vol. 14(2), pages 278-303, July.
    24. Chan, Joshua C.C. & Kroese, Dirk P., 2010. "Efficient estimation of large portfolio loss probabilities in t-copula models," European Journal of Operational Research, Elsevier, vol. 205(2), pages 361-367, September.
    25. Jorion, Philippe, 1985. "International Portfolio Diversification with Estimation Risk," The Journal of Business, University of Chicago Press, vol. 58(3), pages 259-278, July.
    26. Silverstein, J. W. & Choi, S. I., 1995. "Analysis of the Limiting Spectral Distribution of Large Dimensional Random Matrices," Journal of Multivariate Analysis, Elsevier, vol. 54(2), pages 295-309, August.
    27. William F. Sharpe, 1967. "A Linear Programming Algorithm for Mutual Fund Portfolio Selection," Management Science, INFORMS, vol. 13(7), pages 499-510, March.
    28. Kroll, Yoram & Levy, Haim & Markowitz, Harry M, 1984. "Mean-Variance versus Direct Utility Maximization," Journal of Finance, American Finance Association, vol. 39(1), pages 47-61, March.
    29. Markowitz, Harry M, 1991. "Foundations of Portfolio Theory," Journal of Finance, American Finance Association, vol. 46(2), pages 469-477, June.
    30. Vijay K. Chopra & Chris R. Hensel & Andrew L. Turner, 1993. "Massaging Mean-Variance Inputs: Returns from Alternative Global Investment Strategies in the 1980s," Management Science, INFORMS, vol. 39(7), pages 845-855, July.
    31. Elton, Edwin J & Gruber, Martin J & Padberg, Manfred W, 1978. "Simple Criteria for Optimal Portfolio Selection: Tracing out the Efficient Frontier," Journal of Finance, American Finance Association, vol. 33(1), pages 296-302, March.
    32. Markowitz, Harry M & Perold, Andre F, 1981. "Portfolio Analysis with Factors and Scenarios," Journal of Finance, American Finance Association, vol. 36(4), pages 871-877, September.
    33. Chan, Chia-Ying & de Peretti, Christian & Qiao, Zhuo & Wong, Wing-Keung, 2012. "Empirical test of the efficiency of the UK covered warrants market: Stochastic dominance and likelihood ratio test approach," Journal of Empirical Finance, Elsevier, vol. 19(1), pages 162-174.
    34. Zhao, Yonggan & Ziemba, William T., 2008. "Calculating risk neutral probabilities and optimal portfolio policies in a dynamic investment model with downside risk control," European Journal of Operational Research, Elsevier, vol. 185(3), pages 1525-1540, March.
    35. Egozcue, Martín & García, Luis Fuentes & Wong, Wing-Keung & Zitikis, Ricardas, 2011. "Do investors like to diversify? A study of Markowitz preferences," European Journal of Operational Research, Elsevier, vol. 215(1), pages 188-193, November.
    36. Luciano, Elisa & Peccati, Lorenzo & Cifarelli, Donato M., 2003. "VaR as a risk measure for multiperiod static inventory models," International Journal of Production Economics, Elsevier, vol. 81(1), pages 375-384, January.
    37. Harry Markowitz, 1952. "Portfolio Selection," Journal of Finance, American Finance Association, vol. 7(1), pages 77-91, March.
    38. Andre F. Perold, 1984. "Large-Scale Portfolio Optimization," Management Science, INFORMS, vol. 30(10), pages 1143-1160, October.
    39. Clark, Peter K, 1973. "A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices," Econometrica, Econometric Society, vol. 41(1), pages 135-155, January.
    40. Ma, Chenghu & Wong, Wing-Keung, 2010. "Stochastic dominance and risk measure: A decision-theoretic foundation for VaR and C-VaR," European Journal of Operational Research, Elsevier, vol. 207(2), pages 927-935, December.
    41. Dominic Gasbarro & Wing-Keung Wong & J. Kenton Zumwalt, 2007. "Stochastic Dominance Analysis of iShares," The European Journal of Finance, Taylor & Francis Journals, vol. 13(1), pages 89-101.
    42. Eugene F. Fama, 1963. "Mandelbrot and the Stable Paretian Hypothesis," The Journal of Business, University of Chicago Press, vol. 36, pages 420-420.
    43. Jansen, Dennis W. & Li, Qi & Wang, Zijun & Yang, Jian, 2008. "Fiscal policy and asset markets: A semiparametric analysis," Journal of Econometrics, Elsevier, vol. 147(1), pages 141-150, November.
    44. Zhidong Bai & Yongchang Hui & Wing-Keung Wong & Ričardas Zitikis, 2012. "Prospect Performance Evaluation: Making a Case for a Non-asymptotic UMPU Test," Journal of Financial Econometrics, Oxford University Press, vol. 10(4), pages 703-732, September.
    45. Sharpe, William F., 1971. "A Linear Programming Approximation for the General Portfolio Analysis Problem," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 6(5), pages 1263-1275, December.
    46. Yu, Mei & Takahashi, Satoru & Inoue, Hiroshi & Wang, Shouyang, 2010. "Dynamic portfolio optimization with risk control for absolute deviation model," European Journal of Operational Research, Elsevier, vol. 201(2), pages 349-364, March.
    47. Bai, Zhidong & Wang, Keyan & Wong, Wing-Keung, 2011. "The mean-variance ratio test--A complement to the coefficient of variation test and the Sharpe ratio test," Statistics & Probability Letters, Elsevier, vol. 81(8), pages 1078-1085, August.
    48. Hiroshi Konno & Hiroaki Yamazaki, 1991. "Mean-Absolute Deviation Portfolio Optimization Model and Its Applications to Tokyo Stock Market," Management Science, INFORMS, vol. 37(5), pages 519-531, May.
    49. William F. Sharpe, 1964. "Capital Asset Prices: A Theory Of Market Equilibrium Under Conditions Of Risk," Journal of Finance, American Finance Association, vol. 19(3), pages 425-442, September.
    50. Lam, Kin & Liu, Taisheng & Wong, Wing-Keung, 2010. "A pseudo-Bayesian model in financial decision making with implications to market volatility, under- and overreaction," European Journal of Operational Research, Elsevier, vol. 203(1), pages 166-175, May.
    51. Leung, Pui-Lam & Ng, Hon-Yip & Wong, Wing-Keung, 2012. "An improved estimation to make Markowitz’s portfolio optimization theory users friendly and estimation accurate with application on the US stock market investment," European Journal of Operational Research, Elsevier, vol. 222(1), pages 85-95.
    52. Wong, Wing-Keung, 2007. "Stochastic dominance and mean-variance measures of profit and loss for business planning and investment," European Journal of Operational Research, Elsevier, vol. 182(2), pages 829-843, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bai, Zhidong & Liu, Huixia & Wong, Wing-Keung, 2016. "Making Markowitz's Portfolio Optimization Theory Practically Useful," MPRA Paper 74360, University Library of Munich, Germany.
    2. Chang, C-L. & McAleer, M.J. & Wong, W.-K., 2015. "Informatics, Data Mining, Econometrics and Financial Economics: A Connection," Econometric Institute Research Papers EI2015-34, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leung, Pui-Lam & Ng, Hon-Yip & Wong, Wing-Keung, 2012. "An improved estimation to make Markowitz’s portfolio optimization theory users friendly and estimation accurate with application on the US stock market investment," European Journal of Operational Research, Elsevier, vol. 222(1), pages 85-95.
    2. Zhihui Lv & Amanda M. Y. Chu & Wing Keung Wong & Thomas C. Chiang, 2021. "The maximum-return-and-minimum-volatility effect: evidence from choosing risky and riskless assets to form a portfolio," Risk Management, Palgrave Macmillan, vol. 23(1), pages 97-122, June.
    3. Bai, Zhidong & Liu, Huixia & Wong, Wing-Keung, 2016. "Making Markowitz's Portfolio Optimization Theory Practically Useful," MPRA Paper 74360, University Library of Munich, Germany.
    4. GUORUI BIAN & MICHAEL McALEER & WING-KEUNG WONG, 2013. "Robust Estimation And Forecasting Of The Capital Asset Pricing Model," Annals of Financial Economics (AFE), World Scientific Publishing Co. Pte. Ltd., vol. 8(02), pages 1-18.
    5. Chia-Lin Chang & Michael McAleer & Wing-Keung Wong, 2018. "Big Data, Computational Science, Economics, Finance, Marketing, Management, and Psychology: Connections," JRFM, MDPI, vol. 11(1), pages 1-29, March.
    6. Chia-Lin Chang & Michael McAleer & Wing-Keung Wong, 2018. "Decision Sciences, Economics, Finance, Business, Computing, and Big Data: Connections," Documentos de Trabajo del ICAE 2018-09, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico.
    7. Chia-Lin Chang & Michael McAleer & Wing-Keung Wong, 2018. "Decision Sciences, Economics, Finance, Business, Computing, And Big Data: Connections," Advances in Decision Sciences, Asia University, Taiwan, vol. 22(1), pages 36-94, December.
    8. Chia-Lin Chang & Michael McAleer & Wing-Keung Wong, 2018. "Big Data, Computational Science, Economics, Finance, Marketing, Management, and Psychology: Connections," Journal of Risk and Financial Management, MDPI, Open Access Journal, vol. 11(1), pages 1-29, March.
    9. Kim-Hung Pho & Tuan-Kiet Tran & Thi Diem-Chinh Ho & Wing-Keung Wong, 2019. "Optimal Solution Techniques in Decision Sciences A Review," Advances in Decision Sciences, Asia University, Taiwan, vol. 23(1), pages 114-161, March.
    10. Chia-Lin Chang & Michael McAleer & Wing-Keung Wong, 2016. "Management science, economics and finance: A connection," Documentos de Trabajo del ICAE 2016-07, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico.
    11. Kim-Hung Pho & Thi Diem-Chinh Ho & Tuan-Kiet Tran & Wing-Keung Wong, 2019. "Moment Generating Function, Expectation And Variance Of Ubiquitous Distributions With Applications In Decision Sciences: A Review," Advances in Decision Sciences, Asia University, Taiwan, vol. 23(2), pages 65-150, June.
    12. Bai, Zhidong & Phoon, Kok Fai & Wang, Keyan & Wong, Wing-Keung, 2013. "The performance of commodity trading advisors: A mean-variance-ratio test approach," The North American Journal of Economics and Finance, Elsevier, vol. 25(C), pages 188-201.
    13. Zhidong Bai & Hua Li & Michael McAleer & Wing-Keung Wong, 2015. "Stochastic dominance statistics for risk averters and risk seekers: an analysis of stock preferences for USA and China," Quantitative Finance, Taylor & Francis Journals, vol. 15(5), pages 889-900, May.
    14. Ephraim Clark & Zhuo Qiao & Wing-Keung Wong, 2016. "Theories Of Risk: Testing Investor Behavior On The Taiwan Stock And Stock Index Futures Markets," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 907-924, April.
    15. Nguyen Huu Hau & Tran Trung Tinh & Hoa Anh Tuong & Wing-Keung Wong, 2020. "Review of Matrix Theory with Applications in Education and Decision Sciences," Advances in Decision Sciences, Asia University, Taiwan, vol. 24(1), pages 28-69, March.
    16. Broll, Udo & Wong, Wing-Keung & Wu, Mojia, 2013. "Banking Firm and Two-Moment Decision Making," MPRA Paper 51687, University Library of Munich, Germany.
    17. Guo, Xu & Lam, Kin & Wong, Wing-Keung & Zhu, Lixing, 2012. "A New Pseudo-Bayesian Model of Investors' Behavior in Financial Crises," MPRA Paper 42535, University Library of Munich, Germany.
    18. Hooi Hooi Lean & Michael McAleer & Wing-Keung Wong, 2013. "Risk-averse and Risk-seeking Investor Preferences for Oil Spot and Futures," Documentos de Trabajo del ICAE 2013-31, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico, revised Aug 2013.
    19. Li, Hua & Bai, Zhidong & Wong, Wing-Keung & McAleer, Michael, 2022. "Spectrally-Corrected Estimation for High-Dimensional Markowitz Mean-Variance Optimization," Econometrics and Statistics, Elsevier, vol. 24(C), pages 133-150.
    20. Lean, Hooi Hooi & McAleer, Michael & Wong, Wing-Keung, 2015. "Preferences of risk-averse and risk-seeking investors for oil spot and futures before, during and after the Global Financial Crisis," International Review of Economics & Finance, Elsevier, vol. 40(C), pages 204-216.

    More about this item

    Keywords

    Markowitz mean-variance optimization; Optimal Return; Optimal Portfolio Allocation; Large Random Matrix; Bootstrap Method;
    All these keywords.

    JEL classification:

    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • C3 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:43862. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.