[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1905.02061.html
   My bibliography  Save this paper

Estimation of high-dimensional factor models and its application in power data analysis

Author

Listed:
  • Xin Shi
  • Robert Qiu
Abstract
In dealing with high-dimensional data, factor models are often used for reducing dimensions and extracting relevant information. The spectrum of covariance matrices from power data exhibits two aspects: 1) bulk, which arises from random noise or fluctuations and 2) spikes, which represents factors caused by anomaly events. In this paper, we propose a new approach to the estimation of high-dimensional factor models, minimizing the distance between the empirical spectral density (ESD) of covariance matrices of the residuals of power data that are obtained by subtracting principal components and the limiting spectral density (LSD) from a multiplicative covariance structure model. The free probability theory (FPT) is used to derive the spectral density of the multiplicative covariance model, which efficiently solves the computational difficulties. The proposed approach connects the estimation of the number of factors to the LSD of covariance matrices of the residuals, which provides estimators of the number of factors and the correlation structure information in the residuals. Considering a lot of measurement noise is contained in the power data and the correlation structure is complex for the residuals, the approach prefers approaching the ESD of covariance matrices of the residuals through a multiplicative covariance model, which avoids making crude assumptions or simplifications on the complex structure of the data. Theoretical studies show the proposed approach is robust against noise and sensitive to the presence of weak factors. The synthetic data from IEEE 118-bus power system is used to validate the effectiveness of the approach. Furthermore, the application to the analysis of the real-world online monitoring data in a power grid shows that the estimators in the approach can be used to indicate the system behavior.

Suggested Citation

  • Xin Shi & Robert Qiu, 2019. "Estimation of high-dimensional factor models and its application in power data analysis," Papers 1905.02061, arXiv.org, revised Oct 2019.
    • Handle: RePEc:arx:papers:1905.02061
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1905.02061
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Cragg, John G. & Donald, Stephen G., 1997. "Inferring the rank of a matrix," Journal of Econometrics, Elsevier, vol. 76(1-2), pages 223-250.
    2. George Kapetanios, 2004. "A New Method for Determining the Number of Factors in Factor Models with Large Datasets," Working Papers 525, Queen Mary University of London, School of Economics and Finance.
    3. George Kapetanios, 2004. "A New Method for Determining the Number of Factors in Factor Models with Large Datasets," Working Papers 525, Queen Mary University of London, School of Economics and Finance.
    4. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    5. Kapetanios, George, 2010. "A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(3), pages 397-409.
    6. Connor, Gregory & Korajczyk, Robert A, 1993. "A Test for the Number of Factors in an Approximate Factor Model," Journal of Finance, American Finance Association, vol. 48(4), pages 1263-1291, September.
    7. Alexei Onatski, 2010. "Determining the Number of Factors from Empirical Distribution of Eigenvalues," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1004-1016, November.
    8. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    9. Lewbel, Arthur, 1991. "The Rank of Demand Systems: Theory and Nonparametric Estimation," Econometrica, Econometric Society, vol. 59(3), pages 711-730, May.
    10. Mario Forni & Lucrezia Reichlin, 1998. "Let's Get Real: A Factor Analytical Approach to Disaggregated Business Cycle Dynamics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 453-473.
    11. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. GUO-FITOUSSI, Liang, 2013. "A Comparison of the Finite Sample Properties of Selection Rules of Factor Numbers in Large Datasets," MPRA Paper 50005, University Library of Munich, Germany.
    2. Freyaldenhoven, Simon, 2022. "Factor models with local factors — Determining the number of relevant factors," Journal of Econometrics, Elsevier, vol. 229(1), pages 80-102.
    3. Kemal Bagzibagli, 2014. "Monetary transmission mechanism and time variation in the Euro area," Empirical Economics, Springer, vol. 47(3), pages 781-823, November.
    4. Li, Hongjun & Li, Qi & Shi, Yutang, 2017. "Determining the number of factors when the number of factors can increase with sample size," Journal of Econometrics, Elsevier, vol. 197(1), pages 76-86.
    5. Alain-Philippe Fortin & Patrick Gagliardini & O. Scaillet, 2022. "Eigenvalue tests for the number of latent factors in short panels," Swiss Finance Institute Research Paper Series 22-81, Swiss Finance Institute.
    6. Simon Freyaldenhoven, 2017. "A Generalized Factor Model with Local Factors," 2017 Papers pfr361, Job Market Papers.
    7. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," DSSR Discussion Papers 96, Graduate School of Economics and Management, Tohoku University.
    8. Alexander Chudik & M. Hashem Pesaran, 2013. "Large panel data models with cross-sectional dependence: a survey," Globalization Institute Working Papers 153, Federal Reserve Bank of Dallas.
    9. Alain-Philippe Fortin & Patrick Gagliardini & Olivier Scaillet, 2023. "Latent Factor Analysis in Short Panels," Papers 2306.14004, arXiv.org, revised May 2024.
    10. Aït-Sahalia, Yacine & Xiu, Dacheng, 2017. "Using principal component analysis to estimate a high dimensional factor model with high-frequency data," Journal of Econometrics, Elsevier, vol. 201(2), pages 384-399.
    11. Fan, Jianqing & Liao, Yuan & Shi, Xiaofeng, 2015. "Risks of large portfolios," Journal of Econometrics, Elsevier, vol. 186(2), pages 367-387.
    12. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019. "A diagnostic criterion for approximate factor structure," Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
    13. Yunus Emre Ergemen & Carlos Vladimir Rodríguez-Caballero, 2016. "A Dynamic Multi-Level Factor Model with Long-Range Dependence," CREATES Research Papers 2016-23, Department of Economics and Business Economics, Aarhus University.
    14. Kapetanios, George, 2010. "A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(3), pages 397-409.
    15. Mao Takongmo, Charles Olivier & Stevanovic, Dalibor, 2015. "Selection Of The Number Of Factors In Presence Of Structural Instability: A Monte Carlo Study," L'Actualité Economique, Société Canadienne de Science Economique, vol. 91(1-2), pages 177-233, Mars-Juin.
    16. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    17. Ergemen, Yunus Emre & Rodríguez-Caballero, C. Vladimir, 2023. "Estimation of a dynamic multi-level factor model with possible long-range dependence," International Journal of Forecasting, Elsevier, vol. 39(1), pages 405-430.
    18. Bai, Jushan & Liao, Yuan, 2016. "Efficient estimation of approximate factor models via penalized maximum likelihood," Journal of Econometrics, Elsevier, vol. 191(1), pages 1-18.
    19. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    20. Kapetanios, George, 2010. "A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(3), pages 397-409.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1905.02061. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.