[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v285y2020i3p1042-1057.html
   My bibliography  Save this article

Benchmarking the benchmarks – Comparing the accuracy of Data Envelopment Analysis models in constant returns to scale settings

Author

Listed:
  • Kohl, Sebastian
  • Brunner, Jens O.
Abstract
Despite the massive use of Data Envelopment Analysis (DEA) models for efficiency estimations in scientific applications, no paper cared about identifying the DEA model, which is able to provide the most accurate efficiency estimates, so far. We develop an established method based on a Monte Carlo data generation process to create artificial data. As we use a Translog production function instead of the commonly utilized Cobb Douglas production function, we are able to construct meaningful scenarios for constant returns to scale. The decision-making units resulting from the generated data are then used to calculate DEA estimators using different DEA models. Finally, the quality of the resulting efficiency estimates is evaluated by five performance indicators and summarized in benchmark scores. With this procedure, we can postulate general statements on parameters that influence the quality of DEA studies in a positive/negative way and determine which DEA model operates in the most accurate way for a range of scenarios. Here, we can show that the Assurance Region and Slacks-Based-Measurement models outperform the CCR (Charnes–Cooper–Rhodes) model in constant returns to scale scenarios. We therefore recommend a reduced utilization of the CCR model in DEA applications.

Suggested Citation

  • Kohl, Sebastian & Brunner, Jens O., 2020. "Benchmarking the benchmarks – Comparing the accuracy of Data Envelopment Analysis models in constant returns to scale settings," European Journal of Operational Research, Elsevier, vol. 285(3), pages 1042-1057.
    • Handle: RePEc:eee:ejores:v:285:y:2020:i:3:p:1042-1057
    DOI: 10.1016/j.ejor.2020.02.031
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221720301661
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2020.02.031?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. William W. Cooper & Lawrence M. Seiford & Joe Zhu (ed.), 2011. "Handbook on Data Envelopment Analysis," International Series in Operations Research and Management Science, Springer, number 978-1-4419-6151-8, December.
    2. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    3. Léopold Simar & Paul Wilson, 2000. "Statistical Inference in Nonparametric Frontier Models: The State of the Art," Journal of Productivity Analysis, Springer, vol. 13(1), pages 49-78, January.
    4. Golany, B & Roll, Y, 1989. "An application procedure for DEA," Omega, Elsevier, vol. 17(3), pages 237-250.
    5. Léopold Simar & Paul W. Wilson, 2011. "Performance of the Bootstrap for DEA Estimators and Iterating the Principle," International Series in Operations Research & Management Science, in: William W. Cooper & Lawrence M. Seiford & Joe Zhu (ed.), Handbook on Data Envelopment Analysis, chapter 0, pages 241-271, Springer.
    6. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    7. Christensen, Laurits R & Jorgenson, Dale W & Lau, Lawrence J, 1973. "Transcendental Logarithmic Production Frontiers," The Review of Economics and Statistics, MIT Press, vol. 55(1), pages 28-45, February.
    8. Emrouznejad, Ali & Yang, Guo-liang, 2018. "A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016," Socio-Economic Planning Sciences, Elsevier, vol. 61(C), pages 4-8.
    9. Iustin Nedelea & James Fannin, 2013. "Technical efficiency of Critical Access Hospitals: an application of the two-stage approach with double bootstrap," Health Care Management Science, Springer, vol. 16(1), pages 27-36, March.
    10. Yu, Chunyan, 1998. "The effects of exogenous variables in efficiency measurement--A monte carlo study," European Journal of Operational Research, Elsevier, vol. 105(3), pages 569-580, March.
    11. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    12. Boussofiane, A. & Dyson, R. G. & Thanassoulis, E., 1991. "Applied data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 52(1), pages 1-15, May.
    13. Cordero, José Manuel & Santín, Daniel & Sicilia, Gabriela, 2015. "Testing the accuracy of DEA estimates under endogeneity through a Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 244(2), pages 511-518.
    14. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    15. Yun Zhang & Robert Bartels, 1998. "The Effect of Sample Size on the Mean Efficiency in DEA with an Application to Electricity Distribution in Australia, Sweden and New Zealand," Journal of Productivity Analysis, Springer, vol. 9(3), pages 187-204, March.
    16. Simar, Leopold & Wilson, Paul W., 2007. "Estimation and inference in two-stage, semi-parametric models of production processes," Journal of Econometrics, Elsevier, vol. 136(1), pages 31-64, January.
    17. Panagiotis Mitropoulos & Nikolaos Mastrogiannis & Ioannis Mitropoulos, 2014. "Seeking interactions between patient satisfaction and efficiency in primary healthcare: cluster and DEA analysis," International Journal of Multicriteria Decision Making, Inderscience Enterprises Ltd, vol. 4(3), pages 234-251.
    18. R. Allen & A. Athanassopoulos & R.G. Dyson & E. Thanassoulis, 1997. "Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions," Annals of Operations Research, Springer, vol. 73(0), pages 13-34, October.
    19. F Pedraja-Chaparro & J Salinas-Jiménez & P Smith, 1999. "On the quality of the data envelopment analysis model," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(6), pages 636-644, June.
    20. Banker, Rajiv D. & Gadh, Vandana M. & Gorr, Wilpen L., 1993. "A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 67(3), pages 332-343, June.
    21. Holland, D. S. & Lee, S. T., 2002. "Impacts of random noise and specification on estimates of capacity derived from data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 137(1), pages 10-21, February.
    22. Perelman, Sergio & Santín, Daniel, 2009. "How to generate regularly behaved production data? A Monte Carlo experimentation on DEA scale efficiency measurement," European Journal of Operational Research, Elsevier, vol. 199(1), pages 303-310, November.
    23. Krüger, Jens J., 2012. "A Monte Carlo study of old and new frontier methods for efficiency measurement," European Journal of Operational Research, Elsevier, vol. 222(1), pages 137-148.
    24. Korostelev, A. P. & Simar, L. & Tsybakov, A. B., 1995. "Estimation of monotone boundaries," LIDAM Reprints CORE 1178, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    25. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    26. William W. Cooper & José L. Ruiz & Inmaculada Sirvent, 2011. "Choices and Uses of DEA Weights," International Series in Operations Research & Management Science, in: William W. Cooper & Lawrence M. Seiford & Joe Zhu (ed.), Handbook on Data Envelopment Analysis, chapter 0, pages 93-126, Springer.
    27. Johannes Van Biesebroeck, 2007. "Robustness Of Productivity Estimates," Journal of Industrial Economics, Wiley Blackwell, vol. 55(3), pages 529-569, September.
    28. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    29. Peter Smith, 1997. "Model misspecification in Data Envelopment Analysis," Annals of Operations Research, Springer, vol. 73(0), pages 233-252, October.
    30. Francisco Pedraja-Chaparro & Javier Salinas-Jimenez & Peter Smith, 1997. "On the Role of Weight Restrictions in Data Envelopment Analysis," Journal of Productivity Analysis, Springer, vol. 8(2), pages 215-230, May.
    31. Resti, Andrea, 2000. "Efficiency measurement for multi-product industries: A comparison of classic and recent techniques based on simulated data," European Journal of Operational Research, Elsevier, vol. 121(3), pages 559-578, March.
    32. Tone, Kaoru, 2001. "A slacks-based measure of efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 130(3), pages 498-509, May.
    33. William W. Cooper & Lawrence M. Seiford & Kaoru Tone, 2007. "Data Envelopment Analysis," Springer Books, Springer, edition 0, number 978-0-387-45283-8, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wang, Derek D. & Ren, Yaoyao, 2024. "Accuracy of Deterministic Nonparametric Frontier Models with Undesirable Outputs," European Journal of Operational Research, Elsevier, vol. 315(2), pages 596-612.
    2. Khezrimotlagh, Dariush, 2022. "Simulation designs for production frontiers," European Journal of Operational Research, Elsevier, vol. 303(3), pages 1321-1334.
    3. Zarrin, Mansour & Brunner, Jens O., 2023. "Analyzing the accuracy of variable returns to scale data envelopment analysis models," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1286-1301.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zarrin, Mansour & Brunner, Jens O., 2023. "Analyzing the accuracy of variable returns to scale data envelopment analysis models," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1286-1301.
    2. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    3. Valentin Zelenyuk, 2019. "Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions," CEPA Working Papers Series WP072019, School of Economics, University of Queensland, Australia.
    4. Wang, Derek D. & Ren, Yaoyao, 2024. "Accuracy of Deterministic Nonparametric Frontier Models with Undesirable Outputs," European Journal of Operational Research, Elsevier, vol. 315(2), pages 596-612.
    5. Emrouznejad, Ali & De Witte, Kristof, 2010. "COOPER-framework: A unified process for non-parametric projects," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1573-1586, December.
    6. Luis R. Murillo‐Zamorano, 2004. "Economic Efficiency and Frontier Techniques," Journal of Economic Surveys, Wiley Blackwell, vol. 18(1), pages 33-77, February.
    7. Zelenyuk, Valentin, 2020. "Aggregation of inputs and outputs prior to Data Envelopment Analysis under big data," European Journal of Operational Research, Elsevier, vol. 282(1), pages 172-187.
    8. Mai, Nhat Chi, 2015. "Efficiency of the banking system in Vietnam under financial liberalization," OSF Preprints qsf6d, Center for Open Science.
    9. Mansour Zarrin & Jan Schoenfelder & Jens O. Brunner, 2022. "Homogeneity and Best Practice Analyses in Hospital Performance Management: An Analytical Framework," Health Care Management Science, Springer, vol. 25(3), pages 406-425, September.
    10. Perelman, Sergio & Santín, Daniel, 2009. "How to generate regularly behaved production data? A Monte Carlo experimentation on DEA scale efficiency measurement," European Journal of Operational Research, Elsevier, vol. 199(1), pages 303-310, November.
    11. Franz R. Hahn, 2007. "Determinants of Bank Efficiency in Europe. Assessing Bank Performance Across Markets," WIFO Studies, WIFO, number 31499.
    12. Jindal, Abhinav & Nilakantan, Rahul, 2021. "Falling efficiency levels of Indian coal-fired power plants: A slacks-based analysis," Energy Economics, Elsevier, vol. 93(C).
    13. Esteve, Miriam & Aparicio, Juan & Rodriguez-Sala, Jesus J. & Zhu, Joe, 2023. "Random Forests and the measurement of super-efficiency in the context of Free Disposal Hull," European Journal of Operational Research, Elsevier, vol. 304(2), pages 729-744.
    14. Zhou, P. & Ang, B.W. & Poh, K.L., 2008. "A survey of data envelopment analysis in energy and environmental studies," European Journal of Operational Research, Elsevier, vol. 189(1), pages 1-18, August.
    15. Adler, Nicole & Yazhemsky, Ekaterina, 2010. "Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction," European Journal of Operational Research, Elsevier, vol. 202(1), pages 273-284, April.
    16. Halkos, George & Tzeremes, Nickolaos, 2007. "Examining the relationship between firm internationalization and firm performance: A nonparametric analysis," MPRA Paper 32082, University Library of Munich, Germany.
    17. Vladimír Holý, 2022. "The impact of operating environment on efficiency of public libraries," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 30(1), pages 395-414, March.
    18. Lee, Boon L. & Wilson, Clevo & Simshauser, Paul & Majiwa, Eucabeth, 2021. "Deregulation, efficiency and policy determination: An analysis of Australia's electricity distribution sector," Energy Economics, Elsevier, vol. 98(C).
    19. Ioannis E. Tsolas, 2021. "Firm Credit Scoring: A Series Two-Stage DEA Bootstrapped Approach," JRFM, MDPI, vol. 14(5), pages 1-12, May.
    20. José Manuel Cordero & Cristina Polo & Daniel Santín & Gabriela Sicilia, 2016. "Monte-Carlo Comparison of Conditional Nonparametric Methods and Traditional Approaches to Include Exogenous Variables," Pacific Economic Review, Wiley Blackwell, vol. 21(4), pages 483-497, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:285:y:2020:i:3:p:1042-1057. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.