[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v307y2023i1p294-312.html
   My bibliography  Save this article

Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation

Author

Listed:
  • Ahn, Heinz
  • Clermont, Marcel
  • Langner, Julia
Abstract
In the last decades, various frontier-based efficiency measurement methods have been developed. So far, there is only limited knowledge with respect to the question to what extent a certain method is advantageous in different scenarios. We address this issue by a simulation study. In the respective scenarios, nine parameters are varied that reflect the specific structure of the data and knowledge about the competitive environment of the units to be evaluated. In contrast to other studies, we consider, e.g., multiple inputs and outputs. Furthermore, among other things, we assume lower levels of efficiency to reflect characteristics of markets with weaker competition. The methods compared are Data Envelopment Analysis, Stochastic Frontier Analysis, Stochastic Non-Smooth Envelopment of Data, and Normalized Additive Analysis. They are assessed with respect to two main purposes of efficiency measurement, the correct estimation of the efficiency degrees as well as the accurate determination of the ranking of the compared units. It is shown that, depending on the chosen purpose and the data scenario, there are major differences regarding the performance of the methods. The results of our study provide a comprehensive basis for selecting between the methods.

Suggested Citation

  • Ahn, Heinz & Clermont, Marcel & Langner, Julia, 2023. "Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 307(1), pages 294-312.
    • Handle: RePEc:eee:ejores:v:307:y:2023:i:1:p:294-312
    DOI: 10.1016/j.ejor.2022.09.039
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221722007676
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2022.09.039?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Papadopoulos, Alecos & Parmeter, Christopher F., 2021. "Type II failure and specification testing in the Stochastic Frontier Model," European Journal of Operational Research, Elsevier, vol. 293(3), pages 990-1001.
    2. Gong, Byeong-Ho & Sickles, Robin C., 1992. "Finite sample evidence on the performance of stochastic frontiers and data envelopment analysis using panel data," Journal of Econometrics, Elsevier, vol. 51(1-2), pages 259-284.
    3. Oleg Badunenko & Daniel J. Henderson & Subal C. Kumbhakar, 2012. "When, where and how to perform efficiency estimation," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(4), pages 863-892, October.
    4. Banker, Rajiv D. & Chang, Hsihui & Cooper, William W., 2004. "A simulation study of DEA and parametric frontier models in the presence of heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 153(3), pages 624-640, March.
    5. Costa, Marcelo Azevedo & Lopes, Ana Lúcia Miranda & de Pinho Matos, Giordano Bruno Braz, 2015. "Statistical evaluation of Data Envelopment Analysis versus COLS Cobb–Douglas benchmarking models for the 2011 Brazilian tariff revision," Socio-Economic Planning Sciences, Elsevier, vol. 49(C), pages 47-60.
    6. Henningsen, Arne & Bělín, Matěj & Henningsen, Géraldine, 2017. "New insights into the stochastic ray production frontier," Economics Letters, Elsevier, vol. 156(C), pages 18-21.
    7. Mark Andor & Christopher Parmeter, 2017. "Pseudolikelihood estimation of the stochastic frontier model," Applied Economics, Taylor & Francis Journals, vol. 49(55), pages 5651-5661, November.
    8. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    9. Yu, Chunyan, 1998. "The effects of exogenous variables in efficiency measurement--A monte carlo study," European Journal of Operational Research, Elsevier, vol. 105(3), pages 569-580, March.
    10. Tsionas, Mike G., 2021. "Optimal combinations of stochastic frontier and data envelopment analysis models," European Journal of Operational Research, Elsevier, vol. 294(2), pages 790-800.
    11. Waldman, Donald M., 1982. "A stationary point for the stochastic frontier likelihood," Journal of Econometrics, Elsevier, vol. 18(2), pages 275-279, February.
    12. Jondrow, James & Knox Lovell, C. A. & Materov, Ivan S. & Schmidt, Peter, 1982. "On the estimation of technical inefficiency in the stochastic frontier production function model," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 233-238, August.
    13. Bojani, Antonio N. & Caudill, Steven B. & Ford, Jon M., 1998. "Small-sample properties of ML, COLS, and DEA estimators of frontier models in the presence of heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 108(1), pages 140-148, July.
    14. Laura Di Giorgio & Abraham D Flaxman & Mark W Moses & Nancy Fullman & Michael Hanlon & Ruben O Conner & Alexandra Wollum & Christopher J L Murray, 2016. "Efficiency of Health Care Production in Low-Resource Settings: A Monte-Carlo Simulation to Compare the Performance of Data Envelopment Analysis, Stochastic Distance Functions, and an Ensemble Model," PLOS ONE, Public Library of Science, vol. 11(1), pages 1-20, January.
    15. Bryon J. Parman & Allen M. Featherstone, 2019. "A comparison of parametric and nonparametric estimation methods for cost frontiers and economic measures," Journal of Applied Economics, Taylor & Francis Journals, vol. 22(1), pages 60-85, January.
    16. Géraldine Henningsen & Arne Henningsen & Uwe Jensen, 2015. "A Monte Carlo study on multiple output stochastic frontiers: a comparison of two approaches," Journal of Productivity Analysis, Springer, vol. 44(3), pages 309-320, December.
    17. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    18. Olson, Jerome A. & Schmidt, Peter & Waldman, Donald M., 1980. "A Monte Carlo study of estimators of stochastic frontier production functions," Journal of Econometrics, Elsevier, vol. 13(1), pages 67-82, May.
    19. Meeusen, Wim & van den Broeck, Julien, 1977. "Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 18(2), pages 435-444, June.
    20. Resti, Andrea, 1997. "Evaluating the cost-efficiency of the Italian Banking System: What can be learned from the joint application of parametric and non-parametric techniques," Journal of Banking & Finance, Elsevier, vol. 21(2), pages 221-250, February.
    21. Banker, Rajiv D. & Gadh, Vandana M. & Gorr, Wilpen L., 1993. "A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 67(3), pages 332-343, June.
    22. Li, Hong-Zhou & Kopsakangas-Savolainen, Maria & Xiao, Xing-Zhi & Tian, Zhen-Zhen & Yang, Xiao-Yuan & Wang, Jian-Lin, 2016. "Cost efficiency of electric grid utilities in China: A comparison of estimates from SFA–MLE, SFA–Bayes and StoNED–CNLS," Energy Economics, Elsevier, vol. 55(C), pages 272-283.
    23. Battese, George E. & Coelli, Tim J., 1988. "Prediction of firm-level technical efficiencies with a generalized frontier production function and panel data," Journal of Econometrics, Elsevier, vol. 38(3), pages 387-399, July.
    24. Leopold Simar & Paul Wilson, 2010. "Inferences from Cross-Sectional, Stochastic Frontier Models," Econometric Reviews, Taylor & Francis Journals, vol. 29(1), pages 62-98.
    25. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    26. Heinz Ahn & Marcel Clermont & Julia Langner, 2022. "The impact of selected input and output factors on measuring research efficiency of university research fields: insights from a purpose-, field-, and method-specific perspective," Journal of Business Economics, Springer, vol. 92(8), pages 1303-1335, October.
    27. Mark Andor & Frederik Hesse, 2014. "The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA)," Journal of Productivity Analysis, Springer, vol. 41(1), pages 85-109, February.
    28. Oh, Seog-Chan & Shin, Jaemin, 2015. "The impact of mismeasurement in performance benchmarking: A Monte Carlo comparison of SFA and DEA with different multi-period budgeting strategies," European Journal of Operational Research, Elsevier, vol. 240(2), pages 518-527.
    29. Andrew L. Johnson & Timo Kuosmanen, 2015. "An Introduction to CNLS and StoNED Methods for Efficiency Analysis: Economic Insights and Computational Aspects," Springer Books, in: Subhash C. Ray & Subal C. Kumbhakar & Pami Dua (ed.), Benchmarking for Performance Evaluation, edition 127, chapter 0, pages 117-186, Springer.
    30. Léopold Simar & Paul W. Wilson, 2023. "Nonparametric, Stochastic Frontier Models with Multiple Inputs and Outputs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(4), pages 1391-1403, October.
    31. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    32. Fan, Yanqin & Li, Qi & Weersink, Alfons, 1996. "Semiparametric Estimation of Stochastic Production Frontier Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 460-468, October.
    33. Nieswand, Maria & Seifert, Stefan, 2018. "Environmental factors in frontier estimation – A Monte Carlo analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 133-148.
    34. Guilkey, David K & Lovell, C A Knox & Sickles, Robin C, 1983. "A Comparison of the Performance of Three Flexible Functional Forms," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 24(3), pages 591-616, October.
    35. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    36. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    37. Mark Andor & Frederik Hesse, "undated". "A Monte Carlo Simulation comparing DEA, SFA and two simple approaches to combine efficiency estimates," Working Papers 201177, Institute of Spatial and Housing Economics, Munster Universitary.
    38. Resti, Andrea, 2000. "Efficiency measurement for multi-product industries: A comparison of classic and recent techniques based on simulated data," European Journal of Operational Research, Elsevier, vol. 121(3), pages 559-578, March.
    39. Peter Bogetoft & Lars Otto, 2011. "Benchmarking with DEA, SFA, and R," International Series in Operations Research and Management Science, Springer, number 978-1-4419-7961-2, December.
    40. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    41. Timo Kuosmanen & Mika Kortelainen, 2012. "Stochastic non-smooth envelopment of data: semi-parametric frontier estimation subject to shape constraints," Journal of Productivity Analysis, Springer, vol. 38(1), pages 11-28, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timo Kuosmanen & Sheng Dai, 2023. "Modeling economies of scope in joint production: Convex regression of input distance function," Papers 2311.11637, arXiv.org.
    2. Wang, Derek D. & Ren, Yaoyao, 2024. "Accuracy of Deterministic Nonparametric Frontier Models with Undesirable Outputs," European Journal of Operational Research, Elsevier, vol. 315(2), pages 596-612.
    3. Yang, Dong & Liao, Shiguan & Venus Lun, Y.H & Bai, Xiwen, 2023. "Towards sustainable port management: Data-driven global container ports turnover rate assessment," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 175(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    2. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    3. Mark Andor & Frederik Hesse, 2014. "The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA)," Journal of Productivity Analysis, Springer, vol. 41(1), pages 85-109, February.
    4. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    5. Christopher F. Parmeter & Valentin Zelenyuk, 2019. "Combining the Virtues of Stochastic Frontier and Data Envelopment Analysis," Operations Research, INFORMS, vol. 67(6), pages 1628-1658, November.
    6. Heinz Ahn & Marcel Clermont & Julia Langner, 2022. "The impact of selected input and output factors on measuring research efficiency of university research fields: insights from a purpose-, field-, and method-specific perspective," Journal of Business Economics, Springer, vol. 92(8), pages 1303-1335, October.
    7. Mark Andor & Frederik Hesse, "undated". "The StoNED age: The Departure Into a New Era of Efficiency Analysis? An MC study Comparing StoNED and the "Oldies" (SFA and DEA)," Working Papers 201285, Institute of Spatial and Housing Economics, Munster Universitary.
    8. Krüger, Jens J., 2012. "A Monte Carlo study of old and new frontier methods for efficiency measurement," European Journal of Operational Research, Elsevier, vol. 222(1), pages 137-148.
    9. Isabel Narbón-Perpiñá & Maria Teresa Balaguer-Coll & Marko Petrović & Emili Tortosa-Ausina, 2020. "Which estimator to measure local governments’ cost efficiency? The case of Spanish municipalities," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 11(1), pages 51-82, March.
    10. Oleg Badunenko & Daniel J. Henderson & Subal C. Kumbhakar, 2012. "When, where and how to perform efficiency estimation," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(4), pages 863-892, October.
    11. Zangin Zeebari & Kristofer Månsson & Pär Sjölander & Magnus Söderberg, 2023. "Regularized conditional estimators of unit inefficiency in stochastic frontier analysis, with application to electricity distribution market," Journal of Productivity Analysis, Springer, vol. 59(1), pages 79-97, February.
    12. Mai, Nhat Chi, 2015. "Efficiency of the banking system in Vietnam under financial liberalization," OSF Preprints qsf6d, Center for Open Science.
    13. Olesen, Ole B. & Petersen, Niels Christian, 2016. "Stochastic Data Envelopment Analysis—A review," European Journal of Operational Research, Elsevier, vol. 251(1), pages 2-21.
    14. Cristina Polo & Julián Ramajo & Alejandro Ricci‐Risquete, 2021. "A stochastic semi‐non‐parametric analysis of regional efficiency in the European Union," Regional Science Policy & Practice, Wiley Blackwell, vol. 13(1), pages 7-24, February.
    15. Sakouvogui Kekoura & Shaik Saleem & Doetkott Curt & Magel Rhonda, 2021. "Sensitivity analysis of stochastic frontier analysis models," Monte Carlo Methods and Applications, De Gruyter, vol. 27(1), pages 71-90, March.
    16. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    17. Ondrich, Jan & Ruggiero, John, 2001. "Efficiency measurement in the stochastic frontier model," European Journal of Operational Research, Elsevier, vol. 129(2), pages 434-442, March.
    18. Chunping Liu & Audrey Laporte & Brian S. Ferguson, 2008. "The quantile regression approach to efficiency measurement: insights from Monte Carlo simulations," Health Economics, John Wiley & Sons, Ltd., vol. 17(9), pages 1073-1087, September.
    19. Alexander Arévalo S & Víctor Giménez G & Diego Prior J, 2022. "Análisis de eficiencia en educación: una aplicación del método StoNED," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 92(2), pages 45-91, October.
    20. Mark A. Andor & David H. Bernstein & Stephan Sommer, 2021. "Determining the efficiency of residential electricity consumption," Empirical Economics, Springer, vol. 60(6), pages 2897-2923, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:307:y:2023:i:1:p:294-312. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.