[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v258y2017i3p801-819.html
   My bibliography  Save this article

Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions

Author

Listed:
  • Werner, Christoph
  • Bedford, Tim
  • Cooke, Roger M.
  • Hanea, Anca M.
  • Morales-Nápoles, Oswaldo
Abstract
Many applications in decision making under uncertainty and probabilistic risk assessment require the assessment of multiple, dependent uncertain quantities, so that in addition to marginal distributions, interdependence needs to be modelled in order to properly understand the overall risk. Nevertheless, relevant historical data on dependence information are often not available or simply too costly to obtain. In this case, the only sensible option is to elicit this uncertainty through the use of expert judgements. In expert judgement studies, a structured approach to eliciting variables of interest is desirable so that their assessment is methodologically robust. One of the key decisions during the elicitation process is the form in which the uncertainties are elicited. This choice is subject to various, potentially conflicting, desiderata related to e.g. modelling convenience, coherence between elicitation parameters and the model, combining judgements, and the assessment burden for the experts. While extensive and systematic guidance to address these considerations exists for single variable uncertainty elicitation, for higher dimensions very little such guidance is available. Therefore, this paper offers a systematic review of the current literature on eliciting dependence. The literature on the elicitation of dependence parameters such as correlations is presented alongside commonly used dependence models and experience from case studies. From this, guidance about the strategy for dependence assessment is given and gaps in the existing research are identified to determine future directions for structured methods to elicit dependence.

Suggested Citation

  • Werner, Christoph & Bedford, Tim & Cooke, Roger M. & Hanea, Anca M. & Morales-Nápoles, Oswaldo, 2017. "Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions," European Journal of Operational Research, Elsevier, vol. 258(3), pages 801-819.
    • Handle: RePEc:eee:ejores:v:258:y:2017:i:3:p:801-819
    DOI: 10.1016/j.ejor.2016.10.018
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221716308517
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2016.10.018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ehrgott, Matthias & Klamroth, Kathrin & Schwehm, Christian, 2004. "An MCDM approach to portfolio optimization," European Journal of Operational Research, Elsevier, vol. 155(3), pages 752-770, June.
    2. repec:hal:pseose:halshs-00978020 is not listed on IDEAS
    3. Greco, Salvatore & Mousseau, Vincent & Słowiński, Roman, 2014. "Robust ordinal regression for value functions handling interacting criteria," European Journal of Operational Research, Elsevier, vol. 239(3), pages 711-730.
    4. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    5. Stephen Hora & Erim Kardeş, 2015. "Calibration, sharpness and the weighting of experts in a linear opinion pool," Annals of Operations Research, Springer, vol. 229(1), pages 429-450, June.
    6. Morales, O. & Kurowicka, D. & Roelen, A., 2008. "Eliciting conditional and unconditional rank correlations from conditional probabilities," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 699-710.
    7. von Winterfeldt, Detlof & Fasolo, Barbara, 2009. "Structuring decision problems: A case study and reflections for practitioners," European Journal of Operational Research, Elsevier, vol. 199(3), pages 857-866, December.
    8. Shafiqah Al-Awadhi & Paul Garthwaite, 2006. "Quantifying expert opinion for modelling fauna habitat distributions," Computational Statistics, Springer, vol. 21(1), pages 121-140, March.
    9. Moskowitz, Herbert & Bunn, Derek, 1987. "Decision and risk analysis," European Journal of Operational Research, Elsevier, vol. 28(3), pages 247-260, March.
    10. Philip M. Lurie & Matthew S. Goldberg, 1998. "An Approximate Method for Sampling Correlated Random Variables from Partially-Specified Distributions," Management Science, INFORMS, vol. 44(2), pages 203-218, February.
    11. Ouchi, Fumika, 2004. "A literature review on the use of expert opinion in probabilistic risk analysis," Policy Research Working Paper Series 3201, The World Bank.
    12. Dietrich, Franz & List, Christian, 2014. "Probabilistic Opinion Pooling," MPRA Paper 54806, University Library of Munich, Germany.
    13. Ali E. Abbas, 2009. "Multiattribute Utility Copulas," Operations Research, INFORMS, vol. 57(6), pages 1367-1383, December.
    14. Bernd Kraan & Tim Bedford, 2005. "Probabilistic Inversion of Expert Judgments in the Quantification of Model Uncertainty," Management Science, INFORMS, vol. 51(6), pages 995-1006, June.
    15. Dubois, Didier & Prade, Henri & Sabbadin, Regis, 2001. "Decision-theoretic foundations of qualitative possibility theory," European Journal of Operational Research, Elsevier, vol. 128(3), pages 459-478, February.
    16. Mkrtchyan, L. & Podofillini, L. & Dang, V.N., 2015. "Bayesian belief networks for human reliability analysis: A review of applications and gaps," Reliability Engineering and System Safety, Elsevier, vol. 139(C), pages 1-16.
    17. Donald L. Keefer & Samuel E. Bodily, 1983. "Three-Point Approximations for Continuous Random Variables," Management Science, INFORMS, vol. 29(5), pages 595-609, May.
    18. Robert T. Clemen & Gregory W. Fischer & Robert L. Winkler, 2000. "Assessing Dependence: Some Experimental Results," Management Science, INFORMS, vol. 46(8), pages 1100-1115, August.
    19. Rita Esther Zapata-V�zquez & Anthony O'Hagan & Leonardo Soares Bastos, 2014. "Eliciting expert judgements about a set of proportions," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(9), pages 1919-1933, September.
    20. Herbert Moskowitz & Rakesh K. Sarin, 1983. "Improving the Consistency of Conditional Probability Assessments for Forecasting and Decision Making," Management Science, INFORMS, vol. 29(6), pages 735-749, June.
    21. Figueira, José Rui & Greco, Salvatore & Slowinski, Roman, 2009. "Building a set of additive value functions representing a reference preorder and intensities of preference: GRIP method," European Journal of Operational Research, Elsevier, vol. 195(2), pages 460-486, June.
    22. A. O'Hagan & E. B. Glennie & R. E. Beardsall, 1992. "Subjective Modelling and Bayes Linear Estimation in the Uk Water Industry," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 41(3), pages 563-577, November.
    23. Blaszczynski, Jerzy & Greco, Salvatore & Slowinski, Roman, 2007. "Multi-criteria classification - A new scheme for application of dominance-based decision rules," European Journal of Operational Research, Elsevier, vol. 181(3), pages 1030-1044, September.
    24. Roger Cooke, 2013. "Uncertainty analysis comes to integrated assessment models for climate change…and conversely," Climatic Change, Springer, vol. 117(3), pages 467-479, April.
    25. Percy, David F., 2002. "Bayesian enhanced strategic decision making for reliability," European Journal of Operational Research, Elsevier, vol. 139(1), pages 133-145, May.
    26. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2001. "Rough sets theory for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 129(1), pages 1-47, February.
    27. Cooke, Roger M. & Kelly, G.N., 2010. "Climate Change Uncertainty Quantification: Lessons Learned from the Joint EU-USNRC Project on Uncertainty Analysis of Probabilistic Accident Consequence Codes," RFF Working Paper Series dp-10-29, Resources for the Future.
    28. Bradley, R. & Dietrich, F.K. & List, C., 2007. "Aggregating causal judgements," Research Memorandum 001, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    29. Ali E. Abbas & David V. Budescu & Yuhong (Rola) Gu, 2010. "Assessing Joint Distributions with Isoprobability Contours," Management Science, INFORMS, vol. 56(6), pages 997-1011, June.
    30. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2009. "Sensitivity to Distance and Baseline Distributions in Forecast Evaluation," Management Science, INFORMS, vol. 55(4), pages 582-590, April.
    31. R. E. J. Neslo & R. M. Cooke, 2011. "Modeling and validating stakeholder preferences with probabilistic inversion," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 27(2), pages 115-130, March.
    32. Arbenz, Philipp & Canestraro, Davide, 2012. "Estimating Copulas for Insurance from Scarce Observations, Expert Opinion and Prior Information: A Bayesian Approach," ASTIN Bulletin, Cambridge University Press, vol. 42(1), pages 271-290, May.
    33. Samuel Kotz & Johan René van Dorp, 2010. "Generalized Diagonal Band Copulas with Two-Sided Generating Densities," Decision Analysis, INFORMS, vol. 7(2), pages 196-214, June.
    34. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    35. Christian Genest & Michel Gendron & Michaël Bourdeau-Brien, 2009. "The Advent of Copulas in Finance," The European Journal of Finance, Taylor & Francis Journals, vol. 15(7-8), pages 609-618.
    36. Hanea, Anca & Morales Napoles, Oswaldo & Ababei, Dan, 2015. "Non-parametric Bayesian networks: Improving theory and reviewing applications," Reliability Engineering and System Safety, Elsevier, vol. 144(C), pages 265-284.
    37. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2004. "Axiomatic characterization of a general utility function and its particular cases in terms of conjoint measurement and rough-set decision rules," European Journal of Operational Research, Elsevier, vol. 158(2), pages 271-292, October.
    38. Paul H. Garthwaite & Shafeeqah A. Al‐Awadhi, 2001. "Non‐conjugate prior distribution assessment for multivariate normal sampling," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(1), pages 95-110.
    39. Cooke, Roger M. & Goossens, Louis L.H.J., 2008. "TU Delft expert judgment data base," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 657-674.
    40. Paul H. Garthwaite & Shafeeqah A. Al-Awadhi & Fadlalla G. Elfadaly & David J. Jenkinson, 2013. "Prior distribution elicitation for generalized linear and piecewise-linear models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 40(1), pages 59-75, January.
    41. Garthwaite, Paul H. & Kadane, Joseph B. & O'Hagan, Anthony, 2005. "Statistical Methods for Eliciting Probability Distributions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 680-701, June.
    42. Paul L. Fackler, 1991. "Modeling Interdependence: An Approach to Simulation and Elicitation," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 73(4), pages 1091-1097.
    43. Robert T. Clemen & Terence Reilly, 1999. "Correlations and Copulas for Decision and Risk Analysis," Management Science, INFORMS, vol. 45(2), pages 208-224, February.
    44. Fadlalla Elfadaly & Paul Garthwaite, 2013. "Eliciting Dirichlet and Connor–Mosimann prior distributions for multinomial models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(4), pages 628-646, November.
    45. van Dorp, J. Rene, 2005. "Statistical dependence through common risk factors: With applications in uncertainty analysis," European Journal of Operational Research, Elsevier, vol. 161(1), pages 240-255, February.
    46. Marichal, Jean-Luc, 2004. "Tolerant or intolerant character of interacting criteria in aggregation by the Choquet integral," European Journal of Operational Research, Elsevier, vol. 155(3), pages 771-791, June.
    47. Luca Regis, 2011. "A Bayesian copula model for stochastic claims reserving," Carlo Alberto Notebooks 227, Collegio Carlo Alberto.
    48. James E. Smith & Detlof von Winterfeldt, 2004. "Anniversary Article: Decision Analysis in Management Science," Management Science, INFORMS, vol. 50(5), pages 561-574, May.
    49. Jyrki Wallenius & James S. Dyer & Peter C. Fishburn & Ralph E. Steuer & Stanley Zionts & Kalyanmoy Deb, 2008. "Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead," Management Science, INFORMS, vol. 54(7), pages 1336-1349, July.
    50. Angilella, Silvia & Greco, Salvatore & Lamantia, Fabio & Matarazzo, Benedetto, 2004. "Assessing non-additive utility for multicriteria decision aid," European Journal of Operational Research, Elsevier, vol. 158(3), pages 734-744, November.
    51. Carl S. Spetzler & Carl-Axel S. Staël Von Holstein, 1975. "Exceptional Paper--Probability Encoding in Decision Analysis," Management Science, INFORMS, vol. 22(3), pages 340-358, November.
    52. Grabisch, Michel, 1996. "The application of fuzzy integrals in multicriteria decision making," European Journal of Operational Research, Elsevier, vol. 89(3), pages 445-456, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lotte Yanore & Jaap Sok & Alfons Oude Lansink, 2024. "Do Dutch farmers invest in expansion despite increased policy uncertainty? A participatory Bayesian network approach," Agribusiness, John Wiley & Sons, Ltd., vol. 40(1), pages 93-115, January.
    2. Irina Vinogradova-Zinkevič, 2021. "Application of Bayesian Approach to Reduce the Uncertainty in Expert Judgments by Using a Posteriori Mean Function," Mathematics, MDPI, vol. 9(19), pages 1-23, October.
    3. Supper, Hendrik & Irresberger, Felix & Weiß, Gregor, 2020. "A comparison of tail dependence estimators," European Journal of Operational Research, Elsevier, vol. 284(2), pages 728-742.
    4. Ferretti, Valentina & Montibeller, Gilberto & von Winterfeldt, Detlof, 2023. "Testing the effectiveness of debiasing techniques to reduce overprecision in the elicitation of subjective continuous probability distributions," LSE Research Online Documents on Economics 115333, London School of Economics and Political Science, LSE Library.
    5. Ahti Salo & Edoardo Tosoni & Juho Roponen & Derek W. Bunn, 2022. "Using cross‐impact analysis for probabilistic risk assessment," Futures & Foresight Science, John Wiley & Sons, vol. 4(2), June.
    6. Abroon Qazi & Mecit Can Emre Simsekler, 2022. "Prioritizing interdependent drivers of financial, economic, and political risks using a data-driven probabilistic approach," Risk Management, Palgrave Macmillan, vol. 24(2), pages 164-185, June.
    7. Abroon Qazi & Mecit Can Emre Simsekler & Steven Formaneck, 2023. "Supply chain risk network value at risk assessment using Bayesian belief networks and Monte Carlo simulation," Annals of Operations Research, Springer, vol. 322(1), pages 241-272, March.
    8. Rafael Schwarzenegger & John Quigley & Lesley Walls, 2023. "Is eliciting dependency worth the effort? A study for the multivariate Poisson-Gamma probability model," Journal of Risk and Reliability, , vol. 237(5), pages 858-867, October.
    9. Kim, Byung-Cheol, 2022. "Multi-factor dependence modelling with specified marginals and structured association in large-scale project risk assessment," European Journal of Operational Research, Elsevier, vol. 296(2), pages 679-695.
    10. Christoph Werner & Tim Bedford & John Quigley, 2018. "Sequential Refined Partitioning for Probabilistic Dependence Assessment," Risk Analysis, John Wiley & Sons, vol. 38(12), pages 2683-2702, December.
    11. Ferretti, Valentina & Montibeller, Gilberto & von Winterfeldt, Detlof, 2023. "Testing the effectiveness of debiasing techniques to reduce overprecision in the elicitation of subjective continuous probability distributions," European Journal of Operational Research, Elsevier, vol. 304(2), pages 661-675.
    12. Suo, Weilan & Wang, Lin & Li, Jianping, 2021. "Probabilistic risk assessment for interdependent critical infrastructures: A scenario-driven dynamic stochastic model," Reliability Engineering and System Safety, Elsevier, vol. 214(C).
    13. Wang, Fan & Li, Heng & Dong, Chao & Ding, Lieyun, 2019. "Knowledge representation using non-parametric Bayesian networks for tunneling risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    14. Lotte Yanore & Jaap Sok & Alfons Oude Lansink, 2023. "Anticipate, wait or don't invest? The strategic net present value approach to study expansion decisions under policy uncertainty," Agribusiness, John Wiley & Sons, Ltd., vol. 39(2), pages 535-548, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beccacece, Francesca & Borgonovo, Emanuele & Buzzard, Greg & Cillo, Alessandra & Zionts, Stanley, 2015. "Elicitation of multiattribute value functions through high dimensional model representations: Monotonicity and interactions," European Journal of Operational Research, Elsevier, vol. 246(2), pages 517-527.
    2. Wang, Fan & Li, Heng & Dong, Chao & Ding, Lieyun, 2019. "Knowledge representation using non-parametric Bayesian networks for tunneling risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    3. Tim Bedford & Alireza Daneshkhah & Kevin J. Wilson, 2016. "Approximate Uncertainty Modeling in Risk Analysis with Vine Copulas," Risk Analysis, John Wiley & Sons, vol. 36(4), pages 792-815, April.
    4. Durbach, Ian N. & Stewart, Theodor J., 2012. "A comparison of simplified value function approaches for treating uncertainty in multi-criteria decision analysis," Omega, Elsevier, vol. 40(4), pages 456-464.
    5. Michel Grabisch & Christophe Labreuche, 2010. "A decade of application of the Choquet and Sugeno integrals in multi-criteria decision aid," Annals of Operations Research, Springer, vol. 175(1), pages 247-286, March.
    6. Mayag, Brice & Bouyssou, Denis, 2020. "Necessary and possible interaction between criteria in a 2-additive Choquet integral model," European Journal of Operational Research, Elsevier, vol. 283(1), pages 308-320.
    7. De Bock, Koen W. & Coussement, Kristof & Caigny, Arno De & Słowiński, Roman & Baesens, Bart & Boute, Robert N. & Choi, Tsan-Ming & Delen, Dursun & Kraus, Mathias & Lessmann, Stefan & Maldonado, Sebast, 2024. "Explainable AI for Operational Research: A defining framework, methods, applications, and a research agenda," European Journal of Operational Research, Elsevier, vol. 317(2), pages 249-272.
    8. Koen W. de Bock & Kristof Coussement & Arno De Caigny & Roman Slowiński & Bart Baesens & Robert N Boute & Tsan-Ming Choi & Dursun Delen & Mathias Kraus & Stefan Lessmann & Sebastián Maldonado & David , 2023. "Explainable AI for Operational Research: A Defining Framework, Methods, Applications, and a Research Agenda," Post-Print hal-04219546, HAL.
    9. Durbach, Ian N. & Stewart, Theodor J., 2012. "Modeling uncertainty in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 223(1), pages 1-14.
    10. Zhao Qiaojiao & Zeng Ling & Liu Jinjin, 2016. "Fuzzy Integral Multiple Criteria Decision Making Method Based on Fuzzy Preference Relation on Alternatives," Journal of Systems Science and Information, De Gruyter, vol. 4(3), pages 280-290, June.
    11. Doumpos, Michael & Zopounidis, Constantin, 2011. "Preference disaggregation and statistical learning for multicriteria decision support: A review," European Journal of Operational Research, Elsevier, vol. 209(3), pages 203-214, March.
    12. Tianyang Wang & James S. Dyer, 2012. "A Copulas-Based Approach to Modeling Dependence in Decision Trees," Operations Research, INFORMS, vol. 60(1), pages 225-242, February.
    13. Silvia Angilella & Marta Bottero & Salvatore Corrente & Valentina Ferretti & Salvatore Greco & Isabella M. Lami, 2016. "Non Additive Robust Ordinal Regression for urban and territorial planning: an application for siting an urban waste landfill," Annals of Operations Research, Springer, vol. 245(1), pages 427-456, October.
    14. Anil Gaba & Dana G. Popescu & Zhi Chen, 2019. "Assessing Uncertainty from Point Forecasts," Management Science, INFORMS, vol. 65(1), pages 90-106, January.
    15. Greco, Salvatore & Mousseau, Vincent & Słowiński, Roman, 2014. "Robust ordinal regression for value functions handling interacting criteria," European Journal of Operational Research, Elsevier, vol. 239(3), pages 711-730.
    16. Zheng, Jun & Lienert, Judit, 2018. "Stakeholder interviews with two MAVT preference elicitation philosophies in a Swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS," European Journal of Operational Research, Elsevier, vol. 267(1), pages 273-287.
    17. Fadlalla G. Elfadaly & Paul H. Garthwaite, 2020. "On quantifying expert opinion about multinomial models that contain covariates," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 959-981, June.
    18. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    19. Kadziński, Miłosz & Wójcik, Michał & Ciomek, Krzysztof, 2022. "Review and experimental comparison of ranking and choice procedures for constructing a univocal recommendation in a preference disaggregation setting," Omega, Elsevier, vol. 113(C).
    20. Hu, Qiwei & Chakhar, Salem & Siraj, Sajid & Labib, Ashraf, 2017. "Spare parts classification in industrial manufacturing using the dominance-based rough set approach," European Journal of Operational Research, Elsevier, vol. 262(3), pages 1136-1163.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:258:y:2017:i:3:p:801-819. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.