[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v4y2007i4p227-242.html
   My bibliography  Save this article

How Effective Are Decision Analyses? Assessing Decision Process and Group Alignment Effects

Author

Listed:
  • Martin S. Schilling

    (Department of Management, London School of Economics, London, WC2A 2AE, United Kingdom and Decision Institute, Berlin, Germany)

  • Nadine Oeser

    (Decision Institute, 10117 Berlin, Germany)

  • Cornelius Schaub

    (Department of Management, London School of Economics, London, WC2A 2AE, United Kingdom and Decision Institute, Berlin, Germany)

Abstract
In this paper, we analyze approaches to assess the effectiveness of decision analyses. We develop an effectiveness framework, categorized in metrics to assess the quality of the decision process (“process effectiveness”), the quality of the model results (“output effectiveness”), and the quality of the long-term consequences of the analysis (“outcome effectiveness”). With a focus on the first two dimensions, “process” and “output,” we introduce two new approaches to assess the effectiveness of decision analyses applied in organizations. First, a new process effectiveness approach serves to compare the effectiveness of decision analyses to existing decision processes. Second, we assess output effectiveness with a before/after preference measurement design, which aids to capture the alignment of group members quantitatively when using decision analysis. We used six case studies, implemented during an applied research project, to test the two measures.

Suggested Citation

  • Martin S. Schilling & Nadine Oeser & Cornelius Schaub, 2007. "How Effective Are Decision Analyses? Assessing Decision Process and Group Alignment Effects," Decision Analysis, INFORMS, vol. 4(4), pages 227-242, December.
    • Handle: RePEc:inm:ordeca:v:4:y:2007:i:4:p:227-242
    DOI: 10.1287/deca.1070.0101
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1070.0101
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.1070.0101?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Timmermans, Danielle & Vlek, Charles, 1996. "Effects on Decision Quality of Supporting Multi-attribute Evaluation in Groups," Organizational Behavior and Human Decision Processes, Elsevier, vol. 68(2), pages 158-170, November.
    2. Timmermans, Danielle & Vlek, Charles, 1994. "An Evaluation Study of the Effectiveness of Multi-attribute Decision Support as a Function of Problem Complexity," Organizational Behavior and Human Decision Processes, Elsevier, vol. 59(1), pages 75-92, July.
    3. Gerd Islei & Geoff Lockett & Barry Cox & Steve Gisbourne & Mike Stratford, 1991. "Modeling Strategic Decision Making and Performance Measurements at ICI Pharmaceuticals," Interfaces, INFORMS, vol. 21(6), pages 4-22, December.
    4. Peter Todd & Izak Benbasat, 1999. "Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection," Information Systems Research, INFORMS, vol. 10(4), pages 356-374, December.
    5. Bill Wooldridge & Steven W. Floyd, 1990. "The strategy process, middle management involvement, and organizational performance," Strategic Management Journal, Wiley Blackwell, vol. 11(3), pages 231-241, March.
    6. Ramesh Sharda & Steve H. Barr & James C. McDonnell, 1988. "Decision Support System Effectiveness: A Review and an Empirical Test," Management Science, INFORMS, vol. 34(2), pages 139-159, February.
    7. Robert E. Quinn & John Rohrbaugh, 1983. "A Spatial Model of Effectiveness Criteria: Towards a Competing Values Approach to Organizational Analysis," Management Science, INFORMS, vol. 29(3), pages 363-377, March.
    8. Steven W. Floyd & Bill Wooldridge, 1997. "Middle Management’s Strategic Influence and Organizational Performance," Journal of Management Studies, Wiley Blackwell, vol. 34(3), pages 465-485, May.
    9. Anne T. McCartt & John Rohrbaugh, 1995. "Managerial Openness to Change and the Introduction of GDSS: Explaining Initial Success and Failure in Decision Conferencing," Organization Science, INFORMS, vol. 6(5), pages 569-584, October.
    10. Bettman, James R. & Johnson, Eric J. & Payne, John W., 1990. "A componential analysis of cognitive effort in choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 45(1), pages 111-139, February.
    11. Dale L. Goodhue, 1995. "Understanding User Evaluations of Information Systems," Management Science, INFORMS, vol. 41(12), pages 1827-1844, December.
    12. Robert L. Ferguson & Curtis H. Jones, 1969. "A Computer Aided Decision System," Management Science, INFORMS, vol. 15(10), pages 550-561, June.
    13. Ronald A. Howard, 1980. "An Assessment of Decision Analysis," Operations Research, INFORMS, vol. 28(1), pages 4-27, February.
    14. Sainfort, Francois C. & Gustafson, David H. & Bosworth, Kris & Hawkins, Robert P., 1990. "Decision support systems effectiveness: Conceptual framework and empirical evaluation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 45(2), pages 232-252, April.
    15. William H. DeLone & Ephraim R. McLean, 1992. "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, INFORMS, vol. 3(1), pages 60-95, March.
    16. Robert T. Clemen & Robert C. Kwit, 2001. "The Value of Decision Analysis at Eastman Kodak Company, 1990-1999," Interfaces, INFORMS, vol. 31(5), pages 74-92, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barbara Bini & Milena Vainieri & Sabina Nuti, 2015. "Definizione delle priorit? di intervento in sanit?: approcci socio-tecnici a confronto," MECOSAN, FrancoAngeli Editore, vol. 2015(93), pages 49-73.
    2. Oussama Marrouni Alami & Otmane Bouksour & Zitouni Beidouri, 2015. "An Intelligent Project Management Maturity Model for Moroccan Engineering Companies," Vikalpa: The Journal for Decision Makers, , vol. 40(2), pages 191-208, June.
    3. Ryszard Rutka & Piotr Wróbel & Ewa Wycinka, 2023. "Team members’ direct participation in decision-making processes and the quality of decisions," Journal of Entrepreneurship, Management and Innovation, Fundacja Upowszechniająca Wiedzę i Naukę "Cognitione", vol. 19(3), pages 169-201.
    4. Hugo Herrera & Nuno Videira & Hubert P.L.M. Korzilius & Kathya Lorena Cordova‐Pozo & Marleen H.F. McCardle‐Keurentjes, 2022. "Reflecting on factors influencing long‐lasting organisational effects of group model‐building interventions," System Dynamics Review, System Dynamics Society, vol. 38(2), pages 190-209, April.
    5. Ram, Camelia, 2020. "Scenario presentation and scenario generation in multi-criteria assessments: An exploratory study," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    6. van Amsterdam, Jan & Phillips, Lawrence D. & Henderson, Graeme & Bell, James & Bowden-Jones, Owen & Hammersley, Richard & Ramsey, John & Taylor, Polly & Dale-Perera, Annette & Melichar, Jan & van den , 2015. "Ranking the harm of non-medically used prescription opioids in the UK," LSE Research Online Documents on Economics 64653, London School of Economics and Political Science, LSE Library.
    7. Yeawon Yoo & Adolfo R. Escobedo, 2021. "A New Binary Programming Formulation and Social Choice Property for Kemeny Rank Aggregation," Decision Analysis, INFORMS, vol. 18(4), pages 296-320, December.
    8. L. Robin Keller, 2009. "From the Editor..," Decision Analysis, INFORMS, vol. 6(3), pages 121-123, September.
    9. Marleen McCardle-Keurentjes & Etiënne A. J. A. Rouwette, 2018. "Asking Questions: A Sine Qua Non of Facilitation in Decision Support?," Group Decision and Negotiation, Springer, vol. 27(5), pages 757-788, October.
    10. G Montibeller & L A Franco, 2011. "Raising the bar: strategic multi-criteria decision analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(5), pages 855-867, May.
    11. Luis C. Dias & Gabriela D. Oliveira & Paula Sarabando, 2021. "Choice-based preference disaggregation concerning vehicle technologies," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 29(1), pages 177-200, March.
    12. S. Bajracharya & G. Carenini & B. Chamberlain & K. Chen & D. Klein & D. Poole & H. Taheri & G. Öberg, 2018. "Interactive Visualization for Group Decision Analysis," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(06), pages 1839-1864, November.
    13. Franco, L. Alberto & Montibeller, Gilberto, 2010. "Facilitated modelling in operational research," European Journal of Operational Research, Elsevier, vol. 205(3), pages 489-500, September.
    14. L. Robin Keller, 2010. "From the Editor..," Decision Analysis, INFORMS, vol. 7(3), pages 235-237, September.
    15. Ricardo J. G. Mateus & João C. Bana e Costa & Pedro Verga Matos, 2017. "Supporting Multicriteria Group Decisions with MACBETH Tools: Selection of Sustainable Brownfield Redevelopment Actions," Group Decision and Negotiation, Springer, vol. 26(3), pages 495-521, May.
    16. L. Robin Keller & Kelly M. Kophazi, 2009. "From the Editors..," Decision Analysis, INFORMS, vol. 6(2), pages 53-56, June.
    17. Christopher W. Karvetski & James H. Lambert, 2012. "Evaluating deep uncertainties in strategic priority‐setting with an application to facility energy investments," Systems Engineering, John Wiley & Sons, vol. 15(4), pages 483-493, December.
    18. Ana Sofia Morais & Wasilios Hariskos, 2018. "Academic coaching and decision analysis: Ways of deciding whether to pursue an academic career," PLOS ONE, Public Library of Science, vol. 13(11), pages 1-18, November.
    19. Ben Ewing & Erin Baker, 2009. "Development of a Green Building Decision Support Tool: A Collaborative Process," Decision Analysis, INFORMS, vol. 6(3), pages 172-185, September.
    20. L. Robin Keller, 2011. "Investment and Defense Strategies, Heuristics, and Games: From the Editor ..," Decision Analysis, INFORMS, vol. 8(1), pages 1-3, March.
    21. Franco, L. Alberto & Rouwette, Etiënne A.J.A. & Korzilius, Hubert, 2016. "Different paths to consensus? The impact of need for closure on model-supported group conflict management," European Journal of Operational Research, Elsevier, vol. 249(3), pages 878-889.
    22. Rakesh K. Sarin & L. Robin Keller, 2013. "From the Editors ---Group Decisions, Preference Elicitation, Experienced Utility, Survival Probabilities, and Portfolio Value of Information," Decision Analysis, INFORMS, vol. 10(2), pages 99-102, June.
    23. Anthony Culyer, 2009. "Deliberative Processes in Decisions about Health Care Technologies," Briefing 000231, Office of Health Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Todd & Izak Benbasat, 1999. "Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection," Information Systems Research, INFORMS, vol. 10(4), pages 356-374, December.
    2. Gary L. Lilien & Arvind Rangaswamy & Gerrit H. Van Bruggen & Katrin Starke, 2004. "DSS Effectiveness in Marketing Resource Allocation Decisions: Reality vs. Perception," Information Systems Research, INFORMS, vol. 15(3), pages 216-235, September.
    3. Chuan-Hoo Tan & Hock-Hai Teo & Izak Benbasat, 2010. "Assessing Screening and Evaluation Decision Support Systems: A Resource-Matching Approach," Information Systems Research, INFORMS, vol. 21(2), pages 305-326, June.
    4. Carlos Martin-Rios, 2016. "Innovative management control systems in knowledge work: a middle manager perspective," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 27(2), pages 181-204, May.
    5. Patrick Krieger & Carsten Lausberg, 2021. "Entscheidungen, Entscheidungsfindung und Entscheidungsunterstützung in der Immobilienwirtschaft: Eine systematische Literaturübersicht [Decisions, decision-making and decisions support systems in r," Zeitschrift für Immobilienökonomie (German Journal of Real Estate Research), Springer;Gesellschaft für Immobilienwirtschaftliche Forschung e. V., vol. 7(1), pages 1-33, April.
    6. Mi, Hwang, 1998. "Did Task Type Matter in the Use of Decision Room GSS? A Critical Review and a Meta-analysis," Omega, Elsevier, vol. 26(1), pages 1-15, February.
    7. Louise Lindbjerg & Theodor Vladasel, 2021. "Hiring Entrepreneurs for Innovation," Working Papers 1309, Barcelona School of Economics.
    8. Daniel Qi Chen & Yanlin Zhang & Jinghua Xiao & Kang Xie, 2021. "Making Digital Innovation Happen: A Chief Information Officer Issue Selling Perspective," Information Systems Research, INFORMS, vol. 32(3), pages 987-1008, September.
    9. Samir L. Vaz & Anneloes M. L. Raes & Mariano L. M. Heyden, 2022. "Realizing implementation through relational exchanges with top managers: the mediating role of middle managers’ divergent strategic behavior," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 33(1), pages 81-108, March.
    10. repec:dau:papers:123456789/7962 is not listed on IDEAS
    11. Gökce Şükran & Arıcıoğlu M. Atilla, 2023. "Strategic Entrepreneurship from Middle Management Perspective," Management Theory and Studies for Rural Business and Infrastructure Development, Sciendo, vol. 45(3), pages 287-298, September.
    12. Jingguo Wang & Yuan Li & H. Raghav Rao, 2017. "Coping Responses in Phishing Detection: An Investigation of Antecedents and Consequences," Information Systems Research, INFORMS, vol. 28(2), pages 378-396, June.
    13. Dwyer, Sean & Richard, Orlando C. & Chadwick, Ken, 2003. "Gender diversity in management and firm performance: the influence of growth orientation and organizational culture," Journal of Business Research, Elsevier, vol. 56(12), pages 1009-1019, December.
    14. Melchor-Ferrer, Elias & Buendía-Carrillo, Dionisio, 2014. "Financial information management for university departments, using open-source software," International Journal of Information Management, Elsevier, vol. 34(2), pages 191-199.
    15. Henri Barki & Ryad Titah & Céline Boffo, 2007. "Information System Use--Related Activity: An Expanded Behavioral Conceptualization of Individual-Level Information System Use," Information Systems Research, INFORMS, vol. 18(2), pages 173-192, June.
    16. Mariano L. M. Heyden & Sebastian P. L. Fourné & Bastiaan A. S. Koene & Renate Werkman & Shahzad (Shaz) Ansari, 2017. "Rethinking ‘Top‐Down’ and ‘Bottom‐Up’ Roles of Top and Middle Managers in Organizational Change: Implications for Employee Support," Journal of Management Studies, Wiley Blackwell, vol. 54(7), pages 961-985, November.
    17. Liguo Lou & Zilu Tian & Joon Koh, 2017. "Tourist Satisfaction Enhancement Using Mobile QR Code Payment: An Empirical Investigation," Sustainability, MDPI, vol. 9(7), pages 1-14, July.
    18. Andreas I. Nicolaou & D. Harrison McKnight, 2006. "Perceived Information Quality in Data Exchanges: Effects on Risk, Trust, and Intention to Use," Information Systems Research, INFORMS, vol. 17(4), pages 332-351, December.
    19. Sun, Jonghak & Teng, James T.C., 2017. "The construct of information systems use benefits: Theoretical explication of its underlying dimensions and the development of a measurement scale," International Journal of Information Management, Elsevier, vol. 37(5), pages 400-416.
    20. Steven W. Floyd & Bill Wooldridge, 1999. "Knowledge Creation and Social Networks in Corporate Entrepreneurship: The Renewal of Organizational Capability," Entrepreneurship Theory and Practice, , vol. 23(3), pages 123-144, April.
    21. Gianina MIHAI, 2017. "End Users and ERP Systems’ Success. Three Models," Risk in Contemporary Economy, "Dunarea de Jos" University of Galati, Faculty of Economics and Business Administration, pages 491-506.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:4:y:2007:i:4:p:227-242. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.