[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v94y2009i11p1735-1763.html
   My bibliography  Save this article

Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models

Author

Listed:
  • Storlie, Curtis B.
  • Swiler, Laura P.
  • Helton, Jon C.
  • Sallaberry, Cedric J.
Abstract
The analysis of many physical and engineering problems involves running complex computational models (simulation models, computer codes). With problems of this type, it is important to understand the relationships between the input variables (whose values are often imprecisely known) and the output. The goal of sensitivity analysis (SA) is to study this relationship and identify the most significant factors or variables affecting the results of the model. In this presentation, an improvement on existing methods for SA of complex computer models is described for use when the model is too computationally expensive for a standard Monte-Carlo analysis. In these situations, a meta-model or surrogate model can be used to estimate the necessary sensitivity index for each input. A sensitivity index is a measure of the variance in the response that is due to the uncertainty in an input. Most existing approaches to this problem either do not work well with a large number of input variables and/or they ignore the error involved in estimating a sensitivity index. Here, a new approach to sensitivity index estimation using meta-models and bootstrap confidence intervals is described that provides solutions to these drawbacks. Further, an efficient yet effective approach to incorporate this methodology into an actual SA is presented. Several simulated and real examples illustrate the utility of this approach. This framework can be extended to uncertainty analysis as well.

Suggested Citation

  • Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    • Handle: RePEc:eee:reensy:v:94:y:2009:i:11:p:1735-1763
    DOI: 10.1016/j.ress.2009.05.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832009001112
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2009.05.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Description of techniques," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 28-54.
    2. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, François & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
    3. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    4. Helton, J.C. & Johnson, J.D. & Sallaberry, C.J. & Storlie, C.B., 2006. "Survey of sampling-based methods for uncertainty and sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1175-1209.
    5. Gramacy, Robert B & Lee, Herbert K. H, 2008. "Bayesian Treed Gaussian Process Models With an Application to Computer Modeling," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1119-1130.
    6. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    7. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Example results," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 55-77.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    2. Touzani, Samir & Busby, Daniel, 2013. "Smoothing spline analysis of variance approach for global sensitivity analysis of computer codes," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 67-81.
    3. Blatman, Géraud & Sudret, Bruno, 2010. "Efficient computation of global sensitivity indices using sparse polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 95(11), pages 1216-1229.
    4. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    5. Helton, Jon C. & Johnson, Jay D. & Sallaberry, Cédric J., 2011. "Quantification of margins and uncertainties: Example analyses from reactor safety and radioactive waste disposal involving the separation of aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1014-1033.
    6. Gatelli, D. & Kucherenko, S. & Ratto, M. & Tarantola, S., 2009. "Calculating first-order sensitivity measures: A benchmark of some recent methodologies," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1212-1219.
    7. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Margins associated with loss of assured safety for systems with multiple weak links and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    8. Helton, Jon C. & Hansen, Clifford W. & Sallaberry, Cédric J., 2012. "Uncertainty and sensitivity analysis in performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 44-63.
    9. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    10. Helton, Jon C. & Pilch, Martin & Sallaberry, Cédric J., 2014. "Probability of loss of assured safety in systems with multiple time-dependent failure modes: Representations with aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 124(C), pages 171-200.
    11. Daneshkhah, Alireza & Bedford, Tim, 2013. "Probabilistic sensitivity analysis of system availability using Gaussian processes," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 82-93.
    12. Crestaux, Thierry & Le Maıˆtre, Olivier & Martinez, Jean-Marc, 2009. "Polynomial chaos expansion for sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1161-1172.
    13. Tatsuya Sakurahara & Seyed Reihani & Ernie Kee & Zahra Mohaghegh, 2020. "Global importance measure methodology for integrated probabilistic risk assessment," Journal of Risk and Reliability, , vol. 234(2), pages 377-396, April.
    14. Mirko Ginocchi & Ferdinanda Ponci & Antonello Monti, 2021. "Sensitivity Analysis and Power Systems: Can We Bridge the Gap? A Review and a Guide to Getting Started," Energies, MDPI, vol. 14(24), pages 1-59, December.
    15. Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.
    16. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    17. Shahsavani, D. & Grimvall, A., 2009. "An adaptive design and interpolation technique for extracting highly nonlinear response surfaces from deterministic models," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1173-1182.
    18. Hansen, C.W. & Behie, G.A. & Bier, A. & Brooks, K.M. & Chen, Y. & Helton, J.C. & Hommel, S.P. & Lee, K.P. & Lester, B. & Mattie, P.D. & Mehta, S. & Miller, S.P. & Sallaberry, C.J. & Sevougian, S.D. & , 2014. "Uncertainty and sensitivity analysis for the nominal scenario class in the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 122(C), pages 272-296.
    19. Di Maio, Francesco & Bandini, Alessandro & Zio, Enrico & Alberola, Sofia Carlos & Sanchez-Saez, Francisco & Martorell, Sebastián, 2016. "Bootstrapped-ensemble-based Sensitivity Analysis of a trace thermal-hydraulic model based on a limited number of PWR large break loca simulations," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 122-134.
    20. Nagel, Joseph B. & Rieckermann, Jörg & Sudret, Bruno, 2020. "Principal component analysis and sparse polynomial chaos expansions for global sensitivity analysis and model calibration: Application to urban drainage simulation," Reliability Engineering and System Safety, Elsevier, vol. 195(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:94:y:2009:i:11:p:1735-1763. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.