[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v53y2008i2p298-311.html
   My bibliography  Save this article

Boosting additive models using component-wise P-Splines

Author

Listed:
  • Schmid, Matthias
  • Hothorn, Torsten
Abstract
An efficient approximation of L2 Boosting with component-wise smoothing splines is considered. Smoothing spline base-learners are replaced by P-spline base-learners, which yield similar prediction errors but are more advantageous from a computational point of view. A detailed analysis of the effect of various P-spline hyper-parameters on the boosting fit is given. In addition, a new theoretical result on the relationship between the boosting stopping iteration and the step length factor used for shrinking the boosting estimates is derived.

Suggested Citation

  • Schmid, Matthias & Hothorn, Torsten, 2008. "Boosting additive models using component-wise P-Splines," Computational Statistics & Data Analysis, Elsevier, vol. 53(2), pages 298-311, December.
    • Handle: RePEc:eee:csdana:v:53:y:2008:i:2:p:298-311
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(08)00441-6
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marx, Brian D. & Eilers, Paul H. C., 1998. "Direct generalized additive modeling with penalized likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 28(2), pages 193-209, August.
    2. Buhlmann P. & Yu B., 2003. "Boosting With the L2 Loss: Regression and Classification," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 324-339, January.
    3. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    4. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, September.
    5. Gerhard Tutz & Harald Binder, 2006. "Generalized Additive Modeling with Implicit Variable Selection by Likelihood-Based Boosting," Biometrics, The International Biometric Society, vol. 62(4), pages 961-971, December.
    6. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, September.
    7. Clifford M. Hurvich & Jeffrey S. Simonoff & Chih‐Ling Tsai, 1998. "Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 271-293.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Guilherme Schultz Lindenmeyer & Hudson Silva Torrent, 2024. "Boosting and Predictability of Macroeconomic Variables: Evidence from Brazil," Computational Economics, Springer;Society for Computational Economics, vol. 64(1), pages 377-409, July.
    2. Christmann, Andreas & Hable, Robert, 2012. "Consistency of support vector machines using additive kernels for additive models," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 854-873.
    3. Hansen, Bjørn Gunnar, 2019. "CO2 – Emission, costs and capacity of different manure management practices- results from an advisory project," Agricultural Systems, Elsevier, vol. 173(C), pages 325-334.
    4. Hendrik van der Wurp & Andreas Groll, 2023. "Introducing LASSO-type penalisation to generalised joint regression modelling for count data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 107(1), pages 127-151, March.
    5. Xin Fang & Bo Fang & Chunfang Wang & Tian Xia & Matteo Bottai & Fang Fang & Yang Cao, 2019. "Comparison of Frequentist and Bayesian Generalized Additive Models for Assessing the Association between Daily Exposure to Fine Particles and Respiratory Mortality: A Simulation Study," IJERPH, MDPI, vol. 16(5), pages 1-20, March.
    6. Kauppi, Heikki & Virtanen, Timo, 2021. "Boosting nonlinear predictability of macroeconomic time series," International Journal of Forecasting, Elsevier, vol. 37(1), pages 151-170.
    7. Guilherme Lindenmeyer & Pedro Pablo Skorin & Hudson da Silva Torrent, 2021. "Using boosting for forecasting electric energy consumption during a recession: a case study for the Brazilian State Rio Grande do Sul," Letters in Spatial and Resource Sciences, Springer, vol. 14(2), pages 111-128, August.
    8. Simon N. Wood, 2020. "Inference and computation with generalized additive models and their extensions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(2), pages 307-339, June.
    9. Holger Reulen & Thomas Kneib, 2016. "Boosting multi-state models," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 22(2), pages 241-262, April.
    10. Emiko Dupont & Nicole H. Augustin, 2024. "Spatial Confounding and Spatial+ for Nonlinear Covariate Effects," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 29(3), pages 455-470, September.
    11. Benjamin Hofner & Torsten Hothorn & Thomas Kneib, 2013. "Variable selection and model choice in structured survival models," Computational Statistics, Springer, vol. 28(3), pages 1079-1101, June.
    12. Heikki Kauppi & Timo Virtanen, 2018. "Boosting Non-linear Predictabilityof Macroeconomic Time Series," Discussion Papers 124, Aboa Centre for Economics.
    13. Wang Zhu, 2011. "HingeBoost: ROC-Based Boost for Classification and Variable Selection," The International Journal of Biostatistics, De Gruyter, vol. 7(1), pages 1-30, February.
    14. Carlos A. Manzanares & Ying Jiang & Patrick Bajari, 2015. "Improving Policy Functions in High-Dimensional Dynamic Games," NBER Working Papers 21124, National Bureau of Economic Research, Inc.
    15. Souhaib Ben Taieb & Rob J Hyndman, 2014. "Boosting multi-step autoregressive forecasts," Monash Econometrics and Business Statistics Working Papers 13/14, Monash University, Department of Econometrics and Business Statistics.
    16. Sobotka, Fabian & Kneib, Thomas, 2012. "Geoadditive expectile regression," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 755-767.
    17. Nikolay Robinzonov & Gerhard Tutz & Torsten Hothorn, 2012. "Boosting techniques for nonlinear time series models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 99-122, January.
    18. Groll Andreas & Kneib Thomas & Mayr Andreas & Schauberger Gunther, 2018. "On the dependency of soccer scores – a sparse bivariate Poisson model for the UEFA European football championship 2016," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 14(2), pages 65-79, June.
    19. Möst Lisa & Hothorn Torsten, 2015. "Conditional Transformation Models for Survivor Function Estimation," The International Journal of Biostatistics, De Gruyter, vol. 11(1), pages 23-50, May.
    20. Souhaib Ben Taieb & Raphael Huser & Rob J. Hyndman & Marc G. Genton, 2015. "Probabilistic time series forecasting with boosted additive models: an application to smart meter data," Monash Econometrics and Business Statistics Working Papers 12/15, Monash University, Department of Econometrics and Business Statistics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    2. Baccini, Michela & Biggeri, Annibale & Lagazio, Corrado & Lertxundi, Aitana & Saez, Marc, 2007. "Parametric and semi-parametric approaches in the analysis of short-term effects of air pollution on health," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4324-4336, May.
    3. Simon N. Wood, 2011. "Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 73(1), pages 3-36, January.
    4. Belitz, Christiane & Lang, Stefan, 2008. "Simultaneous selection of variables and smoothing parameters in structured additive regression models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 61-81, September.
    5. Strasak, Alexander M. & Umlauf, Nikolaus & Pfeiffer, Ruth M. & Lang, Stefan, 2011. "Comparing penalized splines and fractional polynomials for flexible modelling of the effects of continuous predictor variables," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1540-1551, April.
    6. Nagler Thomas & Schellhase Christian & Czado Claudia, 2017. "Nonparametric estimation of simplified vine copula models: comparison of methods," Dependence Modeling, De Gruyter, vol. 5(1), pages 99-120, January.
    7. Morteza Amini & Mahdi Roozbeh & Nur Anisah Mohamed, 2024. "Separation of the Linear and Nonlinear Covariates in the Sparse Semi-Parametric Regression Model in the Presence of Outliers," Mathematics, MDPI, vol. 12(2), pages 1-17, January.
    8. I. Gijbels & I. Prosdocimi & G. Claeskens, 2010. "Nonparametric estimation of mean and dispersion functions in extended generalized linear models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 19(3), pages 580-608, November.
    9. I. Gijbels & I. Prosdocimi, 2011. "Smooth estimation of mean and dispersion function in extended generalized additive models with application to Italian induced abortion data," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(11), pages 2391-2411, December.
    10. Takuma Yoshida & Kanta Naito, 2014. "Asymptotics for penalised splines in generalised additive models," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 26(2), pages 269-289, June.
    11. David O'Donnell & Alastair Rushworth & Adrian W. Bowman & E. Marian Scott & Mark Hallard, 2014. "Flexible regression models over river networks," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 63(1), pages 47-63, January.
    12. Takuma Yoshida, 2019. "Two stage smoothing in additive models with missing covariates," Statistical Papers, Springer, vol. 60(6), pages 1803-1826, December.
    13. Beck, Jonathan, 2008. "Diderot´s rule," Discussion Papers, Research Unit: Competition and Innovation SP II 2008-13, WZB Berlin Social Science Center.
    14. Roca-Pardinas, Javier & Cadarso-Suarez, Carmen & Tahoces, Pablo G. & Lado, Maria J., 2008. "Assessing continuous bivariate effects among different groups through nonparametric regression models: An application to breast cancer detection," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 1958-1970, January.
    15. Hübler, Olaf, 2017. "Health and Body Mass Index: No Simple Relationship," IZA Discussion Papers 10620, Institute of Labor Economics (IZA).
    16. Gerhard Tutz & Jan Gertheiss, 2014. "Rating Scales as Predictors—The Old Question of Scale Level and Some Answers," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 357-376, July.
    17. Simon N. Wood & Zheyuan Li & Gavin Shaddick & Nicole H. Augustin, 2017. "Generalized Additive Models for Gigadata: Modeling the U.K. Black Smoke Network Daily Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1199-1210, July.
    18. Marc Aerts & Niel Hens & Jeffrey Simonoff, 2010. "Model selection in regression based on pre-smoothing," Journal of Applied Statistics, Taylor & Francis Journals, vol. 37(9), pages 1455-1472.
    19. Simon N. Wood, 2020. "Inference and computation with generalized additive models and their extensions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(2), pages 307-339, June.
    20. Adrian W. Bowman & Marco Giannitrapani & E. Marian Scott, 2009. "Spatiotemporal smoothing and sulphur dioxide trends over Europe," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(5), pages 737-752, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:53:y:2008:i:2:p:298-311. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.