LIP6 1997/033
- Thesis
Apprentissage statistique et régularisation pour la régression - C. Goutte
- 137 pages - 07/15/1997- document en - http://www.lip6.fr/lip6/reports/1997/lip6.1997.033.ps.gz - 485 Ko
- Contact : cg (at) nulleivind.imm.dtu.dk
- Ancien Thème : APA
- Keywords : Statistical learning, regularisation, generalisation, neural networks, time series
- Publisher : Valerie.Mangin (at) nulllip6.fr
This thesis deals with the use of statistical learning and regularisation on regression problems, with a focus on time series modelling and system identification. Both linear models and non-linear neural networks are considered as particular modelling techniques.
Linear and non-linear parametric regression are briefly introduced and their limit is shown using the bias-variance decomposition of the generalisation error. We then show that as such, those problems are ill-posed, and thus need to be regularised. Regularisation introduces a number of hyper-parameters, the setting of which is performed by estimating generalisation error. Several such methods are evoked in the course of this work.
The use of these theoretical aspects is targeted towards two particular problems. First an iterative method relying on generalisation error to extract the relevant delays from time series data is presented. Then a particular regularisation functional is studied, that provides pruning of unnecessary parameters as well as a regularising effect. This last part uses Bayesian estimators, and a brief presentation of those estimators is also given in the thesis.
Linear and non-linear parametric regression are briefly introduced and their limit is shown using the bias-variance decomposition of the generalisation error. We then show that as such, those problems are ill-posed, and thus need to be regularised. Regularisation introduces a number of hyper-parameters, the setting of which is performed by estimating generalisation error. Several such methods are evoked in the course of this work.
The use of these theoretical aspects is targeted towards two particular problems. First an iterative method relying on generalisation error to extract the relevant delays from time series data is presented. Then a particular regularisation functional is studied, that provides pruning of unnecessary parameters as well as a regularising effect. This last part uses Bayesian estimators, and a brief presentation of those estimators is also given in the thesis.