Regression on Principal Components and PLS Regression

REG3

Duration : 1.0 day(s)

:: Course Summary

This workshop covers the problem of correlated explanatory variables (multicollinearity) in regression models, its impact on the model performance and discusses PLS regression, the most popular technique to deal with this problem.

It first introduces the underlying concepts in PLS regression through its link with ultivariate methods, namely Principal Component Analysis (PCA). Focus is also put on strategies used to measure the predictive ability of models and their usefulness to optimise the PLS model-building phase.

Case studies are used so that participants gain confidence in the interpretation of software output and to make them aware of the conditions of use and common pitfalls.

:: Learning Objectives

Upon completion of this course, participants will be able to:

Know the context of use for regression on principal components and partial least squares regression

Understand what the underlying assumptions of the techniques are

Construct the regression models

Assess the goodness-of-fit of the models to the data

Identify common issues in these regression models, diagnose problems and fix them

Interpret statistical software output

:: Target Audience

This applied training session in statistics is aimed at all scientific staff who collect data and who must make decisions based on them. The regression techniques covered in this session will be particularly useful for people who are interested in relating/predicting a variable to/from a single or a set of explanatory variables.

:: Prerequisite

This one-day session covers two advanced regression techniques used to relate response variables to a set of predictor variables.

Participants must know the mechanism underlying simple and multiple linear regression techniques, interpretation of the data, model validation, common problems in regression, etc.

Either by having attended the training session Linear and Multiple Linear Regression Techniques or by possessing a similar background.

Participants must also know the mechanism underlying principal componernt analysis as this multivariate technique is used in both advanced methods covered in this session

Either by having attended the training session Principal Component Analysis and its Applications or by possessing a similar background.

:: Topics Covered

Regression on Principal Components (PCR)
- Principle and methods
- The essentiels of PCR
- PCR in practice
- Selection of principal components
- Utilization of PCR results
- Utilization of factor analysis instead of PCA
- Solftware packages for PCR
- Summary
« Partial Least Squares » PLS Regression
- Context of use
- Applications
- Working with several response variables
- Some ideas in PLS regression
- Steps involved in the method
- Extraction of principal components
- Latent variables
- One or several response variables?
Pratical Implementation of PLS
- Initial Data
- General Principle
- Selection of principal components
- Type of results generated by statistical software
- Results of PLS regression : Application to the NFL data
- Another example : The pine data
Conclusion
- PLS : An alternative to multiple linear regression
- Scope of PLS regression
- Limitations of PLS
- Some references

:: Course Content

This training describes regression on principal components and PLS regression, advanced regression techniques used to solve problems in the study on several explanatory variables: multicollinearity, few observations vs. number of variables, missing data, etc.

The training covers the objectives of the two advanced regression techniques, principle of modeling , parameter estimation, interpretation of model coefficients , goodness-of-fit and validation , prediction in regression, problems commonly encountered and ways to detect them