Regression Models for Categorical Data

REG2

Duration : 1.0 day(s)

:: Course Summary

This workshop deals with regression models for categorical response variables. A strong emphasis is put on logistic regression.
Fundamental principles underlying binary data modeling is first explored through the inadequacy of linear regression. Then the alternative model logit or logistic regression is presented. Similarities with linear regression are pointed out and tools specific to logistic regression are covered in detail:

odds ratio used to quantify the magnitude of effects

specificity, sensitivity, ROC curves

cross-validation techniques.
Generalization of the logistic regression model to the case in which the response variable has more than levels is illustrated.

:: Learning Objectives

Upon completion of this course, participants will be able to:

Explain the context of use of logistic regression

Understand why ordinary regression fails for the analysis of categorical response variables

Construct a logistic regression model

Consider the assumptions and conditions underlying logistic regression

Assess the goodness-of-fit of the model to the data

Interpret statistical software output

Understand how ordinal logistic regression works

Understand how polytomous regression works

:: Target Audience

This applied training session in statistics is aimed at all scientific staff who collect categorical data and who must make decisions based on them. The regression techniques covered in this session will be particularly useful for people who deal with qualitative response variables (measurements) in finance, epidemiology, medicine, genetics, social sciences, econometrics and marketing.

:: Prerequisite

This one-day training session covers logistic regression, a statistical technique used to study the relationship between a categorical dependent variable and a set of explanatory or independent variables.

Participants should know the essential tools in statistics - descriptive statistics, both numerical (mean, standard deviation, standard error, etc.) and graphical (histogram, box-plot, scatter plot, etc.), hypothesis testing and confidence intervals - either by having attended the training session Fundamental Tools in Statistics or by possessing a similar background.

A working knowledge of ordinary multiple regression techniques is desirable but not mandatory. Basic concepts in multiple regression will be reviewed at the beginning of the training, before covering logistic regression.

:: Topics Covered

Introduction to Logistic Regression
- Goal: To Study the Relationship between a Categorical Variable and a Set of Explanatory Variables
- Why Does Ordinary Multiple Linear Regression Fail for the Analysis of a Categorical Response Variable?
Refresher on Multiple Linear Regression
- Estimation of the Model
- Interpretation of the Coefficients of the Model Parameters
- Goodness-of-Fit and Validation Techniques
Classical Case: a Binary Response Variable
- Basic Principle: Modeling the probability of observing a given value of the reponse variable
- Example
- Interpretation of Statistical Software Output: Coefficients and Mathematical Transformations, Odds Ratios, Statistical Testing of Model Coefficients
- Comparison of Logistic Regression Software Output with Multiple Linear Regression
- Goodness-of-Fit Measures: Nested Models, Cross-Validation Techniques
- Parallel with Discriminant Analysis
- Using the Model for Prediction
- Principle of Variable Selection
Case of an Ordinal Response Variable: Ordinal Logistic Regression
Case of a Nominal Response Variable: Polytomous Regression
Practical Considerations
- Procedures Available in Statistical Software
- Implementation and Interpretation
Summary

:: Course Content

This session covers logistic regression , a technique used whenever the response variable is binary or categorical, that is, when it can only take on a limited number of values. An example of a categorical variable is the severity of a disease: not severe, mildly severe, moderately severe, very severe. An example of a binary variable is the survival of patients who received particular treatments: yes or no.

The training starts with a refresher on multiple linear regression a statistical modeling technique used to relate a continuous response variable to a set of explanatory variables. The concepts of modeling, parameter estimation, as well as interpretation of model coefficients, goodness-of-fit and validation measures will be reviewed.

Why ordinary multiple linear regression is no longer the appropriate technique to use when the response variable is discussed.

The most classical case of logistic regression is then covered, that is the relationship between a categorical reponse variable and a set of explanatory variables. The modeling principle underlying the method is defined (the probability of observing a given value of the response variable) and illustrated with the help of examples. Time is also devoted to the interpretation of statistical software output so that attendees can learn how to extract the pertinent information for the analysis of data and the interpretation of results. Some indicators are specific to logistic regression: odds ratios, tests on coefficients. Goodness-of-fit and validation measures specific to logistic regression are also covered.

The use of the regression models for prediction and the principle of variable selection are also discussed during the training session.

Ordinal logistic regression and polytomous regression are also reviewed for ordinal and nominal response variables, respectively.

Finally, the different procedures available in statistical software will be listed and ways to implement the regression techniques will be discussed.