### Linear Regression

#### Introduction

#### Simple Regression

- Linear Regression
- Regression Example
- Residual Analysis
- Transformations
- Influential Points
- Slope Estimate
- Slope Test

#### Multiple Regression

### Linear Regession: Table of Contents

#### Introduction

#### Simple Regression

- Linear Regression
- Regression Example
- Residual Analysis
- Transformations
- Influential Points
- Slope Estimate
- Slope Test

#### Multiple Regression

# Transformations of Variables

When a residual plot reveals a data set to be nonlinear, it is often possible to "transform" the raw data to make it more linear. This allows us to use linear regression techniques more effectively with nonlinear data.

**Note:** Your browser does not support HTML5 video. If you view this web page on a different browser
(e.g., a recent version of Edge, Chrome, Firefox, or Opera), you can watch a video treatment of this lesson.

## What is a Transformation to Achieve Linearity?

Transforming a variable involves using a mathematical operation to change its measurement scale. Broadly speaking, there are two kinds of transformations.

**Linear transformation.**A linear transformation preserves linear relationships between variables. Therefore, the correlation between*x*and*y*would be unchanged after a linear transformation. Examples of a linear transformation to variable*x*would be multiplying*x*by a constant, dividing*x*by a constant, or adding a constant to*x*.**Nonlinear tranformation.**A nonlinear transformation changes (increases or decreases) linear relationships between variables and, thus, changes the correlation between variables. Examples of a nonlinear transformation of variable*x*would be taking the square root of*x*or the reciprocal of*x*.

In regression, a transformation to achieve linearity is a
special kind of nonlinear transformation. It is a nonlinear
transformation that *increases* the linear
relationship between two variables.

## Methods of Transforming Variables to Achieve Linearity

There are many ways to transform variables to achieve linearity for regression analysis. Some common methods are summarized below.

Method | Transform | Regression equation | Predicted value (ŷ) |
---|---|---|---|

Standard linear regression | None | y = b_{0} + b_{1}x |
ŷ = b_{0} + b_{1}x |

Exponential model | DV = log(y) | log(y) = b_{0} + b_{1}x |
ŷ = 10^{b0 + b1x} |

Quadratic model | DV = sqrt(y) | sqrt(y) = b_{0} + b_{1}x |
ŷ = ( b_{0} + b_{1}x )^{2} |

Reciprocal model | DV = 1/y | 1/y = b_{0} + b_{1}x |
ŷ = 1 / ( b_{0} + b_{1}x ) |

Logarithmic model | IV = log(x) | y= b_{0} + b_{1}log(x) |
ŷ = b_{0} + b_{1}log(x) |

Power model | DV = log(y)
IV = log(x) |
log(y)= b_{0} + b_{1}log(x) |
ŷ = 10^{b0 + b1log(x)} |

Each row shows a different nonlinear transformation method. The second column shows the specific transformation applied to dependent and/or independent variables. The third column shows the regression equation used in the analysis. And the last column shows the "back transformation" equation used to restore the dependent variable to its original, non-transformed measurement scale.

In practice, these methods need to be tested on the
data to which they are applied to be sure that they
*increase* rather than *decrease* the linearity
of the relationship. Testing the effect of a transformation
method involves looking at
residual
plots and correlation coefficients, as described in the
following sections.

**Note:** The logarithmic model and the power model
require the ability to work with
logarithms.
Use a
graphic calculator
to obtain the log of a number or to transform back from the logarithm
to the original number.
If you need it, the Stat Trek glossary has a brief
refresher on logarithms.

## How to Perform a Transformation to Achieve Linearity

Transforming a data set to enhance linearity is a multi-step, trial-and-error process.

- Conduct a standard regression analysis on the raw data.
- Construct a residual plot.
- If the plot pattern is random, do not transform data.
- If the plot pattern is not random, continue.

- Compute the
coefficient of determination (R
^{2}). - Choose a transformation method (see above table).
- Transform the independent variable, dependent variable, or both.
- Conduct a regression analysis, using the transformed variables.
- Compute the coefficient of determination (R
^{2}), based on the transformed variables.- If the tranformed R
^{2}is greater than the raw-score R^{2}, the transformation was successful. Congratulations! - If not, try a different transformation method.

- If the tranformed R

The best tranformation method (exponential model, quadratic
model, reciprocal model, etc.) will depend on nature of the
original data. The only way to determine which method is best
is to try each and compare the result (i.e.,
residual
plots, correlation coefficients). The best method will yield the highest coefficient of determination (R^{2}).

## A Transformation Example

The table shows data for independent and dependent variables - x and y, respectively.

x | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
---|---|---|---|---|---|---|---|---|---|

y | 2 | 1 | 6 | 14 | 15 | 30 | 40 | 74 | 75 |

When we apply a linear regression to the untransformed raw data, the residual plot shows a non-random pattern (a U-shaped curve), which suggests that the data are nonlinear.

Suppose we repeat the analysis, using a quadratic model to transform the dependent variable. For a quadratic model, we use the square root of y, rather than y, as the dependent variable. Using the transformed data, our regression equation is:

y'_{t} = b_{0} + b_{1}x

where

y_{t} = transformed dependent variable, which is equal to
the square root of y

y'_{t} = predicted value of the transformed dependent variable y_{t}

x = independent variable

b_{0} = y-intercept of transformation regression line

b_{1} = slope of transformation regression line

The table below shows the transformed data we analyzed.

x | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
---|---|---|---|---|---|---|---|---|---|

y_{t} |
1.41 | 1.00 | 2.45 | 3.74 | 3.87 | 5.48 | 6.32 | 8.60 | 8.66 |

Since the transformation was based on the quadratic model
(y_{t} = the square root of y),
the transformation regression equation can be expressed in terms of the
original units of variable Y as:

y' = ( b_{0} + b_{1}x )^{2}

where

y' = predicted value of y in its orginal units

x = independent variable

b_{0} = y-intercept of transformation regression line

b_{1} = slope of transformation regression line

The residual plot above shows residuals based on predicted raw scores from the transformation regression equation. The plot suggests that the transformation to achieve linearity was successful. The pattern of residuals is random, suggesting that the relationship between the independent variable (x) and the transformed dependent variable (square root of y) is linear. And the coefficient of determination was 0.96 with the transformed data versus only 0.88 with the raw data. The transformed data resulted in a better model.

## Test Your Understanding

**Problem**

In the context of regression analysis, which of the following statements is true?

I. A linear transformation increases the linear relationship
between variables.

II. A logarithmic model is the most effective transformation method.

III. A residual plot reveals departures from linearity.

(A) I only

(B) II only

(C) III only

(D) I and II only

(E) I, II, and III

**Solution**

The correct answer is (C). A linear transformation neither increases nor
decreases the linear relationship between variables; it preserves the
relationship. A *nonlinear* transformation is used to
increase the relationship between variables.
The most effective transformation method depends on the data
being transformed. In some cases, a logarithmic model may be more
effective than other methods; but it other cases it may be less
effective.
Non-random patterns in a
residual plot
suggest a departure from linearity in the data being plotted.

# Teach Yourself Flexbox

- Build responsive websites with simple CSS.
- No floats, no workarounds, no javascript.

Bestsellers Statistics and Probability Updated daily | ||

1. How to Lie with Statistics $13.95 $8.06 | ||

2. Naked Statistics: Stripping the Dread from the Data $16.95 $10.89 | ||

3. Practical Statistics for Data Scientists: 50 Essential Concepts $49.99 $27.49 | ||

4. Statistics For Dummies (For Dummies (Math & Science)) $19.99 $11.99 | ||

5. The Cartoon Guide to Statistics $19.99 $12.19 |

Texas Instruments TI-83 Plus Graphing Calculator, Standard $149.99 $83.99 44% off | |

See more Graphing Calculators ... |

Bestsellers Handheld Calculators Updated daily | ||

1. Texas Instruments TI-89 Titanium Graphing Calculator (packaging may differ) $128.45 $128.45 | ||

2. Sharp EL-W535B WriteView Scientific Calculator $24.99 $78.95 | ||

3. Texas Instruments TI-84 Plus CE Graphing Calculator, Black $150.00 $121.28 | ||

4. Texas Instruments Ti-84 plus Graphing calculator - Black $105.00 $105.00 | ||

5. HP 12CP Financial Calculator $79.99 $51.99 |

Statistics Hacks: Tips & Tools for Measuring the World and Beating the Odds $29.99 $4.99 83% off | |

See more Statistics books ... |

Bestsellers Advanced Placement Statistics Updated daily | ||

1. Ultimate AP Statistics Practice Book: 100 Essential Problems Completely Explained on YouTube $14.70 $14.70 | ||

2. Barron's AP Statistics, 9th Edition $18.99 $30.85 | ||

3. Barron's AP Statistics, 8th Edition $18.99 $13.95 | ||

4. Cracking the AP Statistics Exam, 2019 Edition: Practice Tests & Proven Techniques to Help You Score a 5 (College Test Preparation) $19.99 $24.00 | ||

5. 5 Steps to a 5: AP Statistics 2018 $18.00 $14.19 |

# Teach Yourself to Shoot

- A scientific approach to marksmanship training.
- For hand-gunners who want to be marksmen.