Environmental Data Analysis BC
ENV 3017
Statistics 4:
Correlations
and Linear Regressions
Linear regression estimates a linear equation that describes the
relationship between two variables, correlation measures the strength
of that relationship.
Simple linear regression
- regression tutorial
- regression is used to
analyze the predictive relationship between a set of variables, called predictor variables, and another
variable, called the dependent
variable.
- in simple regression,
we have only two variables,, usually designated as X (the predictor variable) and Y (the dependent variable).
- one type of regression is linear regression in which the value of
the dependent variable is estimated by a linear combination of values
of the predictor (or independent) variables.
- in simple linear regression we predict the value of Y using the
equation: y=a+bx
- y is the dependent variable
- x is the independent, or predictor variable
- a and b are the coefficients, intercept and slope (=rise/run)
- residuals are the gaps between the predicted curve and the data
points (in vertical direction) Fig
- the idea of the linear least
square regression is to find the coefficients by minimizing the
sum of the squared residuals
- sum of square residuals = sum
(oberved
Yi - prediced Yi)2 = sum (Yi
- (a+bXi))2
- REGRESSION is an
Excel function
in the data analysis tool pack that calculates all the coefficients (as
well as the statistics of the
correlation)
Pearson correlation
coefficient
The (Pearson) correlation
coefficient r is a measure of the strength of a linear
relationship. It
ranges from -1 to +1. Some examples:
Negative Linear Relationship
Positive linear relationship
No correlation
The square of r, r2or R2, is called the coefficient of determination. It
measures the percentage of variation in the values of the dependent
variable (in this case the mortality index) that can be explained by
the independent variable.
Here are the equations for calculating the coefficients for the linear
regression:
Words of warning about Pearson correlation coefficients:
a) Data may be highly correlated but have a zero linear correlation
coefficient.
b) Outlier in the data set can have a large effect on the value of
r and lead to erroneous conclusions. This does not mean that
you should ignore outliers, but you should be aware of their effect on
the correlation coefficients.
Note: An alternatives to the pearson correlation coefficient is the
spearman
rank correlation coefficient (SPEARMAN( ref1, ref2, )) which is a
non-parametric approach and is often used when no linear relationship
can
be found in the data.
Significance of regression
- could it happen that the correlation occurred by chance only? The
correlation
coefficient r is not a good measure of this. If there is no correlation
between two parameters, then the regression line should have the slope
close to 0.
- in
the framework of the confidence interval we could determine the 95%
confidence interval of the slope and determine if 0 is in it or not
- alternatively, we can determine whether a significant simple
relationship
between X and
Y exists by testing whether the slope could be equal to zero using the
hypothesis framework:
- Ho: The coefficient (slope) of the predictor
variables = 0
- Ha: The coefficient is different from 0
- If this hypothesis
is rejected, we can conclude that there is evidence of a simple linear
relationship. We can use the t-statistic to determine whether the slope
is significant or not
- t - statistic = slope / SE of the slope
- degrees of freedom: n-2
- Excel's REGESSION function gives us the answers in both frameworks
Assumption of the linear regression model
Regression analysis makes the following assumptions:
- there must be a linear relationship between the variables
- the residuals are normally distributed with a mean of 0
- the residuals have constant variance (square of standard devation)
- the rsiduals are independent from each other
A significant regression is no proof that these assumptions haven't
been violated! We need to check them by plotting the residuals.
Residual analysis is a good way to decide if a linear fit
was
a good choice. In a residual analysis, the differences for each data
point
between the true y-value and the predicted y-value as determined from
the
best-fit line are plotted for each x-value of the data points. If the
linear
fit was a good choice, then the scatter above and below the zero line
should
be about the same. If this analysis shows a bias (for example the
residuals
appear to be distributed in a specific pattern as x increases, as in
above
figure), another curve might be a better choice.
Example: Breast cancer as function
of mean annual temperature (Breast Cancer.xlsx, Ch 8)
Correlation matrix and scatter
plot matrix
Statplus provides tools to
look at correlations of many parameters in a
convenient way.
- the correlation matrix gives Pearson correlation coefficients r
and their P-values
- the scatter plot matrix combines scatter plots
of all varaibles plotted against each other in a compact way
Multiple regression
We can extend our regression
analysis to multiple parameters, we would
fit a line with the following equation to our data:
y = bo + b1x1 + b2x2
+ b3x3 +....
Again,
we
would use Excel's regression
function to determine the coefficients
The
ANOVA table helps to choose between two hypotheses:
- Ho: The coefficients of all predictor variables = 0
- Ha: At least one of the coefficients is different from
0
The task is then to find out
which
variables are good predicters, which can be done in the following way:
- eliminate the least
significant
predictor if it is not significant
- refit the model
- repeat the above steps
until
all predictors are significant
Example: daily max values for the ozone data set
Resources:
- See chaper 8 in Berk &
Carey
- Freedman, D., Pisani, R.,
Purves,
R., and Adhikari, A. (1991) Statistics. WW Norton & Company, New
York,
2nd ed. 514pp.
- Fisher, F.E. (1973)
Fundamental
Statistics
Concepts. Canfield Press, San Francisco, 371 pp.
- Berenson, M.L., Levine, D.M.,
and
Rindskopf, D. (1988) Applied statistics - A first course. Prentice
Hall,
Englewood Cliffs, NJ, 557pp.
- Prothero, W. A. (2000) Introduction
to
Geological
Data Analysis 3-1 -3-13pp.
