Reading 9 Correlation and Regression

FinQuiz Notes 2 0 1 8
2. CORRELATION ANALYSIS

Scatter plot and correlation analysis are used to examine
how two sets of data are related.
2.1 Scatter Plots
A scatter plot graphically shows the relationship
between two varaibles. If the points on the scatter plot
cluster together in a straight line, the two variables have
a strong linear relation. Observations in the scatter plot
are represented by a point, and the points are not
portfolio could be diversified or decreased.
If there is zero covariance between two assets, it
means that there is no relationship between the
rates of return of two assets and the assets can be
included in the same portfolio.
Correlation coefficient measures the direction and
strength of linear association between two variables. The
correlation coefficient between two assets X and Y can
be calculated using the following formula:

 =    


   

 
connected.
  

 



2.2 & Correlation Analysis & Calculating and
2.3 Interpreting the Correlation Coefficient
=
(
)(  )
The sample covariance is calculated as:
   
(, )
or


 = =

1
()()
where,
n = sample size NOTE:

 = mean of the variable X observations

Xi = ith observation on variable X
Unlike Covariance, Correlation has no unit of
measurement; it is a simple number.
 = mean of the variable Y observations
Yi = ith observation on variable Y

Example:
The covariance of a random variable with itself is  = 47.78  = 40  = 250

Covariance can range from  to + .

simply a variance of the random variable.
=
47.78
40250
= 0.478
The covariance number doesnt tell the investor if
the relationship between two variables (e.g.
returns of two assets X and ) is strong or weak. It
The correlation coefficient can range from -1 to
only tells the direction of this relationship. For
+1.
example,
Two variables are perfectly positively correlated
o Positive number of covariance shows that rates
if correlation coefficient is +1.
of return of two assets are moving in the same
Correlation coefficient of -1 indicates a perfect
direction: when the rate of return of asset X is
inverse (negative) linear relationship between
negative, the returns of other asset tend to be
the returns of two assets.
negative as well and vice versa.
When correlation coefficient equals 0, there is
o Negative number of covariance shows that rates
no linear relationship between the returns of
of return of two assets are moving in the opposite
two assets.
directions: when return on asset X is positive, the
The closer the correlation coefficient is to 1, the
returns of the other asset Y tend to be negative
stronger the relationship between the returns of
and vice versa.
two assets.

NOTE: Note: Correlation of +/- 1 does not imply that

slope of the line is +/- 1.
If there is positive covariance between two assets
then the investor should evaluate whether or not NOTE:
he/she should include both of these assets in the
same portfolio, because their returns move in the Combining two assets that have zero correlation with
same direction and the risk in portfolio may not be each other reduces the risk of the portfolio. A negative
diversified. correlation coefficient results in greater risk reduction.
If there is negative covariance between the pair of
assets then the investor should include both of
these assets to the portfolio, because their returns
move in the opposite directions and the risk in

Copyright FinQuiz.com. All rights reserved.

 Reading 9 Correlation and Regression FinQuiz.com

Alternative Hypothesis H1: the correlation in the

population is different from 0 ( 0);

NOTE:
The null hypothesis is the hypothesis to be tested. The
alternative hypothesis is the hypothesis that is accepted
if the null is rejected.

The formula for the t-test is (for normally distributed

variables):


2
=
Difference b/w Covariance & Correlation: The
1  
~(
2)
covariance primarily provides information to the investor
about whether the relationship between asset returns is where,
positive, negative or zero, but correlation coefficient tells
r is the sample coefficient of correlation calculated by
the degree of relationship between assets returns.
=

(
,)
 
NOTE:
Correlation coefficients are valid only if the means,
variances & covariances of X and Y are finite and t = t-statistic (or calculated t)
constant. When these assumptions do not hold, then the n 2 = degrees of freedom
correlation between two different variables depends
largely on the sample selected. Decision Rule:
If test statistic is < t-critical or > + t-critical with n-2
2.4 Limitations of Correlation Analysis degrees of freedom, (if absolute value of t > tc), Reject
H0; otherwise Do not Reject H0.

1. Linearity: Correlation only measures linear Example:

relationships properly.
Suppose r = 0.886 and n = 8, and tC = 2.4469 (at 5%
2. Outliers: Correlation may be an unreliable measure significance level i.e. = 5%/2 and degrees of freedom =
when outliers are present in one or both of the series. 8 2 = 6)
82
t = 0.886 = 4.68 Since t-value > tc, we reject
3. No proof of causation: Based on correlation we 1 (0.886)2
cannot assume x causes y; there could be third
variable causing change in both variables. null hypothsis of no correlation.

4. Spurious Correlations: Spurious correlation is a

correlation in the data without any causal
relationship. This may occur when:

i. two variables have only chance relationships.

ii. two variables that are uncorrelated but may be
correlated if mixed by third variable .
iii. correlation between two variables resulting from a
third variable.

NOTE:
Spurious correlation may suggest investment strategies
that appear profitable but actually would not be so, if Magnitute of r needed to reject the null hypothesis (H0:
implemented. = 0) decreases as sample size n increases. Because
as n increases the:
Testing the Significance of the Correlation o number of degrees of freedom increases
2.6 o absolute value of tc decreases.
Coefficient
o t-value increases
t-test is used to determine if sample correlation
coefficient, r, is statistically significant. In other words, type II error decreases when sample size
(n) increases, all else equal.
Two-Tailed Test:
Null Hypothesis H0 : the correlation in the population is 0
( = 0);
 Reading 9 Correlation and Regression FinQuiz.com

NOTE:
Type I error = reject the null hypothesis although it is true. Practice: Example 7, 8, 9 & 10
Type II error = do not reject the null hypothesis although Volume 1, Reading 9.
it is wrong.

3. LINEAR REGRESSION

Regression analysis is used to: Independent variable: The variable used to explain the
dependent variable. Also called exogenous or
Predict the value of a dependent variable based on predicting variable.
the value of at least one independent variable
Explain the impact of changes in an independent Intercept (b0): The predicted value of the dependent
variable on the dependent variable. variable when the independent variable is set to zero.
b0 = y b1 x
Linear regression assumes a linear relationship between
the dependent and the independent variables. Linear Slope Coefficient or regression coefficient (b1): A
regression is also known as linear least squares since it change in the dependent variable for a unit change in
selects values for the intercept b0 and slope b1 that

, 
the independent variable.
 
minimize the sum of the squared vertical distances
between the observations and the regression line. 

     
or
 
Estimated Regression Model: The sample regression line
provides an estimate of the population regression line.    
Note that population parameter values b0 and b1 are
not observeable; only estimates of b0 and b1 are Error Term: It represents a portion of the dependent
observeable. variable that cannot be explained by the independent
varaiable.

Example:
n =100

x = 36,009 .45; (x x)2

s x2 = = 43,528,688
i

n 1
y = 5,411.41; (x x)( yi y )
cov( X , Y ) = = 1,356,256
i

n 1
y = b0 + b1 x = 6,535 0 .0312 x
cov( X , Y ) 1,356,256
b1 = = = 0.0312
s x2 43,528,688
b0 = y b1 x = 5,411.41 ( 0.0312)(36,009.45) = 6,535

Types of data used in regression analysis:

1) Time-series: It uses many observations from different
time periods for the same company, asset class or
country etc.

2) Cross-sectional: It uses many observations for the

same time periodof different companies, asset classes
or countries etc.

3) Panel data: It is a mix of time-series and cross-sectional

data.

Dependent variable: The variable to be explained (or

predicted) by the independent variable. Also called
endogenous or predicted variable.
 Reading 9 Correlation and Regression FinQuiz.com

3.2 Assumptions of the Linear Regression Model 3.4 The Coefficient of Determination

1. The regression model is linear in its parameters b0 and
b1 i.e. b0 and b1 are raised to power 1 only and
neither b0 nor b1 is multiplied or divided by another
regression parameter e.g. b0 / b1.
The coefficient of determination is the portion of the
total variation in the dependent variable that is
explained by the independent variable. The coefficient
of determination is also called R-squared and is denoted
as R2.

"
"
  ""!
 
& 
When regression model is nonlinear in parameters,
regression results are invalid.
'( )  
' *
"+(
" )  

Even if the dependent variable is nonlinear but

'( )  
'
parameters are linear, linear regression can be used. =

2. Independent variables and residuals are +(

" )  
(&)
'( )  
(')
uncorrelated. =
3. The expected value of the error term is 0.
where,
When assumptiuons 2 & 3 hold, linear regression
produces the correct estimates of b0 and b1. 0 R 2 1

4. The variance of the error term is the same for all
observations. (It is known as Homoskedasticity
assumption).
5. Error values () are statistically independent i.e. the
error for one observation is not correlated with any
other observation.
6. Error values are normally distributed for any given
value of x.
3.3 The Standard Error of Estimate
Standard Error of Estimate (SEE) measures the degree of
variability of the actual y-values relative to the estimated
(predicted) y-values from a regression equation. Smaller
the SEE, better the fit.
In case of a single independent variable, the coefficient
of determination is: R2 = r2
where,
R2 = Coefficient of determination
r = Simple correlation coefficient
Example:
Suppose correlation coefficient between returns of two
assets is + 0.80, then the coefficient of determination will
be 0.64. The interpretation of this number is that
approximately 64 percent of the variability in the returns
of one asset (or dependent variable) can be explained
by the returns of the other asset (or indepepnent


variable). If the returns on two assets are perfectly

     !":  = #

$1
correlated (r = +/- 1), the coefficient of determination will
be equal to 100 %, and this means that if changes in
or returns of one asset are known, then we can exactly

( %) 

predict the returns of the other asset.
 =  = # =#

$1
$1
,
NOTE:
Multiple R is the correlation between the actual values
where, and the predicted values of Y. The coefficient of
SSE = Sum of squares error determination is the square of multiple R.
n = Sample size
k = number of independent variables in the model Total variation is made up of two parts:
SST = SSE + SSR(or RSS)
Example:
n = 100
SSE = 2,252,363
Thus,

SSE 2,252,363
s = = = 151.60 where,
n2 98
y = Average value of the dependent variable

% = Estimated value of y for the given value of x

Regression Residual is the difference between the actual y = Observed values of the dependent variable
values of dependent variable and the predicted value
of the dependent variable made by regression
equation. SST (total sum of squares): Measures total variation
 Reading 9 Correlation and Regression FinQuiz.com

in the dependent variable i.e. the variation of the b 1 t /2 s b1

yi values around their mean y. df = n - 2
SSE (error sum of squares): Measures unexplained
variation in the dependent variable.
SSR / RSS (regression sum of squares): Measures Example:
variation in the dependent variable explained by ^ ^

the independent variable. n = 7 b1 = 9.01, s b^1 = 1.50, b1 = 0

Testing H0: b1 = 0 v/s HA: b1 0

9.01 0
T .S . : tobs = = 6.01 R.R. :| tobs | t.025,5 = 2.571
1.50

95% Confidence Interval for b1:

9.01 2.571(1.50) = 9.01 3.86 = (12.87 to 5.15)

As this interval does not include 0, we can reject H0.

Therefore, we can say with 95% confidence that the
regression slope is different from 0.
Practice: Example 13
Volume 1, Reading 9.
Reject H0 because t-value 6.01 > critical tc 2.571.

NOTE:
3.5 Hypothesis Testing
Higher level of confidence or lower level of significance
results in higher values of critical t i.e. tc. This implies
In order to determine whether there is a linear that:
relationship between x and y or not, significance test (i.e.
t-test) is used instead of just relying on b1 value. t-statistic Confidence intervals will be larger.
is used to test the significance of the individual Probability of rejecting the H0 decreases i.e. type II
coefficients (e.g. slope) in a regression. error increases.
The probability of Type-I error decreases.
Null and Alternative hypotheses
Stronger regression results lead to smaller standard errors
H0: b1 = 0 (no linear relationship)
of an estimated parameter and result in tighter
H1: b1 0 (linear relationship does exist)
confidence interval. As a result probability of rejecting H0
b1 b1
)
increases (or probability of Type-I error increases).
Test statistic = t=
s b1 p-value: The p-value is the smallest level of significance
where, at which the null hypothesis can be rejected.
,1 = Sample regression slope coefficient
Decision Rule: If p < significance level, H0 can be
b1 = Hypothesized slope
  Standard error of the slope
rejected. If p > significance level, H0 cannot be rejected.

df= n2 For example, if the p-value is 0.005 (0.5%) & significance

level is 5%, we can reject the hypothesis that true
Decision Rule: parameter equals 0.
If test statistic is < t-critical or > + t-critical with n-2
degrees of freedom, (if absolute value of t > tc), Reject
H0; otherwise Do not Reject H0. Practice: Example 14, 15 & 16
Volume 1, Reading 9.
Two-Sided Test One-sided Test
H0: b1 = 0 H0: b1 = 0
HA: b1 0 HA+: b1> 0or
HA-: b1< 0 Analysis of Variance in a Regression with One
3.6
Independent Variable
Confidence Interval Estimate of the Slope: Confidence
interval is an interval of values that is expected to Analysis of Variance (ANOVA) is a statistical method
include the true parameter value b1 with a given degree used to divide the total variance in a study into
of freedom. meaningful pieces that correspond to different sources.
In regression analysis, ANOVA is used to determine the
 Reading 9 Correlation and Regression FinQuiz.com

usefulness of one or more independent variables in

explaining the variation in dependent variable.
Practice: Example 17
Volume 1, Reading 9.
ANOVA df SS MSS F
&
&.
& $
 -%


$ .
3.7 Prediction Intervals

 $  1
Regression k

  2 3 4" 5#


1

 ; $
7 :  :
where,
 -

nk

$1 5$#  5$ 67 8 8 <
Error
9 9  75$
1
 %


'
and

s f = s 2f
 -


Total n1

 

s2 = squared SEE
n = number of observations

 = estimated mean of X
Or X = value of independent variable

s2X= variance of independent variable

Source of Sum of Mean Sum of
DoF tc = critical t-value for n k 1 degrees of freedom.
Variability Squares Squares
Regression Example:
1 RSS MSR = RSS/1
(Explained)
Calculate a 95% prediction interval on the predicted
Error value of Y. Assume the standard error of the forecast is
n-2 SSE MSE = SSE/n-2 3.50%, and the forecasted value of X is 8%. And n = 36.
(Unexplained)
Assume: Y = 3% + (0.50)(X)
Total n-1 SST=RSS + SSE
The predicted value for Y is: Y =3% + (0.50)(8%)= 7%
F-Statistic or F-Test evaluates how well a set of
independent variables, as a group, explains the variation The 5% two-tailed critical t-value with 34 degrees of
in the dependent variable. In multiple regression, the F- freedom is 2.03. The prediction interval at the 95%
statistic is used to test whether at least one independent confidence level is:
variable, in a set of independent variables, explains a
significant portion of variation of the dependent 7% +/- (2.03 3.50%) = - 0.105% to 14.105%
variable. The F statistic is calculated as the ratio of the
average regression sum of squares to the average sum This range can be interpreted as, given a forecasted
of the squared errors, value for X of 8%, we can be 95% confident that the
dependent variable Y will be between 0.105% and
0&  


 
14.105%.
0  
! !

df numerator = k = 1 Practice: Example 18

df denominator = n k 1 = n 2 Volume 1, Reading 9.

Decision Rule: Reject H0 if F>F-critical.

Note: F-test is always a one-tailed test.
In a regression with just one independent variable, the F
statistic is simply the square of the t-statistic i.e. F= t2. F-
test is most useful for multiple independent variables
while the t-test is used for one independent variable.
NOTE:
When independent variable in a regression model does
not explain any variation in the dependent variable,
then the predicted value of y is equal to mean of y. Thus,
RSS = 0 and F-statistic is 0.
Sources of uncertianty when using regression model &
estimated parameters:
1. Uncertainty in Error term.
2. Uncertainty in the estimated parameters b0 and b1.
3.8 Limitations of Regression Analysis
Regression relations can change over time. This
problem is known as Parameter Instability.
If public knows about a relation, this results in no
Reading 9 Correlation and Regression FinQuiz.com

relation in the future i.e. relation will break down.

Regression is based on assumptions. When these
assumptions are violated, hypothesis tests and
predictions based on linear regression will be
invalid.

Practice: End of Chapter Practice

Problems for Reading 9 & FinQuiz
Item-set ID# 15579, 15544 & 11437.