Centre for Computer Technology

ICT114 Mathematics for Computing

Week 11

Linear Regression

Objectives
Review

week 10 Curve Fitting Regression Dependent/Independent variable Method of Least Squares Least Squares line

March 20, 2012

Copyright Box Hill Institute

Numerical Differentiation
Newtons

Forward Difference Formula

f/(x)= (1/h) [ 1/2 2 ++1/3 3 1/4 4 +.]

Newtons

Backward Difference Formula f/(x)= (1/h) [ +1/2 2+1/3 3 + 1/4 4+..]

Copyright Box Hill Institute

March 20, 2012

Numerical Integration
Trapezoid

Rule = (h/2) [ f(a0) + 2 f(a1) + 2f(a2) +. + 2 f(an-1) + f(an)]

Simpsons

One Third Rule (the number of intervals have to be even) = (h/3)[ ( f(a0) + f (an) ) + 4 (f(a1) + f (a3) + f(a5)+ f(an-1) ) + 2 (f(a2) + f (a4) + f(a6) + .. f(an-2) ) ]
Copyright Box Hill Institute

March 20, 2012

Centre for Computer Technology

Curve Fitting

Curve Fitting (1)

A

relation between two variables is expressed mathematically by an equation connecting both. For example if x and y are the height and weight of an individual Then a sample of n individuals will have height x1, x2, .., xn weight y1, y2, .., yn
March 20, 2012

Copyright Box Hill Institute

Curve Fitting (2)

Plotting

the points (x1, y1), (x2,y2), .(xn,yn) on a rectangular coordinate system will result in a scatter diagram. The data in the scatter diagram can be generally approximated by a smooth curve. The resulting curve is called the approximating curve.
March 20, 2012

Copyright Box Hill Institute

March 20, 2012

Copyright Box Hill Institute

Curve Fitting (3)

If

the data is approximated by

straight

line there is a linear relationship between the variables else, there is a nonlinear relationship
Finding

equations to approximating curves that fit the given set of data is called curve fitting.
Copyright Box Hill Institute

March 20, 2012

March 20, 2012

Copyright Box Hill Institute

Curve Fitting (4)

The

general equation of a straight is y = a +bx (linear relation)

The

general equation for a parabola or a quadratic equation is y = a + bx + cx2 (nonlinear relation)

March 20, 2012

Copyright Box Hill Institute

March 20, 2012

Copyright Box Hill Institute

Centre for Computer Technology

Regression/ Method of Least Squares

Regression
In

curve fitting we need to determine one of the variables (the dependent variable) from the other (the independent variable) process of estimation is called regression

This

March 20, 2012

Copyright Box Hill Institute

Dependent/Independent Variable
y

= a + bx determine the variables? the corresponding equation is called the regression equation of y on x the corresponding curve is called the regression curve of y on x

March 20, 2012

Copyright Box Hill Institute

Method of Least Squares (1)

It

is a method to find the best fitting curve for the data points (scatter diagram) Let (x1, y1), (x2,y2), .(xn,yn) be the data points. Let C be the best fitting curve For a given value of x, x1 there will be a difference between the corresponding y and the value determined by C.
March 20, 2012

Copyright Box Hill Institute

March 20, 2012

Copyright Box Hill Institute

Method of Least Squares (2)

For

the data points (x1, y1), (x2,y2), . (xn,yn). Let the corresponding deviations be d1, d2,, dn. measure of the curve is provided by D = d12 + d22++dn2

The

If

D is small the fit is good If D is large the fit is bad

March 20, 2012

Copyright Box Hill Institute

Method of Least Squares (3)

The

curve having this property is said to fit the data in least squares sense The curve is called a least squares regression curve or a least squares curve Dependent Var Independent Var Offset
March 20, 2012

y x vertical
Copyright Box Hill Institute

Least Squares Line

For Let

the data points (x1, y1), (x2,y2), . (xn,yn)

y = a+bx is the least squares line a and b are determined by solving the (normal) equations y = an + b x xy = a x + b x2
March 20, 2012

Copyright Box Hill Institute

Least Squares Line (y on x)

( y) (x2) ( x) (xy) a = --------------------------------------------n x2 (x)2 n xy ( x)( y) b = ---------------------------------------------n x2 (x)2
March 20, 2012

Copyright Box Hill Institute

Example
The table below shows the respective heights x and y of a sample of 12 fathers and their oldest sons. Find the least squares regression line of y on x.
Father (x) Son (y) 65 63 67 64 68 62 70 66 68 67 69 71 68 66 68 65 69 66 68 65 71 67 68 70

March 20, 2012

Copyright Box Hill Institute

x 65 63 67 64 68 62 70 66 68 67 69 71 x = 800
March 20, 2012

y 68 66 68 65 69 66 68 65 71 67 68 70 y = 811

x2 4225 3969 4489 4096 4624 3844 4900 4356 4624 4489 4761 5041

xy 4420 4158 4556 4160 4692 4092 4760 4290 4828 4489 4692 4970

y2 4624 4356 4624 4225 4761 4356 4624 4225 5041 4489 4624 4900

x2 = 53418 xy = 54107 y2 = 54849

Copyright Box Hill Institute

the normal equations are 12a + 800b = 811 800a + 53418b = 54107 solving the above equations we get a = 35.82, b = 0.4776 y = 35.82 + 0.4776x is the equation for the regression line
March 20, 2012

Copyright Box Hill Institute

Question
The table below shows the Consider the variation of the bulk modulus of Silicon Carbide as a function of temperature. Find the least squares regression line of y (G) on x (T).

March 20, 2012

Copyright Box Hill Institute

Summary
Curve Fitting
A

relation between two variables is expressed mathematically by an equation connecting both. Finding equations approximating curves that fit the given set of data is called curve fitting.
March 20, 2012

Copyright Box Hill Institute

Summary
Method of Least Squares
For

the data points (x1, y1), (x2,y2), . (xn,yn) If y = a+bx is the least squares line a and b are determined by solving the (normal) equations y = an + b x xy = a x + b x2
March 20, 2012

Copyright Box Hill Institute

Summary
Method of Least Squares
( y) (x2) ( x) (xy) a = --------------------------------------------n x2 (x)2 n xy ( x)( y) b = ---------------------------------------------n x2 (x)2
March 20, 2012

Copyright Box Hill Institute

References
H

L Verma and C W Gross : Introduction to Quantitative Methods,John Wiley JB Scarborough : Numerical Mathematical Analysis, Jon Hopkins Hall, New Jersey Gerald W. Recktenwald, Numerical Methods with MATLAB, Implementation and Application, Prentice Hall Murray Spiegel, John Schiller, Alu Srinivasan, Probability and Statistics, Schaums easy Outlines http://mathworld.wolfram.com
March 20, 2012

Copyright Box Hill Institute