Numerical Differentiation

Ng Tin Yau (PhD)

Department of Mechanical Engineering
The Hong Kong Polytechnic University

Jan 2016

By Ng Tin Yau (PhD) 1/18

Introduction
The main problem of computational calculus is to compute derivatives
and integrals. There are two directions that we can take for such
problems, numerical computing and symbolic computing. Both
derivatives and integrals have clear definitions, but the type of answer
wanted by a user often depends on the way in which the functions is
specified.
In practice, ther are two other ways for a function to be known. A
function may be specified as a tabulated list, for example, a collection
of time/temperature pairs measured from an experiment, perhaps at
evenly spaced times. In this case, finding the derivative or
anti-derivative from the rules of freshman calculus is impossible.
Finally, a function may be specified as the output of an experiment or
computer simulation whose input is specified by the user. In the latter
two cases, symbolic computing methods cannot be applied, and
numerical differentiation and integration are required to solve the
problems.
By Ng Tin Yau (PhD) 2/18

Differentiability
Let f be a function defined on a set X of real numbers and x0 X.
Then f is continuous at x0 if
lim f (x) = f (x0 )

xx0

The function f is continuous on the set X if it is continuous at each

number in X. The set of all functions that are continuous on the set X
is denoted C(X).
Let f be a function defined in an open interval containing x0 . The
function f is differentiable at x0 if
f 0 (x0 ) = lim

xx0

f (x) f (x0 )
x x0

(1)

exists. The number f 0 (x0 ) is called the derivative of f at x0 .

By Ng Tin Yau (PhD) 3/18

Taylors Theorem
Suppose f C n [a, b], that f (n+1) exists on [a, b] and x0 [a, b]. For
every x [a, b], there exists a number (x) between x0 and x with
f (x) = Pn (x) + Rn (x), where
Pn (x) =

n
X
f (k) (x0 )
k=0

and
Rn (x) =

k!

(x x0 )k

f (n+1) ((x))
(x x0 )n+1
(n + 1)!

(2)

(3)

Here Pn (x) is called the n-th Taylor polynomial for f about x0 , and
Rn (x) is called the remainder term associated with Pn (x).
The infinite series obtained by taking the limit of Pn (x) as n is
called the Taylor series for f about x0 . In the case x0 = 0, the Taylor
polynomial is often called a Maclaurin polynomial, and the Taylor
series is often called a Maclaurin series.
By Ng Tin Yau (PhD) 4/18

Example 1

Let f (x) = cos x and x0 = 0. Determine (a) the second Taylor

polynomial for f about x0 ; and (b) the third Taylor polynomial for f
about x0 .
Solution: Since f C (R), Taylors theorem can be applied for any
n 0. Also,
f 0 (x) = sin x

f 00 (x) = cos x

f (3) (x) = sin x

f (4) (x) = cos x

so
f (0) = 1 f 0 (0) = 0 f 00 (0) = 1

f (3) (0) = 0

By Ng Tin Yau (PhD) 5/18

Example 1a
For n = 2 and x0 = 0, we have
x2 00
x3
f (0) + f (3) ((x))
2!
3!
1 2 1 3
= 1 x + x sin (x)
2
6

cos x = f (0) + f 0 (0)x +

where (x) is some (generally unknown) number between 0 and x.

When x = 0.01, this becomes
cos 0.01 = 0.99995 +

106
sin (0.01)
6

The approximation to cos 0.01 given by the Taylor polynomial is

therefore 0.99995. The error bound is given by
| cos 0.01 0.99995| = 0.1
6 106 | sin (0.01)| 0.16 106
By Ng Tin Yau (PhD) 6/18

Example 1b
For n = 3 and x0 = 0, we have
1
1
cos x = 1 x2 + x4 cos (x)
2
24
where 0 < (x) < 0.01. When x = 0.01, the error term


1

 (0.01)4 cos (0.01) 1 (0.01)4 4.2 1010
 24
 24
The approximation to cos 0.01 given by the Taylor polynomial is still
0.99995. However, the error bound is given by
| cos 0.01 0.99995| 4.2 1010
Remarks: This example illustrates the two objectives of numerical
analysis:
1. Find an approximation to the solution of a given problem.
2. Determine a bound for the accuracy of the approximation.
By Ng Tin Yau (PhD) 7/18

An Alternative Form
If we set x 7 x0 + h in the Taylors series, then
f (x0 + h) =

f (0) (x0 ) 0 f (1) (x0 ) 1

f (n+1) ((x)) n+1
h +
h + +
h
0!
1!
(n + 1)!

where x0 < < x0 + h. Clean it up a bit to yield

f (x0 + h) = f (x0 ) + f 0 (x0 )h +

f (n+1) ((x)) n+1

f 00 (x0 ) 2
h + +
h
2
(n + 1)!

In view of this, the definition of the derivative of a function f (x) at

x = x0 can be written as:
f (x0 + h) f (x)
h0
h

f 0 (x0 ) = lim

In fact, this is the definition of the one-dimensional directional

derivative.
By Ng Tin Yau (PhD) 8/18

The first-order derivative

Suppose that f is sufficiently smooth, then we have the Taylors
formulas:
h2 00
f (x0 ) +
2!
h2
f (x0 h) = f (x0 ) hf 0 (x0 ) + f 00 (x0 )
2!

f (x0 + h) = f (x0 ) + hf 0 (x0 ) +

h3 (3)
f (x0 ) +
3!
h3 (3)
f (x0 ) +
3!

(4)
(5)

In the first order estimation and for h > 0, we have

f 0 (x0 ) =

f (x0 + h) f (x0 ) h 00
f ()
h
2

(6)

For small values of h, the difference quotient [f (x0 + h) f (x0 )]/h can
be used to approximate f 0 (x0 ) with an error bounded by M |h|/2 where
M is a bound on |f 00 (x)| for x (x0 , x0 + h). The formula is known as
the forward-difference formula and in the case where h < 0, it is
called the backward-difference formula.
By Ng Tin Yau (PhD) 9/18

Example 2
Use the forward-difference formula to approximate the derivative of
f (x) = ln x at x0 = 1.8 using h = 0.1 and determine bounds for the
approximation errors.
Solution: For h = 0.1, the forward-difference formula gives
f 0 (1.8)

0.64185389 0.58778667
ln 1.9 ln 1.8
=
= 0.5406722
0.1
0.1

Because f 00 (x) = 1/x2 and 1.8 < < 1.9, a bound for this
approximation error is
|h|
|hf 00 ()|
0.1
= 2 <
= 0.0154321
2
2
2(1.8)2
In
On the other hand, f 0 (x) = 1/x and the exact value of f 0 (1.8) is 0.5.
this case the error bounds are quite close to the true approximation
error.
By Ng Tin Yau (PhD) 10/18

3-point Endpoint Formula

To obtain the first 3-point formula, let us consider the following
Taylors series:
4h2 00
4h3 (3)
f (x0 ) +
f (x0 ) +
2!
3!
4h2 00
8h3 (3)
f (x0 + 2h) = f (x0 ) + 2hf 0 (x0 ) +
f (x0 ) +
f (x0 ) +
2!
3!

4f (x0 + h) = 4f (x0 ) + 4hf 0 (x0 ) +

Subtract the second equation from the first equation yields

4f (x0 + h) f (x0 + 2h) = 3f (x0 ) + 2hf 0 (x0 )

4h3 (3)
f ()
3!

Rearranging terms gives the 3-point endpoint formula:

f 0 (x0 ) =

h2
1
[3f (x0 ) + 4f (x0 + h) f (x0 + 2h)] + f (3) ()
2h
3

(7)

where lies between x0 and x0 + 2h.

By Ng Tin Yau (PhD) 11/18

3-point Midpoint Formula

If we consider f (x0 + h) f (x0 h), then from the Taylor series gives
f 0 (x0 ) =

1
h2
[f (x0 + h) f (x0 h)] f (3) ()
2h
6

(8)

where lies between x0 h and x0 + h. This is the so-called 3-point

midpoint formula.

Discussions
Although the errors in both 3-point formulas are O(h2 ), the error in
the midpoint formula is approximately half the error in the endpoint
formula. However, the approximation in the endpoint formula is useful
near the ends of an interval, because information about f outside the
interval may not be available.

By Ng Tin Yau (PhD) 12/18

5-point Formulas
It is possible to obtain 5-point formulas that involve evaluating the
function at two additional points. The error term for these formulas is
O(h4 ). Common 5-point formulas include the 5-point midpoint
formula:
h4
1
[f (x0 2h)8f (x0 h)+8f (x0 +h)f (x0 +2h)]+ f (5) ()
2h
30
(9)
and the 5-point endpoint formula:
f 0 (x0 ) =

f 0 (x0 ) =

1
[25f (x0 ) + 48f (x0 + h) 36f (x0 + 2h)
2h
h4
+16f (x0 + 3h) 3f (x0 + 4h)] + f (5) ()
5

(10)

By Ng Tin Yau (PhD) 13/18

Second derivative Midpoint Formula

To obtain an approximation of the second derivative of a function f ,

we consider f (x0 h) + f (x0 + h), that is
f 00 (x0 ) =

h2 (4)
1
[f
(x

h)

2f
(x
)
+
f
(x
+
h)]

f ()
0
0
0
h2
12

(11)

where (x0 h, x0 + h). This is the second derivative midpoint

formula. If f (4) (x) is continuous on [x0 h, x0 + h] it is also bounded,
and the approximation is O(h2 ).

By Ng Tin Yau (PhD) 14/18

Example 3
Given f (x) = xex and using h = 0.1, estimate f 0 (2.0) and f 00 (2.0) using
the first and second derivative midpoint formulas, respectively.
Solution: Using the function f , we generate the following data:
x
f (x)

1.9
12.703199

2.0
14.778112

2.1
17.148957

In this case, f 0 (x) = (x + 1)ex and hence, f 0 (2.0) = 22.167168. On the

other hand,
f 0 (2.0) =

f (2.1) f (1.9)
= 5(17.148957 12.7703199) = 22.228790
2(0.1)

For the second-order derivative, f 00 (x) = (x + 2)ex and hence,

f 0 (2.0) = 29.556224. The midpoint formula gives
f 00 (2.0) =

f (1.9) 2f (2.0) + f (2.1)

= 29.593200
0.12

By Ng Tin Yau (PhD) 15/18

Example 4
Use
0.1 the third Taylor polynomial and its reminder term to approximate
0 cos xdx.
Solution: Recall that
1
1
cos x = 1 x2 + x4 cos (x)
2
24
Now

0.1

Therefore,
0.1

cos xdx
0


0.1
1
1
1 x2 dx +
x4 cos (x)dx
2
24 0

0.1 

cos xdx =

0.1 


1 2
1
1 x dx = 0.1 (0.1)3 = 0.09983
2
6

By Ng Tin Yau (PhD) 16/18

Example 4 cont
A bound for the error in this approximation is determined from the
integral of the Taylor remainder term and the fact that | cos (x)| 1
for all x:


0.1

1
1  0.1 4

x cos (x)dx
x4 | cos (x)|dx
24  0
24 0
0.1
1
0.15

x4 dx =
= 8.
3 108
24 0
120
The true value of this integral is
0.1
cos xdx = sin 0.1 0.099833416647
0

so the actual error for this approximation is 8.3314 108 , which is

within the error bound.
By Ng Tin Yau (PhD) 17/18

Exercises
(1) Find the second Taylor polynomial P2 (x) for the function
f (x) = ex cos x about x0 = 0.
a. Use P2 (0.5) to approximate f (0.5). Find an upper bound for error
|f (0.5) P2 (0.5)| using the error formula, and compare it to the
actual error.
b. Find a bound for the error |f (x) P2 (x)| in using P2 (x) to
approximate f (x) on the interval [0, 1].
(2) Let f (x) = 3xex cos x. Using the following data
x
f (x)

1.20
11.59006

1.29
13.78176

1.30
14.04276

1.31
14.30741

1.40
16.86187

and the second derivative midpoint formula to approximate f 00 (1.3)

with h = 0.1 and with h = 0.01. Compare your results to f 00 (1.3).
By Ng Tin Yau (PhD) 18/18