Module 1: Introduction to
Numerical Methods
Prepared by:
Overview
This module is designed to guide you to numerical methods and provide motivation by demonstrating
how these techniques can be used in the engineering modeling process. The computer-related
aspects of numerical methods are introduced and the skills that you should acquire in order to
implement numerical techniques using computers. Also, approximation and error analysis are
discussed to understand the effective use of numerical methods.
Learning Outcomes
At the end of this module, you should be able to:
1. Explain reasons to study numerical methods.
2. Apply measurement concepts to describe and quantify errors.
3. Use Taylor series in approximating various functions.
1.1 INTRODUCTION
Numerical methods are techniques by which mathematical problems are formulated so that they can
be solved with arithmetic operations. Although there are many kinds of numerical methods, they have
one common characteristic: they invariably involve large numbers of tedious arithmetic calculations.
It is little wonder that with the development of fast, efficient digital computers, the role of numerical
methods in engineering problem solving has increased dramatically in recent years. Numerical
methods are sometimes referred to as computer mathematics because digital computers excel at
performing such operations.
Mathematical Procedures
In this course, the numerical methods for different mathematical procedures will be discussed such
as:
1. Roots of Equations
These problems are concerned with the value of a variable or a parameter that satisfies a single
nonlinear equation. These problems are especially valuable in engineering design contexts where
it is often impossible to explicitly solve design equations for parameters.
2. Systems of Linear Algebraic Equations
These problems are similar in spirit to roots of equations in the sense that they are concerned
with values that satisfy equations. However, in contrast to satisfying a single equation, a set of
values is sought that simultaneously satisfies a set of linear algebraic equations. Such equations
arise in a variety of problem contexts and in all disciplines of engineering.
3. Curve Fitting
You will often have occasion to fit curves to data points. The techniques developed for this
purpose can be divided into two general categories: regression and interpolation. Regression is
employed where there is a significant degree of error associated with the data. Experimental
results are often of this kind. For these situations, the strategy is to derive a single curve that
represents the general trend of the data without necessarily matching any individual points. In
contrast, interpolation is used where the objective is to determine intermediate values between
relatively error-free data points. Such is usually the case for tabulated information. For these
situations, the strategy is to fit a curve directly through the data points and use the curve to predict
the intermediate values.
4. Integration and Differentiation
As depicted, a physical interpretation of numerical integration is the determination of the area
under a curve. Numerical integration formulas play an important role in the solution of differential
equations. Numerical differentiation, as you know from your study of calculus, involves the
determination of a function’s slope or its rate of change.
5. Ordinary Differential Equations
Ordinary differential equations are of great significance in engineering practice. This is because
many physical laws are couched in terms of the rate of change of a quantity rather than the
magnitude of the quantity itself. In this course, only initial-value problems are addressed.
Computer Programs
Computer programs are set of instructions that direct the computer to perform a certain task. To be
able to perform engineering-oriented numerical calculations, you should be familiar with the following
programming topics:
• Simple information representation (constants, variables, and type declaration)
• Advanced information representation (data structure, arrays, and records)
• Mathematical formulas (assignment, priority rules, and intrinsic functions)
• Input/Output
• Logical representation (sequence, selection, and repetition)
• Modular programming (functions and subroutines)
Assuming that you have some prior exposure to programming, we will focus our discussion on
the last two topics.
Modular Programming
The approach where computer programs can be divided into subprograms, or modules, that can be
developed and tested separately, is called modular programming.
The most important attribute of modules is that they be as independent and self-contained as
possible. In addition, they are typically designed to perform a specific, well-defined function and have
one entry and one exit point. As such, they are usually short (generally 50 to 100 instructions in
length) and highly focused.
The advantages of modular programming are:
• It is easier to understand the underlying logic of smaller modules.
• They are easier to debug and test.
• Facilitate program maintenance and modification.
• Allows you to maintain your own library of modules for later use.
Excel
Excel is the spreadsheet developed by Microsoft, Inc. Spreadsheet are a special type of software that
allow the user to enter and perform calculations on rows and columns of data. When any value on
the sheet is changed, entire calculation is updated, therefore, spreadsheets are ideal for “what if?”
sorts of analysis.
Excel has some built in numerical capabilities including equation solving, curve fitting and
optimization. It also includes VBA as a macro language that can be used to implement numerical
calculations. Finally, has several visualization tools, such as graphs and three-dimensional plots, that
serve as valuable adjuncts for numerical analysis.
MATLAB
MATLAB is the flagship software product of The MathWorks, Inc., which was cofounded by the
numerical analysts Cleve Moler and John N. Little. As the name implies, MATLAB was originally
developed as a matrix laboratory. To this day, the major element of MATLAB is still the matrix. A
variety of numerical functions, symbolic computations, and visualization tools have been added to the
matrix manipulations.
MATLAB has a variety of functions and operators that allow convenient implementation of many
of the numerical methods. Programs can be written as so-called M-files that can be used to implement
numerical calculations. The use of MATLAB is closely related to programming.
True Error
True error denoted by 𝐸𝑡 is the difference between the true value in a calculation and the approximate
value found using a numerical method etc.
True Error (𝐸𝑡 ) = True Value − Approximate Value (1.1)
Example 1.1
The derivative, 𝑓′(𝑥) of a function 𝑓(𝑥) at a particular value of 𝑥 can be approximated by the equation,
𝑓(𝑥 + ℎ) − 𝑓(𝑥)
𝑓 ′ (𝑥) ≈ .
ℎ
Find the true error and the relative true error for 𝑓(𝑥) = 7𝑒 0.5𝑥 at 𝑓′(2) with ℎ = 0.3.
Solution
Before solving for the true error, the approximate value and the true value of the given function must
be calculated first. The approximate value of the function at 𝑓 ′ (2) is
𝑓(2 + 0.3) − 𝑓(2) 𝑓(2.3) − 𝑓(2)
𝑓 ′ (2) ≈ =
0.3 0.3
7𝑒 0.5(2.3) − 7𝑒 0.5(2) 22.107 − 19.028
= = = 10.263
0.3 0.3
The exact value of 𝑓′(2) can be calculated using our knowledge of differential calculus.
𝑓(𝑥) = 7𝑒 0.5𝑥
𝑓 ′ (𝑥) = 7 × 0.5 × 𝑒 0.5𝑥 = 3.5𝑒 0.5𝑥
Therefore, the true value of 𝑓′(2) is
𝑓 ′ (2) = 3.5𝑒 0.5(2) = 9.514
The true error is calculated as
𝐸𝑡 = True Value − Approximate Value = 9.514 − 10.263 = −0.749
The relative true error is
True Error −0.749
𝜖𝑡 = = = −0.07873
True Value 9.514
In percentage, the relative true error is
𝜖𝑡 = −0.07873 × 100% = −7.873%
Absolute relative true errors may also need to be calculated,
|𝜖𝑡 | = |−0.07873| = 0.07873 or 7.873%
Example 1.2
The derivative, 𝑓′(𝑥) of a function 𝑓(𝑥) at a particular value of 𝑥 can be approximated by the equation,
𝑓(𝑥 + ℎ) − 𝑓(𝑥)
𝑓 ′ (𝑥) ≈ .
ℎ
For 𝑓(𝑥) = 7𝑒 0.5𝑥 at 𝑥 = 2, find the following,
a. 𝑓′(2) using ℎ = 0.3
b. 𝑓′(2) using ℎ = 0.15
c. the approximate error for the value of 𝑓′(2) for part b.
d. the relative approximate error using values from ℎ = 0.3 and ℎ = 0.15.
Solution
a. For 𝑥 = 2 and ℎ = 0.3
𝑓(2 + 0.3) − 𝑓(2) 𝑓(2.3) − 𝑓(2)
𝑓 ′ (2) ≈ =
0.3 0.3
7𝑒 0.5(2.3) − 7𝑒 0.5(2) 22.107 − 19.028
= = = 10.263
0.3 0.3
b. For 𝑥 = 2 and ℎ = 0.15
𝑓(2 + 0.15) − 𝑓(2) 𝑓(2.15) − 𝑓(2)
𝑓 ′ (2) ≈ =
0.15 0.15
7𝑒 0.5(2.15) − 7𝑒 0.5(2) 20.51 − 19.028
= = = 9.880
0.15 0.15
c. So, the approximate error, 𝐸𝑎 , is
𝐸𝑎 = Present Approximation − Previous Approximation
𝐸𝑎 = 9.880 − 10.263 = −0.383
d. The relative approximate error is
Approximate Error
𝜖𝑎 =
Present Approximation
−0.383
𝜖𝑎 = = −0.038765
9.880
as a percentage,
𝜖𝑎 = −0.038765 × 100% = −3.8765%
Absolute relative approximate errors may also need to be calculated,
|𝜖𝑎 | = |−0.038765| = 0.038765 or 3.8765%
Note: In the combination of a number and a unit of measurement, we can easily determine the
significant digit by supplying the suitable unit prefix.
Example 1.3
Determine the number of significant digits in following numbers:
a. 2.745
b. 20.003
c. 0.0021
d. 0.002100
e. 20000
f. 0.20100̅00
g. 1.05 × 105
h. 2.0300 × 107
Solution
a. 2.745 has four significant digits: 2, 7, 4, 5
b. 20.003 has five significant digits: 2, 0, 0, 0, 3
c. 0.0021 has two significant digits: 2, 1
d. 0.002100 has four significant digits: 2, 1, 0, 0
e. 20000 has three significant digits: 2, 0, 0
f. 0.20100̅00 has five significant digits: 2, 0, 1, 0, 0
g. 1.05 × 105 has three significant digits: 1, 0, 5
h. 2.0300 × 107 has five significant digits: 2, 0, 3, 0, 0
Round-off Error
Round-off error occurs because digital computers cannot represent some quantities exactly. Its value
can be calculated by deviating the rounded digits and the exact value.
There are two major components of round of error involved in numerical calculations: (1) the
digital computers have size and precision limits in their ability to represent the numbers, and (2)
certain numerical manipulations are highly sensitive to round off errors.
Example 1.4
Find the round-off error of the following if each number is represented using 5 significant digits.
a. 𝑒
b. 2.003458764
c. 2/3
Solution:
a. Since we are just interested in 5 significant digits, 𝑒 = 2.7183.
Round-off Error = 𝑒 − 2.7183 = −0.000018171
b. Round-off Error = 2.003458764 − 2.0035 = −0.000041236
2
c. Round-off Error = − 0.66667 = −0.0000033
3
Truncation Error
Truncation error is caused by truncating a mathematical procedure; that is, replacing an infinite
process by a finite one. Its value can also be calculated by getting the deviation between the exact
value and its approximate value.
Example 1.5
Find 𝑓 ′ (3) for 𝑓(𝑥) = 𝑥 2 using the formula below and Δ𝑥 = 0.2.
𝑓(𝑥 + Δ𝑥) − 𝑓(𝑥)
𝑓 ′ (𝑥) ≈
Δ𝑥
Solution
Solving for 𝑓 ′ (3) using the formula above, we have
𝑓(3 + 0.2) − 𝑓(3) 𝑓(3.2) − 𝑓(3) 3.22 − 32 10.24 − 9 1.24
𝑓 ′ (3) ≈ = = = = = 6.2
0.2 0.2 0.2 0.2 0.2
The actual value is
𝑓 ′ (𝑥) = 2𝑥; 𝑓 ′ (3) = 2(3) = 6
The truncation error is then
Truncation Error = 6 − 6.2 = −0.2
90
y = x2
60
30
0 x
0 3 6 9 12
Figure 1.2 Plot of 𝑦 = 𝑥 2 showing the approximate area under the curve from 𝑥 = 3 to
𝑥 = 9 using two rectangles
Solution
Solving for the exact value of the area under the curve 𝑓(𝑥) = 𝑥 2 over the interval [3, 9],
9 9
𝑥3 93 − 33
∫ 𝑥 2 𝑑𝑥 =[ ] =[ ] = 234
3 3 3
3
The approximate value of the integral using two rectangles is
9
∫ 𝑥 2 𝑑𝑥 = (𝑥 2 )|𝑥=3 (6 − 3) + (𝑥 2 )|𝑥=6 (9 − 6)
3
= (32 )3 + (62 )3 = 27 + 108 = 135
The truncation error is then
Truncation Error = 234 − 135 = 99
Solution
𝑥2 𝑥3 𝑥4
𝑒𝑥≈ 1+𝑥+ + +
2! 3! 4!
0.252 0.253 0.254
≈ 1 + 0.25 + + + = 1.2840
2! 3! 4!
The exact value of 𝑒 0.25 up to 5 significant digits is also 1.2840.
But the above discussion and example do not answer our question of what a Taylor series is.
Here it is, for a function 𝑓(𝑥)
𝑓 ′′ (𝑥) 2 𝑓′′′(𝑥) 3
𝑓(𝑥 + ℎ) = 𝑓(𝑥) + 𝑓 ′ (𝑥)ℎ + ℎ + ℎ +⋯ (1.4)
2! 3!
provided all derivatives of 𝑓(𝑥) exist and are continuous between 𝑥 and 𝑥 + ℎ.
What does this mean in plain English? As Archimedes would have said, “Give me the value of
the function at a single point, and the value of all (first, second, and so on) its derivatives, and I can
give you the value of the function at any other point”.
It is very important to note that the Taylor series is not asking for the expression of the function
and its derivatives, just the value of the function and its derivatives at a single point.
Yes, all the derivatives have to exist and be continuous between 𝑥 (the point where you are) to
the point, 𝑥 + ℎ where you are wanting to calculate the function at. However, if you want to calculate
the function approximately by using the nth order Taylor polynomial, then 1st, 2nd, …, nth derivatives
need to exist and be continuous in the closed interval [𝑥, 𝑥 + ℎ], while the (𝑛 + 1)𝑡ℎ derivative needs
to exist and be continuous in the open interval (𝑥, 𝑥 + ℎ).
Example 1.8
Derive the Maclaurin series of
𝑥3 𝑥5 𝑥7
sin 𝑥 = 𝑥 − + − +⋯
3! 5! 7!
Solution
Maclaurin series is simply a Taylor series for the point 𝑥 = 0.
𝑓(𝑥) = sin 𝑥 , 𝑓(0) = 0 𝑓 ′′′ (𝑥) = − cos 𝑥 , 𝑓 ′′′ (0) = −1
𝑓 ′ (𝑥) = cos 𝑥 , 𝑓 ′ (0) = 1 𝑓 ′′′′ (𝑥) = sin 𝑥 , 𝑓 ′′′′ (0) = 0
𝑓 ′′ (𝑥) = − sin 𝑥 , 𝑓 ′′ (0) = 0 𝑓 ′′′′′ (𝑥) = cos 𝑥 , 𝑓 ′′′′′ (0) = 1
Using the Taylor series now,
ℎ2 ℎ3 ℎ4 ℎ5
𝑓(𝑥 + ℎ) = 𝑓(𝑥) + 𝑓 ′ (𝑥)ℎ + 𝑓 ′′ (𝑥) + 𝑓 ′′′ (𝑥) + 𝑓 ′′′′ (𝑥) + 𝑓 ′′′′′ (𝑥) + ⋯
2! 3! 4! 5!
2 3 4 5
ℎ ℎ ℎ ℎ
𝑓(0 + ℎ) = 𝑓(0) + 𝑓 ′ (0)ℎ + 𝑓 ′′ (0) + 𝑓 ′′′ (0) + 𝑓 ′′′′ (0) + 𝑓 ′′′′′ (0) + ⋯
2! 3! 4! 5!
2 3 4 5
ℎ ℎ ℎ ℎ
𝑓(ℎ) = 𝑓(0) + 𝑓 ′ (0)ℎ + 𝑓 ′′ (0) + 𝑓 ′′′ (0) + 𝑓 ′′′′ (0) + 𝑓 ′′′′′ (0) + ⋯
2! 3! 4! 5!
2 3 4 5 3 5
ℎ ℎ ℎ ℎ ℎ ℎ
= 0 + 1(ℎ) − 0 − 1 + 0 + 1 + ⋯ = ℎ − + + ⋯
2! 3! 4! 5! 3! 5!
Example 1.9
Find the value of 𝑓(6) given that 𝑓(4) = 125, 𝑓 ′ (4) = 74, 𝑓 ′′ (4) = 30, 𝑓 ′′′ (4) = 6 and all other higher
derivatives of 𝑓(𝑥) at 𝑥 = 4 are zero.
Solution
ℎ2 ℎ3
𝑓(𝑥 + ℎ) = 𝑓(𝑥) + 𝑓 ′ (𝑥)ℎ + 𝑓 ′′ (𝑥) + 𝑓 ′′′ (𝑥) + ⋯
2! 3!
𝑥=4
ℎ =6−4 =2
Since fourth and higher derivatives of 𝑓(𝑥) are zero at 𝑥 = 4.
22 23
𝑓(4 + 2) = 𝑓(4) + 𝑓 ′ (4)(2) + 𝑓 ′′ (4) + 𝑓 ′′′ (4)
2! 3!
22 23
𝑓(6) = 125 + 74(2) + 30 ( ) + 6 ( )
2! 3!
𝑓(6) = 125 + 148 + 60 + 8
𝑓(6) = 341
Example 1.10
The Taylor series for 𝑒 𝑥 at point 𝑥 = 0 is given by
𝑥2 𝑥3 𝑥4 𝑥5
𝑒𝑥 = 1 + 𝑥 + + + + +⋯
2! 3! 4! 5!
What is the truncation (true) error in the representation of 𝑒 1 if only four terms of the series are used?
Solution
If only four terms of the series are used, then
𝑥2 𝑥3
𝑒𝑥 ≈ 1 + 𝑥 + +
2! 3!
1 2 13
𝑒𝑥 ≈ 1 + 1 + +
2! 3!
𝑒 𝑥 = 2.66667
The truncation (true) error would be the unused terms of the Taylor series, which then are
𝑥4 𝑥5
𝐸𝑡 = + +⋯
4! 5!
14 15
= + +⋯
4! 5!
𝐸𝑡 ≅ 0.0516152
REFERENCES
1. Cabero, J. B., Acula, D. D. (2013). Numerical Methods. Anvil Publishing, Inc.
2. Chapra, S. C., Canale, R. P. (2015). Numerical Methods for Engineers (7th ed.). The McGraw-Hill
Companies, Inc.
3. Chapra, S. C. (2018). Applied Numerical Methods with MATLAB® for Engineers and Scientists
(4th ed.). The McGraw-Hill Companies, Inc.
4. Kaw, A. (n.d.). Hollistic Numerical Methods: Transforming Numerical Methods Education for the
STEM Undergraduate. Retrieved from http://mathforcollege.com