Introduction

Mathematical Modeling and

Scientific Problem Solving
 Course Objectives
• To develop the ability to solve scientific problems numerically

• To evaluate numerical solution methods (knowing the

advantages and limitations of numerical methods)

• To design, implement, and test computer programs

• To improve the skills of writing Python 3 codes

 Objective: “Numerical methods”

• Numerical methods give solutions to math

problems written as algebraic statements
that computers can execute

• We will learn to formulate the solutions

and evaluate their applicability and
performance.
 Objective: “Design”
• Writing the solutions as a series of steps a
computer can execute
 flow chart and pseudo-code

Objective: “Implementation”
Converting the pseudo-code solution into a
computer program.
 Objective: “Testing”

• Checking that the computer program actually

solves the equations you mean to solve
• Evaluating the success of the numerical
solution chosen

• Numerical accuracy, stability, and efficiency

 Objective: “Evaluation”

• Critical evaluation of the solution

the program gives for the actual
scientific problem.

• This requires all your scientific and

computer skills
 Objective: “Presentation”

• Communication of the results of a computer

program to the people who need to know
the answer
• Clients
• Bosses
• Regulators
• Contractors
 Scientific Problems
Empirical
• observation and experiment
• certain aspects of empirical studies occur repeatedly
• such general behavior can be expressed as
fundamental laws that essentially embody the
cumulative wisdom of past experience
Theoretical / Numerical
• formulation of fundamental laws
• Algebraic F  ma ;   E
• ODE dv d2x
F m m 2
dt dt
• PDE  2T  2T
 2 q
 x  y
2
 Mathematical Models
• Modeling is the development of a mathematical
representation of a physical/biological/chemical/
economic/etc. system
• Putting our understanding of a system into math

• Problem Solving Tools:

Analytic solutions, statistics, numerical methods,
graphics, etc.
• Numerical methods are one means by which
mathematical models are solved
Mathematical Modeling
The process of solving an engineering or physical problem.

E ngineering or Physical problem s

(D escription)

M athem atical M odeling

A pproxim ation & A ssum ption

Form ulation or G overning C om m on features

E quations operation

A nalytical & N um erical M ethods

S olutions

A pplications
 Bungee Jumper
 You are asked to predict the velocity of a
bungee jumper as a function of time
during the free-fall part of the jump
 Use the information to determine the
length and required strength of the
bungee cord for jumpers of different mass
 The same analysis can be applied to a
falling parachutist or a rain drop
Bungee Jumper / Falling Parachutist
Newton’s Second Law

F = ma = Fdown - Fup
= mg - cdv2
(gravity minus air resistance)

Observations / Experiments
Where does mg come from?
Where does -cdv2 come from?
 Now we have fundamental physical laws, so
we combine those with observations to model
the system
 A lot of what you will do is “canned” but need
to know how to make use of observations

 How have computers changed problem solving

in engineering?
 Allow us to focus more on the correct
description of the problem at hand, rather than
worrying about how to solve it.
Exact (Analytic) Solution
 Newton’s Second Law
dv
m  mg  c d v 2
dt
dv cd 2
 g v
dt m

 Exact Solution
mg  gc d 
v( t )  tanh t 
cd  m 
 Numerical Method
 What if cd = cd (v)  const?
 Solve the ODE numerically!

dv v
 lim
dt t 0  t

v v ( t i  1 )  v ( t i )

t ti1  ti
Assume constant slope
(i.e, constant drag force)
over t
Numerical (Approximate) Solution

 Finite difference (Euler’s) method

dv v v ( t i  1 )  v ( t i )
 
dt t ti1  ti
v( ti1 )  v( ti ) cd
 g  v( t i ) 2

ti1  ti m

 Numerical Solution

 cd 2
v( t i 1 )  v( t i )   g  v( t i ) ( t i 1  t i )
 m 
 Example 1.2 Hand Calculations
A stationary bungee jumper with m = 68.1 kg leaps
from a stationary hot air balloon. Use the Euler’s
method with a time increment of 2 s to compute the
velocity for the first 12 s of free fall. Assume a drag
coefficient of 0.25 kg/m.
 cd 2
v( t i  1 )  v( t i )   g  v( t i ) ( t i  1  t i )
 m 
t0  0; v( t0 )  0

Explicit time-marching scheme

m = 68.1 kg, g = 9.81 m/s2, cd = 0.25 kg/m
 Euler’s Method
 Use a constant time increment t = 2 s
 0.25 
Step 1 t  2 s; v  0  9.81  (0 ) 2  ( 2  0 )  19.6200 m / s
 68.1 
 0.25 
Step 2 t  4 s; v  19.6200  9.81  ( 19.6200 ) 2  ( 4  2 )  36.4317 m / s
 68.1 
 0.25 
Step 3 t  6 s; v  36.4137  9.81  ( 36.4137 ) 2  (6  4 )  46.2983 m / s
 68.1 
 0.25 
Step 4 t  8 s; v  46.2983  9.81  ( 46.2983 ) 2  ( 8  6 )  50.1802 m / s
 68.1 
 0.25 
Step 5 t  10 s; v  50.1802  9.81  ( 50.1802 ) 2  ( 10  8 )  51.3123 m / s
 68.1 
 0.25 
Step 6 t  12 s; v  51.3123  9.81 
 68.1
( 51.3123 ) 2  ( 12  10 )  51.6008 m / s


The solution accuracy depends on time increment

Example: Bungee Jumper
 Olympic 10-m Platform Diving

dv a
Air : m  mg  c da v 
2
mg
dt  Buoyant Force
dv w cda  cdw
Water : m  mg  c dw v v  mg
dt 
 Example: Finite Elements and
Structural Analysis
Simple truss - force balance Complex truss

Instead of limiting our analysis to simple cases, numerical

method allows us to work on realistic cases.
 Numerical Accuracy
Truncation error : Method dependent
• Errors which result from using an approximation
rather than an exact procedure
h2 h3
f ( x i  h )  f ( x i )  hf ( x i )  f ( x i )  f ( x i )  ....
2! 3!
Round-off error : Machine dependent
• Errors which result from not being able to adequately
represent the true value
• Result from using an approximate number to
represent exact number

  3.1416 , e  2.71828
 Taylor Series Expansion
• Construction of finite-difference formula
• Numerical accuracy: discretization error
a x
Base point x = a

 f ( x )  co  c1 ( x  a)  c 2 ( x  a) 2  c 3 ( x  a) 3    co  f ( a)

 f ( x )  c 1  2 c 2 ( x  a )  3 c 3 ( x  a ) 2
  c 1  f ( a )
 f ( x )  2 c  6 c ( x  a )    c 2  f ( a ) / 2!

2 3

 f ( x )  6 c 3    c 3  f ( a ) / 3!
 
 (m)
 f ( x )  ( m! )c m  ( m  1) m( m  1)  2 c m  1 ( x  x o )    c m  f ( m ) ( a ) / m!

 
f ( m ) (a)
 f ( x)  c
m 0
m ( x  a ) m
 
m 0 m!
( x  a) m
 Taylor series expansions
h2 h3
f ( xi 1 )  f ( xi  h)  f ( xi )  hf ( xi )  f ( xi )  f ( xi )  ....
2! 3!
 Taylor Series and Remainder
•Taylor series (base point x = a)

f ( m ) (a)
f ( x)  
m 0 m!
( x  a) m

f ( a ) f ( a ) f ( n) ( a)
 f ( a )  f ( a )( x  a )  ( x  a) 
2
( x  a )  ... 
3
( x  a ) n  Rn
2! 3! n!

•Remainder

f ( )
( n 1 )
n1
Rn  ( x  a)
( n  1)!
 Truncation Error
• Taylor series expansion

h2 h3
1 )  f ((higher-order
f ( x• iExample xi  h)  f terms hf ( xi ) 
( xi ) truncated) f ( xi )  f ( xi )  ....
2! 3!

x x2 x3 x4 x5
e  1 x      ....
2! 3! 4! 5!
3 5 7 9
x x x x
sin x  x      ....
3! 5! 7 ! 9!

(xi = 0, h = x  xi+1 = x)
Power series
Polynomials

The function
becomes
more
nonlinear as
m increases
 Round-off Errors
•Computers can represent numbers to a
finite precision
•Most important for real numbers - integer
math can be exact, but limited
•How do computers represent numbers?
•Binary representation of the integers and
real numbers in computer memory
32 bits (23, 8, 1)

28 = 256


smallest  .
100  00(2) 128
 0 .
14693  10 38



largest  .
111 11(2)127
 0 .
18905  10 38

64 bits (52, 11, 1)

211 = 2048

smallest  .100  00(2)1024


largest  .111 11(2)1023
Order of operation
Addition problem:

0.99  0.0044  0.0042  0.9986 exact result

with 3-digit arithmetic:

(0.99  0.0044)  0.0042  0.994  0.0042  0.998

0.99  (0.0044  0.0042)  0.99  0.0086  0.999

Round-off error
Cancellation error
If b is large, r is close to b
x  bx  1  0
2

br Difference of two numbers very close to

x1  each other  potential for greater error!
2
br
x2 
2
r  b2  4
Rationalize:

x2 
b  r  b  r  b2  r 2
 
4

2
2 b  r  2b  r  2b  r  b  r
Try b = 97 x  97 x  1  0
2

(r = 96.9794)

x2 (3 sig. figs.)

exact: 0.01031
standard: 0.01050
rationalized: 0.01031

Corresponding to “cancellation, critical arithmetic”

Significant Figures
48.9 mph? 48.95 mph?
 Significant Digits
•The places which can be used with confidence
•32-bit machine: 7 significant digits
•64-bit machine: 17 significant digits
•Double precision: reduce round-off error, but
increase CPU time

  3.141592653589793238462643 
2  1.41421356237310 
e  2.71828182845904 
 False Significant Figures
3.25/1.96 = 1.65816326530162...
But in practice only report 1.65 (chopping) or
1.66 (rounding)! Why??
Because we don’t know what is beyond the second
decimal place

 3.259 / 1.960  1.66275510204082...

Chopping 
 3.250 / 1.969  1.65058405281869...
 3.254 / 1.955  1.66445012787724...
Rounding 
 3.245 / 1.964  1.65224032586558...
 Accuracy and precision
• Accuracy - How closely a measured or computed
value agrees with the true value
• Precision - How closely individual measured or
computed values agree with each other

More More
Accurate Precise

• Accuracy is getting all your shots near the target.

• Precision is getting them close together.
 Numerical Errors
The difference between the true value and the
approximation

Approximation = true value + true error

Et = true value  approximation = x*  x

True Error x *  x
Relative Error  
True Value x*
or in percent
x * x
t   100%
x*
 Approximate Error
•But the true value is not known
•If we knew it, we wouldn’t have a problem
•Use approximate error
approximat e error
a  100%
approximat ion
present approx.  previous approx.
 100%
present approximat ion
x new  xold
Relative error   100%
x new
 Number Systems
• Base-10 (Decimal): 0,1,2,3,4,5,6,7,8,9
• Base-8 (Octal): 0,1,2,3,4,5,6,7
• Base-2 (Binary): 0,1 – off/on, close/open,
negative/positive charge
• Other non-decimal systems
• 1 lb = 16 oz, 1 ft = 12 in, ½”, ¼”, …..
5 ,129  5  10 3  1  10 2  2  10 1  9  10 0
base  10 :  1 2 3 4
0 .3125  3  10  1  10  2  10  5  10
101101  1  2 5  0  2 4  1  2 3  1  2 2  0  2 1  1  2 0  45

base  2 :  1 2 3 4 11
0.1011  1  2  0  2  1  2  1  2 
 16
 Decimal
System
(base 10)

Binary
System
(base 2)
 Integer Representation
Signed magnitude method
•Use the first bit of a word to indicate the sign
– 0: negative (off), 1: positive (on)
•Remaining bits are used to store a number

+ 1 0 1 0 0 1 0 1 1 0

               
Sign Number
off / on, close / open, negative / positive
 Integer Representation
• 8-bit word

 2 6
2 5
2 4
2 3
2 2
2 1
2 0

               
Sign Number

 smallest number  0000000base2  0base10


largest number  1111111base2  127 base10

• +/- 0000000 are the same, therefore we may use

“-0” to represent “-128”
• Total numbers = 28 = 256 (-128 127)
 Integer Representation
16-bit word

1  2 14  1  2 13  .... 1  2 1  1  2 0  32 ,767
• Range: -32,768 to 32,767
• Overflow: > 32,767 (cannot represent 43,000
A&M students)
• Underflow: < -32,768 (magnitude too large)
32-bit word
• Range: -2,147,483,648 to 2,147,483,647
• 9 significant digits
• Overflow: world population 6 billion
• Underflow: budget deficit -$100 billion
 Integer Operations
• Integer arithmetic can be exact as long as you
don't get remainders in division
• 7/2 = 3 in integer math

• or overflow the maximum integer

• For a 8-bit computer max = 128 (or -127)
• So 123 + 45 = overflow
• and -74 * 2 = underflow
 Floating-Point Representation
• Real numbers (also called floating-point
numbers) are represented differently
• For fraction or very large numbers
• Store as

sign signed exponent mantissa

• sign is 1 or 0 for negative or positive
• exponent is maximum value (positive or
negative) of base
• mantissa contains significant digits
 Floating-Point Representation
           
e m

  e1 e 2  e m d 1 d 2 d 3  d p
sign of
number
signed exponent mantissa

N   .d1 d 2 d 3  d p B  mB
e e

•m: mantissa
•B: Base of the number system
•e: “signed” exponent
•Note: the mantissa is usually “normalized” if
the leading digit is zero
 Integer representation

Floating-point number representation

 Decimal Representation
• 8-bit word
1 2 3 4
  10 10 10
1 0
10 10 10

              
sign signed exponent number
1|095|1467 (base: B = 10)
mantissa: m = -(1*10-1 + 4*10-2 + 6*10-3 + 7*10-4 ) = -0.1467
signed exponent: e = + (9*101 + 5*100) = 95

10951467 base 10  mB  0.1467  10

e 95
 Floating-Point Representation
• 8-bit word (without normalization)
1 2 3 4
  2 1
2 0
2 2 2 2
              
sign signed exponent number

0|111|0101 (base: B = 2)
mantissa: m = +(0*2-1 + 1*2-2 + 0*2-3 + 1*2-4 ) = 5/16
signed exponent: e = - (1*21 + 1*20) = -3

3
10111001base 2  mB  (5/16)  2 e
 5/128
 Normalization
1 in 2  (1/144) ft 2  0.006944 ft 2 (Less accurate)
 2
 1 in 2
 0 .694444  10 ft 2
(Normalization)
• Remove the leading zero by lowering the exponent (d1
= 1 for all numbers)
 1
1 base  10 : 10  m  1  0.1  m  1
m1 
B base  2 : 1
m1
 2
• if m < 1/2, multiply by 2 to remove the leading 0
• floating-point allow fractions and very large numbers to
be represented, but take up more memory and CPU time
 Binary Representation
• 8-bit word (with normalization)
1 2 3 4
  2 1
2 0
2 2 2 2
              
sign signed exponent number

1|011|1001 (base: B = 2)
mantissa: m = -(1*2-1 + 0*2-2 + 0*2-3 + 1*2-4 ) = -9/16
signed exponent: e = + (1*21 + 1*20) = 3

10111001base 2  mB  (9/16)  2  9/2

e 3
 Single Precision
• A real variable (number) is stored in four words, or
32 bits (64 bits for Supercomputers)
• bit (binary digit): 0 or 1
• byte: 4 bits, 24 = 16 possible values
• word: 2 bytes = 8 bits, 28 = 256 possible values

23 for the digits

32 bits 8 for the signed exponent
1 for the sign
 smallest  .100  00(2)127  0.29387  10 38

 largest  .111  11(2)128
  0.34028  10 39
 Double Precision
• A real variable is stored in eight words, or 64 bits
• 16 words, 128 bits for supercomputers

52 for the digits

64 bits 11 for the signed exponent
1 for the sign
• signed exponent  210 =  1024

 smallest  .100  00(2) 1023


largest  .111 11(2) 1024
 Round-off Errors
• Floating point characteristics contribute to round-off
error (limited bits for storage)
• Limited range of quantities can be represented
• A finite number of quantities can be represented
• The interval between numbers increases as the numbers
grow
•Example - three significant digits

0.0100 0.0101 0.0102 …… 0.0999 (0.0001 increment)

0.100 0.101 0.102 ……. 0.999 (0.001 increment)
1.00 1.01 1.02 ……. 9.99 (0.01 increment)