th
, PT 2)
Objective: Solve for x, given that f(x) = 0
-or equivalently, solve for x such that
g(x) = h(x)
==> f(x) = g(x) h(x) = 0
Applicti!"# (C&C 4
th
, Tble PT2$%, p$ %&'):
heat balance
mass balance
total energy balance in mechanical
an structural systems
!irchhoff"s #a$%
V
i
i &
n
aroun a loo'
(inimi)ing an maximi)ing functions% 0
dS
dF
*n many esign a''lications+
- one as'ect of system is stuie
- equations are nonlinear an im'licit, no analytical or close form
solution is 'ossible
- force to consier numerical methos
Illustrative examples of performance of rootfinding methods can be found in C&C Ch 8.
(ete)*i"e )el )!!t# !+ :
,lgebraic -quations
.ranscenental -quations
/ombinations thereof
E",i"ee)i", Ec!"!*ic# E-*ple:
(unici'al bon has annual 'ayout of 0&,000 for 10 years2
*t costs 03,400 to 'urchase no$2 *m'licit interest rate, i 5
Solution+ 6resent-value, 67, is+
n
& (& i)
67 ,
i
1
+
1
1
]
in $hich+ 67 = 'resent value or 'urchase 'rice = 03,400
, = annual 'ayment = 0&,000
n = number of years = 10
i = interest rate = 5
8e nee to solve the equation for i+
( )
10
& & i
3, 400 &, 000
i
1
+
1
1
]
-quivalently, fin the root of+
( )
10
& & i
f (i) 3, 400 &, 000
i
1
+
1
1
]
= 0
T.! F/"0*e"tl App)!che#:
&2 Bracketing or Closed Methods
- 9isection (etho
- :alse-'osition (etho
12 Open Methods
- ;ne 6oint *teration
- <e$ton-=a'hson *teration
- Secant (etho
1i#ecti!" 2eth!0 (C&C 4
th
, 3$2, p$ %%')
>iven lo$er an u''er bouns, xl an xu $hich brac?et the root+
f(x
&
) f(x
u
) @ 0
&) -stimate the =oot by mi'oint+
l u
r
x x
x
1
+
a
x
r
new
x
r
old
x
r
new
&00C
(c)
u l
x x
() maximum D of iterations is reache
f(x
1
) f(x
r
) > 0
x
f(x)
l u
r
x x
x
1
+
f(x
u
)
(x
u
)
(x
1
)
f(x
1
)
f(x
r
)
f(x
u
)
f(x
1
)
(x
1
)
(x
u
)
(x
r
)
f(x
r
)
f(x)
x
x
r
=> x
l
S/**)4 !+ 1i#ecti!" 2eth!0
Advantages:
&2 Sim'le
12 >oo estimate of maximum error
& u
max
x x
-
1
A2 /onvergence guarantee
i i
E E
max
&
max
4 2 0
+
Disadvantages:
&2 Slo$
12 =equires initial interval aroun root+
Ese gra'h of function,
incremental search, or
trial F error
Fl#e5p!#iti!" 2eth!0 (C&C 4
th
, 3$6, p$ %24)
Similar to bisection2 Eses linear inter'olation to a''roximate root x
r
&)
u & u
r u
& u
f (x ) (x x )
x x
f (x ) f (x )
a
x
r
new
x
r
old
x
r
new
&00C
(c) Bxu x& B
() maximum D of iterations is reache
s
x
f(x)
f(x
u
)
(x
u
)
(x
1
)
f(x
1
)
f(x
r
)
u & u
r u
&
f (x )(x x )
x x
f (x ) f (xu)
f(x
1
) f(x
r
) > 0
x
1
= x
r
f(x)
f(x
u
)
f(x
1
)
(x
1
)
(x
u
)
(x
r
)
f(x
r
)
x
Sc!)e Sheet +!) R!!t+i"0i", E-*ple:
*nitial -st(s) <o2 of *terations <o2 of *terations
(etho Gexact = 02&&HA4I for
s
= 1x&0
-1
for
s
= 1x&0
-3
9isection (0200, &200) H 1J
(0204, 02&4) J 11
:alse-'os2 (0200, &200) && 1K
(0204, 02&4) A &L
S/**)4 !+ Fl#e5P!#iti!" *eth!0
Advantages
&2 Sim'le
12 9rac?ets root
A2 >ives maximum error
Disadvantages
&2 /an be 7-=M slo$
12 #i?e 9isection, nees an initial interval aroun the root
R!!t# !+ E7/ti!"# 5 Ope" 2eth!0# (C&C 4
th
, Ch$ ', p$ %66)
/haracteristics+
&2 *nitial estimates nee not brac?et root
12 >enerally converge faster
A2 NOT guarantee to converge
;'en (ethos /onsiere+
- ;ne 6oint *teration
- <e$ton-=a'hson *teration
- Secant (etho
O"e5p!i"t Ite)ti!" (C&C 4
th
, '$%, p$ %64)
'reict a value xiN& as a function of xi2
/onvert f(x) = 0 to x = g(x)
iteration ste's+ x
iN&
= g(x
i
)
ne$ ol
i i &
x x
+
<e$ton-=a'hson iteration+
x
i+&
x
i
f(x
i
)
f (x
i
)
ne$ ol
i i &
x x
+
a
x
i+&
x
i
x
i+&
&00C
s
A2 (ax2 D iterations is reache
x
i
= x
i+1
i
i & i
i
f (x )
x x
f "(x )
+
Tangent
w/slope=f '(x
i
)
x
f(x)
f(x
i
)
x
i
f(x
i+1
)
i
i
i i &
f (x ) 0
f " (x )
x x
+
x
i+
1
x
f(x)
f(x
i
)
(x
i
)
f(x
i+1
)
i
i
i i &
f (x ) 0
f " (x )
x x
+
x
i+1
1!"0 E-*ple:
.o a''ly <e$ton-=a'hson metho to+
10
& (& i)
f (i) 3400 &000 0
i
1
+
1
1
]
<ee erivative of function+
10
1&
&000 & (& i)
f " (i) 10(& i)
i i
1
+
+
1 ' ;
1
]
Score Sheet for Newton-Raphson Example:
Method Initial Est(s). e
s
= 2x10
-2
e
s
= 2x10
-7
9isection (0200, &200) H 1J
(0204, 02&4) J 11
:alse-'os2 (0200, &200) && 1K
(0204, 02&4) A &L
<-= &20 iverges iverges
02 4 1, but $rong LK
0214 4 3
02&4 A 4
0204 L 4
E))!) A"l4#i# +!) N5R :
=ecall that
i
i & i
i
f(x )
x x
f " (x )
+
.aylor Series gives+
1
r i i r i r i
f "" ( )
f (x ) f (x ) f " (x )(x x ) (x x )
1S
+ +
$here x
r
x
i
an f(x
r
) = 0
Piviing through by f "(x
i
) yiels
1
i i r i r i
i
f "" ( )
0 f(x )Of " (x ) (x x ) (x x )
1f " (x )
+ +
1
i & i r i r i
i
f "" ( )
(x x ) (x x ) (x x )
1f " (x )
+
+ +
;=+
1
r i & r i
i
f "" ( )
(x x ) (x x )
1f " (x )
+
1
i & i
i
f "" ( )
- -
1f " (x )
+
-
iN&
is 'ro'ortional to -
i
1
, uadratic rate of convergence
-
i
small -
iN&
very small% -
i
large anything can ha''en2
S/**)4 !+ Ne.t!"5Rph#!" *eth!0
Advantages
/an be very fast
Disadvantages
may not converge
requires erivative to be evaluate
)ero erivative blo$s u'
Sec"t 2eth!0 (C&C 4
th
, '$6, p$ %43)
Secant metho solution+ ,''rox2 f " (x) $ith bac?$ar :PP+
i & i
i
i & i
f(x ) f(x )
f " (x )
x x
to obtain the iterative ex'ression+
x
i+&
x
i
f(x
i
)(x
i&
x
i
)
f(x
i&
) f(x
i
)
&) =equires t$o initial estimates+
xi-& an xi
.hese o <;. have to brac?et root S
1) (aintains a strict sequence+
x
i+&
x
i
f x
i
( )x
i&
x
i
( )
f x
i& ( )
f x
i ( )
=e'eate until+
a2 B f(x
iN&
) B @ $ith = small number
b2
a
x
i+&
x
i
x
i+&
&00C
s
c2 (ax2 D iterations reache (note no )
A) *f xi an xiN& $ere chosen to brac?et the root, this $oul be the same as the
:alse-6osition (etho2 9E. 8- P;<".S
x
i
= x
i+1
i i & i
i & i
i & i
f (x )(x x )
x x
f (x ) f (x )
x
f(x)
f(x
i
)
x
i
f(x
i-1
)
i-& i
i
i-& i
f(x ) - f(x )
f " (x )
x - x
f(x)
x
i-1
x
i+
1
x
f(x
i
)
x
i
f(x
i-1
)
i-& i
i
i-& i
f(x ) - f(x )
f " (x )
x - x
x
i-1
x
i+1
Score Sheet for Secant Example
no. of iterationsno. of iterations
Method Initial Est(s). !ith es = 2x10
-2
es = 2E-7
Bisection (0.00, 1.00) 9 26
(0.05, 0.15) 6 22
False-os. (0.00, 1.00) 11 2!
(0.05, 0.15) " 1#
$-% 1.0 &i'er(es &i'er(es
0.5 2, )ut *ron( #!
0.25 5 +
0.15 " 5
0.05 # 5
,ecant (0, 1) &i'er(es &i'er(es
(0.00, 0.50)#, )ut *ron( (near c-aotic) 2+
(0.05, 0.15) " 6
$-% 1.00 " #
.as i f(i)/ 0.150 2 #
0.050 # 5
0.0#+cra01 results 22
0.0" con'er(es to i=0
22
:h4 0! !pe" *eth!0# +il;
:unction may not loo? linear2
Remedy: recast into a linear form2 :or exam'le,
f(i) = 7,500 - 1,000
1-(1+i)
-20
i
1
]
1
= 0
is a 'oorly constraine 'roblem in that there is a large, nearly flat )one for $hich
the erivative is near )ero2 =ecast as+
i f(i) = 0 = 3,400 i &000 G & (&Ni)
-10
I
.he recast function, Ti f(i)T $ill have the same roots as f(i) 'lus an aitional
root at i = 02
*t $ill not have a large, flat )one2
.hus, h(i) = i f(i) = 3,400 i &000 G & (&Ni)
-10
I
<-= also nees the first erivative+ h"(i) = 3,400 10,000 (&Ni)
-1&
Fi,/)e: +(i) (#!li0 li"e) , i +(i) (0!tte0 li"e)
R!!t# !+ E7/ti!"# < C#e# !+ 2/ltiple R!!t# (C&C 4
th
, '$4, p$ %3&)
2/ltiple R!!t#: f(x) = (x 1)
1
(x L)
x = 1 re'resents t$o of the three roots2
P)!ble*# "0 App)!che#: C#e# !+ 2/ltiple R!!t#
&2 9rac?eting (ethos fail locating x = 12
f(x
l
) f(x
r
) > 02
12 ,t x = 1, f(x) = f
"(x) = 02
<e$ton-=a'hson an Secant may ex'erience 'roblems2
=ate of convergence ro's to linear2
#uc?ily, f(x) 0 faster than f
"(x) 0
A2 ;ther remeies, recasting 'roblem+
( )
( )
( )
f x
u x
f " x
= 0
u(x) an f(x) have same roots2
<;.-+ /F/ /h2 3 on U=oots of 6olynomialsV is not covere in etail in this
course exce't the useful sections 3212& an 3232
#.0 ".0 2.0 1.0
-"
-2
-1
0
1
2
"
-0.2
0.0 0.2 0.# 0.6 0.! 1.0
x
f
(
x
)
-#000
-2000
0
2000
#000
S/**)4 < Rte# !+ C!"ve),e"ce
i
m
i
i-&
-
lim , 0
-
>
m = &+ linear convergence
m = 1+ quaratic convergence
(etho m
9isection &
:alse 6osition &
Secant, mult2 =oot &
<=, mult2 =oot &
Secant, single root &2J&K Usu'er linearV
<=, single root 1
,ccel2 <=, mult2 =oot (/F/, 42L) 1
NR #!l/ti!"# !+ */ltiv)ite e7/ti!"# (C&C 4
th
, '$3$2, p$ %33)
3e -a'e n nonlinear e4uations, f
5
= 0, 5 = 1,6,n, eac- *it- n
in&een&ent 'aria)les, x
7
, 7 = 1,6,n. 3e see5 a solution for t-e
set of x 'alues t-at si8ultaneousl1 satis9es t-e con&ition t-at all
f
5
= 0.
:n 'ector an& 8atrix notation, t-e $e*ton-%a-son al(orit8
see5s t-e roots ;x< of ;f< = 0 )1 a se4uence of aroxi8ations
;x<
0
, ;x<
1
, 6, ;x<
i
, ;x<
i+1
,6, in *-ic- all t-e 'ectors in&icate&
)1 )races are of &i8ension n x 1 an& ;x<
0
is a suita)le initial
(uess of t-e roots. =i'en t-e i
t-
esti8ate, *e use a 9rst-or&er
>a1lor series exansion to see5 t-e (i+1)
st
?
{ } ( ) { }
{ } ( ) { }
{ } { } ( )
& &
0
i i i i
i
f
f x f x x x
x
+ +
1
+
1
]
(1)
@ere *e &e9ne t-e n x n s4uare 8atrix of artial &eri'ati'es
[ ]
i
i
f
!
x
1
]
as t-e Aaco)ian of t-e s1ste8, an& t-e ele8ent in
t-e 5
t-
ro* an& 7
t-
colu8n of .J/
i
is
"
#
f
x
1
1
]
CDC E4uations (6.21) reresents t-e solution to e4uations (2)
an& (") usin( Cra8erFs rule.
Alte)"tive St!ppi", C)ite)i
&2 ,l$ays limit the number of iterations using an outer P; loo'2
6roblem may not converge an coul try to go on forever2
12 ,bsolute error criteria for UsmallV ifferences+ G x
i
x
i-1
G H
A2 =elative error criteria for Urelatively smallV changes
G x
i
x
i-1
G H Gx
i
G
L2 /an combine error criteria 1 an A so that $or?s for both large an small
x-values+
G x
i
x
i-1
G H + Gx
i
G
42 Sto' if resiual small enough+ G f(x
i)
G H
Th)ee Pe)+!)*"ce C)ite)i
Sto''ing /riteria+
B x
i
x
i-&
B @ N Bx
i
B or B f(x
i)
B @ or (ax2 iterations
/onvergence /riteria+
B x
i
x
i-&
B @ N Bx
i
B and B f(x
i)
B @
<-= or Secant /onfirmation of /onvergent 9ehavior+
x in feasible region and B f(x
i)
B W 024 B f(x
i-&)
B
and B x
i
x
i-&
B W 02J B x
i-&
x
i-1
B
other$ise, o 9isection for a $hile2
A Th)ee Ph#e R!!t+i"0i", St)te,4
, real rootfining 'roblem can be vie$ as having thee 'hases+
%) Opening moves: one nees to fin the region of the 'arameter s'ace in $hich
the esire root can be foun2
8hen using brac?eting methos (bisection, false 'osition), this involves fining
an interval $ith f(x
l
)2f(x
u
) H 02
Enerstaning of the 'roblem, 'hysical insight, an common sense are valuable
here2 (*f it is feasible to gra'h the function, loo?ing at a gra'h hel's2)
Esing a 'o$erful <e$ton-=a'hson algorithm to loo? for a root $here none
exists is a futile effort2
2) Middle !ame: Qere one uses a robust algorithm to reuce the initial region of
uncertainty so that the value of the root can be roughly etermine2 *n one
imension, bisection is a goo strategy here2
*n multivariate 'roblems, a sequence of one-imensional line searches in the
graient irection may $or?% minimi)ing the sum of squares of the iniviual
errors (UresiualsV) allo$s an algorithm to Xuge if 'rogress is being mae2
6) "nd game: ;nce a relatively goo estimate of a root is foun, a <e$ton-
=a'hson or Secant algorithm can use use to generate a highly accurate
solution is a fe$ iterations2
.his may not have been 'ossible in ste' (1) because a linear a''roximation of
the function may not be an aequate escri'tion of the function unless one is
very near the root2
Q/e#ti!": :or the A cases belo$ escribe the 'rogress of the <e$ton-=a'hson metho $hen
solving fi(x) = 0 as either linear, su'erlinear, quaratic, ivergent, chaotic, stuc? near a singularity,
or another a''ro'riate term, an ex'lain 8QM2
555 CASE A 555 555 C#e 1 555 555 C#e C 555
- +(-) - +b(-) - +c(-)
12000 -&12JHJ A2000 -K2000 02&00 &200-N01
A23&K 1&2&0K A2JJ3 -12A30 02&40 L2LL-N0&
A210J L24H& L2&&& -02301 02114 &2HK-N0&
A2010 02L01 L2L03 -0210K 02AAK K23K-N00
A2000 0200L L2J04 -020J1 0240J A2H0-N00
A2000 02000 L23A3 -020&K 0234H &23A-N00
A2000 02000 L2K1L -02004 &2&AH 323&--0&
A2000 02000 L2KKA -02001 &230H A2LA--0&
A2000 02000 L2H11 02000 124JA &241--0&
A2000 02000 L2HLK 02000 A2KLL J233--01
A2000 02000 L2HJ4 02000 423J3 A20&--01
A2000 02000 L2H33 02000 K2J40 &2AL--01
A2000 02000 L2HK4 02000 &12H34 42HL--0A
A2000 02000 L2HH0 02000 &H2LJ1 12JL--0A
8hen <e$ton-=a'hson is $or?ing $ell, one has quaratic convergence $hen /#;S- to the
root2 .his means that near to the root x
r
+ (x
r
x
iN&
) = G024f
V(x
r
)Of
Y(x
r
)I (x
r
x
i
)
1
Qere the ifference bet$een the next estimate of the root x
iN&
,<P the real root x
r
is
'ro'ortional to (x
r
x
i
)
1
2 .hus, once the errors (x
r
x
i
) get small, they get very small very fast2 See
/ase ,2 Mou can create your o$n exam'les $ith f
Y(x
r
) Z 02 Similarly $hen the secant metho is
$or?ing $ell, one obtains su'erlinear convergence2 .his means the ex'onent is about &2J instea
of 1% but the 'erformance loo"s about the same2
,t a multi'le root, the quaratic convergence of <e$ton-=a'hson fails an instea one
obtains linear convergence $hich means that (x
r
x
iN&
) = g
Y() (x
r
x
i
) $here is a 'oint
$ithin the interval an x
iN&
= g(x
i
)2 .his occurs because the linear moel <= uses is no longer
goo near the root2 .he f
Y(x
r
) in the enominator of the error ex'ression for the <= metho
suggests there $ill be trouble2 See case 92 .ry fining $ith <= the root of (x-r)
?
for integer ? > &2
*n this case the <= metho yiels+ (x
r
x
iN&
) = G& (&O?)I (x
r
x
i
)
<ear a 'ole or singularity $here f(x) infinity, the <= algorithm can get stuc?+ it ta?es
very small ste's because the ste' length xiN& xi
= f (xi) O f " (xi) can be 7-=M small even
though f(x) is large2 , istinguishing feature of such cases is the large value of f "(x)2 ,lternatively
if the function in question is something li?e ex'(-x), $hich a''roaches )ero as x , the <=
algorithm can iverge in the sense that x continues to increase $ithout boun% but f(xi) oes get
smaller slo$lyS See case /2 Pivergent behavior $oul also occur if x
i
oscillate bet$een
increasing larger values2 G.ry f(x) = ln(x
1
) 023 = 0 for large x2I
/haotic behavior is obtaine if one searches for the root of f(x) = (x-1)
1
N 020&2 .ry it2 <=
can almost fin a root, an then it shoots 'ast an starts searching again from the other sie2
;ther exam'les $ere 'rovie in class2 %&ese 'e&aviors easily arise in m$ltivariate rootfinding
pro'lems and are t&en &arder to recogni(e