Numerical Methods for Differential Equations

Chapter 5: Partial differential equations – elliptic and parabolic

Gustaf Söderlind and Carmen Arévalo

Numerical Analysis, Lund University

Textbooks: A First Course in the Numerical Analysis of Differential Equations, by Arieh Iserles
and Introduction to Mathematical Modelling with Differential Equations, by Lennart Edsberg

c Gustaf Söderlind, Numerical Analysis, Mathematical Sciences, Lund University, 2008-09

Numerical Methods for Differential Equations – p. 1/50

 1. Brief overview of PDE problems

Classification: Three basic types, four prototype equations

◮ Elliptic
∆u = 0 + BC
◮ Parabolic
ut = ∆u + BC & IC
◮ Hyperbolic
utt = ∆u + BC & IC
ut + a(u)ux = 0 + BC & IC

Numerical Methods for Differential Equations – p. 2/50

 Classification of PDEs

Linear PDE with two independent variables

Auxx + 2Buxy + Cuyy + L(ux , uy , u, x, y) = 0

with L linear in ux , uy , u. Study

 
A B
δ := det   = AC − B 2
B C

Elliptic δ > 0 Parabolic δ = 0 Hyperbolic δ < 0

Numerical Methods for Differential Equations – p. 3/50

 Standard PDEs. Prototypical problems

δ>0 Elliptic PDE Laplace equation

uxx + uyy = 0 A = C = 1; B = 0

δ=0 Parabolic PDE Diffusion equation

ut = uxx A = 1; B = C = 0

δ<0 Hyperbolic PDEs Wave equation

utt = uxx A = 1; B = 0; C = −1

Numerical Methods for Differential Equations – p. 4/50

 PDE method types

FDM Finite difference methods

FEM Finite element methods
FVM Finite volume methods
BEM Boundary element methods

We will mostly study FDM to cover basic theory

Industrial relevance: FEM

Numerical Methods for Differential Equations – p. 5/50

 PDE methods for elliptic problems

Simple geometry FDM or Fourier methods

Complex geometry FEM
Special problems FVM or BEM

Large sparse systems

Combine with iterative solvers such as multigrid methods

Numerical Methods for Differential Equations – p. 6/50

 PDE methods for parabolic problems

Simple geometry FDM or Fourier methods

Complex geometry FEM

Stiffness
Always use A–stable time-stepping methods

Need Newton-type solvers for large sparse systems

Numerical Methods for Differential Equations – p. 7/50

 PDE methods for hyperbolic problems

FDM, FVM. Sometimes FEM

Shocks
Solutions may be discontinuous – example: “sonic boom”

Turbulence
Multiscale phenomena

Hyperbolic problems have several complications and many

highly specialized techniques are often needed

Numerical Methods for Differential Equations – p. 8/50

 2. Elliptic problems. FDM

∂2 ∂2 ∂2
Laplacian ∆= 2
+ 2+ 2
∂x ∂y ∂z

Laplace equation ∆u = 0
with boundary conditions u = u0 (x, y, z) x, y, z ∈ ∂Ω

Poisson equation −∆u = f

with boundary conditions u = u0 (x, y, z) x, y, z ∈ ∂Ω

Other boundary conditions also of interest (∂Ω = bdry of Ω)

Numerical Methods for Differential Equations – p. 9/50

 Elliptic problems. Some applications

◮ Equilibrium problems
Structural analysis (strength of materials)
Heat distribution

◮ Potential problems
Potential flow (inviscid, subsonic flow)
Electromagnetics (fields, radiation)

◮ Eigenvalue problems
Acoustics
Microphysics

Numerical Methods for Differential Equations – p. 10/50

 Poisson equation – an elliptic model problem

∂ 2u ∂ 2u
2
+ 2 = f (x, y)
∂x ∂y

Computational domain Ω = [0, 1] × [0, 1] (unit square)

Dirichlet conditions u(x, y) = 0 on boundary

Uniform grid {xi , yj }N,M

i,j=1 with equidistant mesh widths
∆x = 1/(N + 1) and ∆y = 1/(M + 1)

Discretization Finite differences with ui,j ≈ u(xi , yj )

ui−1,j − 2ui,j + ui+1,j ui,j−1 − 2ui,j + ui,j+1

2
+ 2
= f (xi , yj )
∆x ∆y

Numerical Methods for Differential Equations – p. 11/50

 Equidistant mesh ∆x = ∆y
ui−1,j + ui,j−1 − 4ui,j + ui,j+1 + ui+1,j
2
= f (xi , yj )
∆x

Participating approximations and mesh points

yj+1

yj

yj−1

xi−1 xi xi+1
Numerical Methods for Differential Equations – p. 12/50
Computational “stencil” for ∆x = ∆y

ui−1,j + ui,j−1 − 4ui,j + ui,j+1 + ui+1,j

2
= f (xi , yj )
∆x

1 “Five-point operator”

1 −4 1

Numerical Methods for Differential Equations – p. 13/50

 The FDM linear system of equations
Lexicographic ordering of unknowns ⇒ partitioned system
    
T I 0 ... u·,1 f (x· , y1 )
 I T I   u·,2   f (x· , y2 )
    

    
1  I T I   u·,3   f (x· , y3 ) 
=
    
 . ..
2
∆x    ..
 
  . 
..
    
. I 
    
   
... 0 I T u·,N f (x· , yN )

with Toeplitz matrix T = tridiag(1 −4 1)

The system is N 2 × N 2 , hence large and very sparse

Numerical Methods for Differential Equations – p. 14/50

 3. Elliptic problems. FEM

Finite Element Method

PDE Lu = 0
P P
Ansatz u= ci ϕ i ⇒ Lu = ci Lϕi
Requirement hϕi , Lui = 0 gives coefficients {ci }

FEM is a least squares approximation, fitting a linear

combination of basis functions {ϕi } to the differential
equation using orthogonality

Simplest case Piecewise linear basis functions

Numerical Methods for Differential Equations – p. 15/50

 Strong and weak forms

Strong form

−∆u = f ; u = 0 on ∂Ω

Take v with v = 0 on ∂Ω
Z Z
−∆u · v = f ·v
Ω Ω

Integrate by parts to get weak form

Z Z
∇u · ∇v = f ·v
Ω Ω

Numerical Methods for Differential Equations – p. 16/50

 Strong and weak forms. 1D case

Recall integration by parts in 1D

Z 1 Z 1
1
−u′′ v = [−u′ v]0 + u′ v ′
0 0

or in terms of an inner product

−hu′′ , vi = hu′ , v ′ i

Generalization to 2D, 3D uses vector calculus

Numerical Methods for Differential Equations – p. 17/50

 Weak form of −∆u = f

Define inner product

Z
hv, ui = vu dΩ
Ω

and energy norm (note scalar product!)

Z
a(v, u) = ∇v · ∇u dΩ
Ω

to get the weak form of −∆u = f as

a(v, u) = hv, f i

Numerical Methods for Differential Equations – p. 18/50

 Galerkin method (Finite Element Method)

1. Basis functions {ϕi }

P
2. Approximate u = cj ϕ j
P R R
3. Determine cj from cj ∇ϕi · ∇ϕj = f ϕi

The cj are determined by the linear system

Kc = F

The matrix K is called stiffness matrix

R
Stiffness matrix elements kij = ∇ϕi · ∇ϕj = a(ϕi , ϕj )
R
Right-hand side Fi = ϕi f = hϕi , f i

Numerical Methods for Differential Equations – p. 19/50

 The FEM mesh. Domain triangulation
Piecewise linear basis {ϕj } require domain triangulation
1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Numerical Methods for Differential Equations – p. 20/50

 4. Parabolic problems

The prototypical equation is the

Diffusion equation ut = ∆u

Nonlinear diffusion

ut = div (k(u)gradu)

Boundary and initial conditions are needed

Numerical Methods for Differential Equations – p. 21/50

 Parabolic problems. Some applications

◮ Diffusive processes
Heat conduction ut = d · uxx

◮ Chemical reactions
Reaction–diffusion ut = d · uxx + f (u)

1
Convection–diffusion ut = ux + u
Pe xx

Irreversibility ut = −∆u is not well-posed!

Numerical Methods for Differential Equations – p. 22/50

 Diffusion – a parabolic model problem

Equation ut = uxx
Initial values u(x, 0) = g(x)
Boundary values u(0, t) = u(1, t) = 0

Separation of variables u(x, t) := X(x)T (t) ⇒

Ṫ X ′′
ut = X Ṫ , uxx = X ′′ T ⇒ = =: λ
T X

λt
√ √
T = Ce X = A sin −λ x + B cos −λ x

Numerical Methods for Differential Equations – p. 23/50

 Parabolic model problem. . .

Boundary values
X(0) = X(1) = 0 ⇒ λk = −(kπ)2 , therefore
√ −(kπ)2 t
Xk (x) = 2 sin kπx Tk (t) = e

Initial values
P∞ √
Fourier expansion g(x) = 1 ck 2 sin kπx ⇒

√ X∞
−(kπ)2 t
u(x, t) = 2 ck e sin kπx
k=1

Numerical Methods for Differential Equations – p. 24/50

 5. Method of lines (MOL) discretization

In ut = uxx , discretize ∂ 2 /∂x2 by

ui−1 − 2ui + ui+1

uxx ≈
∆x2

System of ODEs (semidiscretization) u̇ = T∆x u

 
−2 1
1  1 −2 1
 
 
u̇ = ...
u
∆x2 



1 −2

Numerical Methods for Differential Equations – p. 25/50

 Full FDM discretization

Note that ui (t) ≈ u(x, t) along the line x = xi in (x, t) plane

Use time-stepping to solve the IVP; Explicit Euler with

ui,j ≈ u(xi , tj ) implies

ui,j+1 − ui,j ui−1,j − 2ui,j + ui+1,j

=
∆t ∆x2

With the Courant number µ = ∆t/∆x2 we obtain recursion

ui,j+1 = ui,j + µ · (ui−1,j − 2ui,j + ui+1,j )

Numerical Methods for Differential Equations – p. 26/50

 Method of lines. The grid
Rectangular grid {(i · ∆x, j · ∆t), i = 0 : N + 1, j ≥ 0} with
∆x = 1/(N + 1)
8

4
t

0
0 1 2 3 4 5 6 7
x

ui,j ≈ u(i · ∆x, j · ∆t)

Numerical Methods for Differential Equations – p. 27/50

 Method of lines. Computational stencil

Explicit Euler time stepping. Participating grid points

tj+1
tj

xi−1 xi xi+1

1
Courant number µ = ∆t/∆x2

−µ 2µ−1 −µ

Numerical Methods for Differential Equations – p. 28/50

Method of lines. Stability and the CFL condition
Explicit Euler with ∆x = 1/(N + 1) implies recursion

u·,j+1 = u·,j + ∆t · T∆x u·,j

2 kπ
2
Recall λk [T∆x ] = −4(N + 1) sin for k = 1 : N
2(N + 1)

Stability requires ∆t · λk ∈ S for all eigenvalues

1.5

1
 
4∆t 2 0.5

∆t · λk ∈ − 2
, −π ∆t S
∆x

Im
0

−0.5

−1

−1.5
−2.5 −2 −1.5 −1 −0.5 0 0.5
Re

Numerical Methods for Differential Equations – p. 29/50

 The CFL condition

For stability we need 4∆t/∆x2 ≤ 2

CFL condition (Courant, Friedrichs, Lewy 1928)

∆t 1
2
≤
∆x 2

The CFL condition is a severe restriction on time step ∆t

Stiffness The CFL condition can be avoided by using

A-stable methods, e.g. Trapezoidal Rule or Implicit Euler

Numerical Methods for Differential Equations – p. 30/50

 Experimental stability investigation
N = 30 internal pts in [0, 1], M = 187 time steps on [0, 0.1]
Stable solution at CFL = .514

0.8

0.6

0.4

0.2

0
1
0.8 0.1
0.6 0.08
0.4 0.06
0.04
0.2
0.02
0 0
Numerical Methods for Differential Equations – p. 31/50
 Violating the CFL condition. Instability
N = 30 internal pts in [0, 1], M = 184 time steps on [0, 0.1]
Unstable solution at CFL = .522

0.8

0.6

0.4

0.2

−0.2

−0.4
1
0.8 0.1
0.6 0.08
0.4 0.06
0.04
0.2
0.02
0 0
Numerical Methods for Differential Equations – p. 32/50
 Crank–Nicolson method (1947)

Crank–Nicolson method ⇔ Trapezoidal Rule for PDEs

The trapezoidal rule is

◮ implicit ⇒ more work/step

◮ A–stable ⇒ no restriction on ∆t

Theorem Crank–Nicolson is unconditionally stable

There is no CFL condition on the time-step ∆t

Numerical Methods for Differential Equations – p. 33/50

 Crank–Nicolson method. . .

Courant number µ = ∆t/∆x2 ⇒ recursion

µ µ
(I − T )u·,j+1 = (I + T )u·,j
2 2
with Toeplitz matrix T = tridiag(1 − 2 1)

Tridiagonal structure ⇒ low complexity

Refactorize only if Courant number µ = ∆t/∆x2 changes

Numerical Methods for Differential Equations – p. 34/50

 6. Error analysis. Convergence

MOL with explicit Euler for ut = uxx

Global error ei,j = ui,j − u(xi , tj )

Local error Insert exact solution to get

u(xi , tj+1 ) − u(xi , tj )

=
∆t
u(xi−1 , tj ) − 2u(xi , tj ) + u(xi+1 , tj )
= 2
− li,j
∆x

Expand local error li,j in Taylor series

Numerical Methods for Differential Equations – p. 35/50

 Local error

Taylor expansion

∆t ∆x2
−li,j = ut − uxx + utt + uxxxx + O(∆t2 , ∆x4 )
2 12

Therefore

∆t ∆x2
−li,j = utt + uxxxx + O(∆t2 , ∆x4 )
2 12

Numerical Methods for Differential Equations – p. 36/50

 The Lax Principle

Conclusion
Consistency li,j → 0 as ∆t, ∆x → 0
Stability CFL condition ∆t/∆x2 ≤ 1/2
Convergence ei,j → 0 as ∆t, ∆x → 0

Theorem (Lax Principle)

Consistency + Stability ⇒ Convergence

Note Choice of norm is very important

Numerical Methods for Differential Equations – p. 37/50

 The order of the method
With local error
∆t ∆x2
−li,j = utt + uxxxx = O(∆t, ∆x2 )
2 12

and stability in terms of CFL condition µ = ∆t/∆x2 ≤ 1/2

we have global error ei,j = O(∆t, ∆x2 )

For fixed µ we have ∆t ∼ ∆x2 and it follows that

Global error ei,j = O(∆t, ∆x2 ) = O(∆x2 ) ⇒

Theorem The order of convergence is p = 2

Numerical Methods for Differential Equations – p. 38/50

 Order of PDE discretizations

Discretization process from PDE to SD to FD

ut = uxx → v̇∆x = P∆x v∆x + h∆x (t) → uj+1

µ = A uj
µ µ + kµ
j

Suppose
◮ order of SD scheme for spatial variables is p1
◮ order of ODE time discretization is p2

Theorem If ∆t = µ∆x2 the FD order of convergence is

p = min{p1 , 2p2 }

Numerical Methods for Differential Equations – p. 39/50

 Example. The Crank–Nicolson method
From Trapezoidal rule yn+1 = yn + h2 [f (yn ) + f (yn+1 )]

∆t
uj+1
i = uji + 2
[(uj
i−1 − 2uj
i + uj
i+1 ) + (uj+1
i−1 − 2uj+1
i + uj+1
i+1 )]
2∆x

µ j+1 j+1 µ j+1 µ j j µ j

− ui−1 + (1 + µ)ui − ui+1 = ui−1 + (1 − µ)ui + ui+1
2 2 2 2

Same order p = min{2, 4} = 2 as with Explicit Euler

tj+1
tj

xi−1 xi xi+1
Numerical Methods for Differential Equations – p. 40/50
 Crank–Nicolson. Stability

µ −1 µ
Aµ = (I − T ) (I + T ) with usual Toeplitz matrix T
2 2
1 + µ2 λ[T ]
Theorem The eigenvalues are λ[Aµ ] =
1 − µ2 λ[T ]

Note λ[T ] ∈ (−4, 0) ⇒ −1 < λ[Aµ ] < 1 This implies that

there is no CFL stability condition on the Courant ratio µ!
The method is stable for all ∆t > 0

Theorem Crank–Nicolson is unconditionally stable

Numerical Methods for Differential Equations – p. 41/50

 Experimental stability investigation
N = 30 internal pts in [0, 1], M = 30 time steps on [0, 0.1]
Stable solution at CFL = 3.2

0.8

0.6

0.4

0.2

0
1
0.8 0.1
0.6 0.08
0.4 0.06
0.04
0.2
0.02
0 0
Numerical Methods for Differential Equations – p. 42/50
 7. Parabolic problems. FEM

Consider diffusion problem in strong form ut − uxx = 0 with

Dirichlet boundary conditions

Multiply by test function v and integrate by parts

Z 1 Z 1
vut dx + v ′ u′ dx = 0
0 0

In terms of inner product and energy norm –

Weak form hv, ut i + a(v, u) = 0 for all v with v(0) = v(1) = 0

Numerical Methods for Differential Equations – p. 43/50

 Galerkin method (Finite Element Method)

1. Basis functions {ϕi }

P
2. Approximate u(t, x) = cj (t)ϕj (x)
3. Determine cj from hϕi , ut i + a(ϕi , u) = 0
P P
Note hϕi , ut i = ċj hϕi , ϕj i and a(ϕi , u) = cj hϕ′i , ϕ′j i

We get an initial value problem

B∆x ċ + K∆x c = 0

for the determination of the coefficients cj (t) with c(0)

determined by the initial condition

Numerical Methods for Differential Equations – p. 44/50

 Galerkin method

Simplest case
Piecewise linear basis functions on equidistant grid

Stiffness matrix elements kij = hϕ′i , ϕ′j i

1
K∆x = tridiag(−1 2 −1)
∆x

and mass matrix elements bij = hϕi , ϕj i

∆x
B∆x = tridiag(1 4 1)
6

Numerical Methods for Differential Equations – p. 45/50

 Simplest Galerkin FEM. . .

Note that in the initial value problem

B∆x ċ + K∆x c = 0

the matrix B∆x is tridiagonal ⇒ no advantage from

explicit time stepping methods

Explicit Euler

B∆x (cn+1 − cn ) = −∆t · K∆x cn

requires the solution of a tridiagonal system on every step

Numerical Methods for Differential Equations – p. 46/50

 Simplest Galerkin FEM. . .

As the system is stiff, consider implicit A-stable method

∆t
B∆x (cn+1 − cn ) = − · K∆x (cn + cn+1 )
2

and solve tridiagonal system

∆t ∆t
(B∆x + K∆x )cn+1 = (B∆x − K∆x )cn
2 2

on every step

Trapezoidal rule has same cost, but better stability

Numerical Methods for Differential Equations – p. 47/50

 8. Well-posedness

Linear partial differential equation

ut = Lu + f , 0 ≤ x ≤ 1, t ≥ 0, u(x, 0) = h(x),
u(0, t) = φ0 (t), u(1, t) = φ1 (t)

(
wt = Lw + f, w(x, 0) = h(x)
Suppose
vt = Lv + f, v(x, 0) = h(x) + g(x)

Subtract to get homogeneous Dirichlet problem

ut = Lu, u(x, 0) = g(x), φ0 (t) ≡ 0, φ1 (t) ≡ 0

Numerical Methods for Differential Equations – p. 48/50

 Well-posedness. Time evolution

Suppose time evolution u(x, t) = E(t)g(x)

Definition The equation is well-posed if for every t∗ > 0

there is a constant 0 < C(t∗ ) < ∞ such that
kE(t)k ≤ C(t∗ ) for all 0 ≤ t ≤ t∗

Theorem A well-posed equation has a solution that

◮ depends continuously on the initial value (the “data”)
◮ is uniformly bounded in any compact interval

Numerical Methods for Differential Equations – p. 49/50

 ut = uxx is well posed
√ P∞
Fourier series expansion g(x) = 2 1 ck sin kπx implies

√ X∞
−(kπ)2 t
u(x, t) = 2 ck e sin kπx
k=1

Z 1
kE(t)gk22 = |u(x, t)|2 dx
0
∞ X
X ∞ Z 1
−(k2 +j 2 )π 2 t
= 2 ck cj e sin kπx sin jπx dx
k=1 j=1 0
∞
X ∞
X
2 −2(kπ)2 t
= ck e ≤ c2k = kgk22
k=1 k=1

Hence kE(t)k2 ≤ 1 for every t ≥ 0

Numerical Methods for Differential Equations – p. 50/50