Supplemental Material for ELEC 603

Spring 2006

Variational Methods in Optimization

References 1. I. M. Gelfand and S. V. Formin, Calculus of Variations, Dover Publications Inc., 2000. 2. L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D., vol. 60, pp. 259-268, 1992. 3. L. Rudin and S. Osher, Total variation based image restoration with free local constraints, Proc. ICIP, pp. 31-35, Austin, TX., 1994. 4. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, SIAM Journal on Multiscale Modeling and Simulation, vol. 4, no. 2, pp. 460-489, 2005. 5. Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series, vol. 22, American Mathematical Society, Providence, RI., 2001. 6. UCLA Reports: http://www.math.ucla.edu/applied/cam/index.html 1. Elements of Calculus of Variations 1.1 Functionals

Roughly speaking, a functional is a function of function. We all know the definition of function, say f(x) which is taken to mean a correspondence that assigns a definite (real) number for each variable value x in a certain domain. Similarly, a functional is a correspondence that assigns a real number to each function (such as a curve, or a surface, etc.) that belongs to some class of functions. For example, for each curve that connects two fixed points a and b, the length of the curve is a functional. As another example, the time required for a mass point traveling down along a smooth and monotonically decreasing plane curve with two fixed ends is a functional. From these examples, we see that a functional is defined over a certain class of functions. 1.2 Variational Problems: Examples

Example 1 A smooth plane curve that joins points pa = (a, A) and pb = (b, B) can be described by a differentiable function y = y(x) with A= y(a) and B = y(b), and its length is known to be
J [ y] =
b

1 + y '2 dx

(1)

Here J[y] is obviously a functional for the class of smooth functions y(x) that satisfy the boundary conditions A = y(a) and B = y(b). A simple variational problem associated with the functional in (1) is to find the shortest curve joining points pa and pb. Typically, for the one-variable case a functional of interest assumes the form

J [ y ] = F ( x, y, y ') dx
a

(2)

and for the two-variable case

J [u ] = F ( x, y, u , u x , u y ) dxdy
R

(3)

where u = u(x, y), u x = u / x and u y = u / y. Example 2 Let u(x, y) denote a two-variable smooth function that models the light intensity of an (analog) image over region R. The total variation (TV) of the image is defined as
J [u ] =
2 2 u x + u y dx dy

(4)

which is obviously a functional of type (3) for the class of smooth functions defined over region R. It has been argued that the noise removal problem for images can be cast as minimizing the TV subject to some statistical constraints.
1.3 Several Lemmas [1]

The key analytical tool for investigating various image processing problems in a variational optimization setting is the Euler-Lagrange (EL) equation. The lemmas presented below are instrumental in deriving the EL equation. In what follows the set of continuous functions over interval [a, b] is denoted by C(a, b) and the set of functions that are continuous and have up to p-th order continuous derivatives on [a, b] is denoted by Dp(a, b).
Lemma 1 If ( x) is continuous on [a, b], and if

( x)h( x) dx = 0

for every function h(x) C (a, b) such that h(a) = h(b) = 0, then ( x) = 0 for all x in [a, b].
Proof. The lemma will be proved by the method of contradiction. Suppose the function ( x ) is nonzero, say positive, at some point in [a, b]. Then ( x) is also positive in some interval [x1, x2] contained in [a, b]. If we set h(x) = (x x1)(x2 x) for x in [x1, x2] and h(x) = 0 otherwise, then h(x) obviously satisfies the condition of the lemma. However,

( x)h( x) dx = ( x)( x x1 )( x2 x)dx > 0

x1

x2

because the integrand is positive (except at x1 and x2). This contradiction proves the lemma.
Lemma 2 If ( x) is continuous in [a, b], and if

( x)h '( x) dx = 0

for every h(x) D1 (a, b) such that h(a) = h(b) = 0, then ( x) = c for all x in [a, b], where c is a constant. Proof. Let c be a constant given by
c= which implies that 1 b ( x) dx b a a

and define function h(x) as

[ ( x) c]dx = 0 ,

h( x) = [ ( ) c]d .
a

Evidently, h(x) belongs to D1(a, b) and satisfies h(a) = 0 and h(b) = 0. Then on the one hand,

[ ( x) c]h '( x) dx = ( x)h '( x) dx c[h(b) h(a)] = 0 .

a

while on the other hand,

[ ( x) c]h '( x) dx = [ ( x) c]2 dx .

a

Therefore, ( x) c = 0 i.e. ( x) = c for all x in [a, b].

Lemma 3 If ( x) is continuous in [a, b], and if

( x)h ''( x) dx = 0

for every h(x) D2 (a, b) such that h(a) = h(b) = 0 and h(a) = h(b) = 0, then ( x) = c0 + c1x for all x in [a, b], where c0 and c1 are constants. Proof. Let c0 and c1 be constants defined by the conditions

and let

[ ( x) c0 c1 x] dx = 0 dx [ ( ) c0 c1 ] d = 0
a x

h( x) = d [ (t ) c0 c1t ] dt
a a

It is easy to verify that h(x) D2 (a, b) and satisfies h(a) = h(b) = 0 and h(a) = h(b) = 0. Then on the one end,

[ ( x) c0 c1 x]h ''( x) dx
b b

= ( x)h ''( x) dx c0 [h '(b) h '(a)] c1 xh ''( x) dx

a a

= c1[bh '(b) ah '(a)] c1[h(b) h(a )] = 0

while on the other hand,

[ ( x) c0 c1 x]h ''( x) dx = [ ( x) c0 c1 x]2 dx = 0

a

It follows that ( x) c0 c1 x = 0 , hence ( x) = c0 + c1 x for all x in [a, b].

Lemma 4 If ( x) and ( x) are continuous in [a, b], and if

[ ( x)h( x) + ( x)h '( x)]dx = 0

(5)

for every function h(x) D1 (a, b) such that h(a) = h(b) = 0, then ( x) is differentiable, and '( x) = ( x) for all x in [a, b]. Proof. Let A(x) be the function defined by

A( x) = ( ) d
a

Using integration by parts, we can verify that

hence (5) can be written as

( x)h( x) dx = A( x)h '( x) dx

a

[ A( x) + ( x)]h '( x) dx = 0

(6)

By Lemma 2, (6) implies that ( x) A( x) = const, i.e., ( x ) = A( x ) + const, thus function ( x) is differentiable and by definition of A(x), '( x) = ( x) .
1.4 Variation of a Functional

Let J[y] be a functional defined on some normed linear space and let J [h] = J [ y + h] J [ y ] be its increment corresponding to the increment h = h(x) of the independent variable y = y(x). If y is fixed, then J [ h] is a functional of h, in general a nonlinear functional. Suppose that
J [h] = [h] + h

where [h] is a linear functional and 0 as h 0. Then the functional J[y] is said to be differentiable, and the principal linear part of the increment J [h] , i.e., the linear functional [h] which differs from J [h] by an infinitesimal of order higher than 1 relative to h , is called the variation of J[y] and is denoted by J [h] . It can be shown that the variation of a differentiable functional is unique [1]. We now use the concept of the variation of a functional to establish a necessary condition for a functional to have an extremum. Similar to the concept of the extremum of a function from analysis, we say that the functional J[y] has a relative extremum for y = y if J [ y ] J [ y ] does not change its sign in some neighborhood of the curve y = y ( x) . At this point, it is important that the meaning of the term neighborhood is explicitly clarified. To this end we need to define two norms, one for space C(a, b) and the other for D1(a, b):

 For a function y(x) in space C(a, b), we define

y 0 = max | y ( x) |
a x b

For a function y(x) in space D1(a, b), we define

y 1 = max | y ( x) | + max | y '( x) |
a x b a x b

Depending on which of these two norms (topology) is employed, a neighborhood of the curve y = y ( x) may be referred to those y such that y y 0 < , or to those y such that y y 1 < . Accordingly, we shall say that the functional J[y] has a weak extremum for y = y , if there exist an > 0, such that J [ y ] J [ y ] has the same sign for all y in the neighborhood of y ( x) characterized by y y 1 < . On the other hand, we shall say that the functional J[y] has a strong extremum for y = y , if there exist an > 0, such that J [ y ] J [ y ] has the same sign for all y in the neighborhood of y ( x) characterized by y y 0 < .
Theorem 1 [1] A necessary condition for the differentiable functional J[y] to have an extremum for y = y is that its variation vanish for y = y , namely,

J [ h] = 0
for y = y and all admissible h. Proof. To be explicit, suppose J[y] has a minimum for y = y . By the definition of the variation J [h] , we have
J [ h ] = [ h ] + h

(7)

where 0 as h 0. Hence, for sufficiently small h , the sign of J [h] is the same as the sign of J [h] . Now, suppose that J [h0 ] 0 for some admissible h0. Then for any > 0 , no matter how small, we have

J [ h0 ] = J [ h0 ]
because J [h] is a linear functional of h. Hence (7) can be made to have either sign for arbitrary small h , which is not possible according to the hypothesis J[y] has a minimum for y = y . This contradiction proves the theorem.

1.5

Euler-Lagrange Equation: The One-Variable Case

We now study the simplest variational problem with one variable: Let F(x, y, z) be a function with continuous first and second (partial) derivatives with respect to all its variables. Then, among all functions in D1(a, b) that satisfy the boundary conditions y(a) = A, find the function for which the functional
J [ y ] = F ( x, y, y ') dx
a b

y(b) = B,

(8)

(9)

has a weak extremum. Suppose we give y(x) an increment h(x). In order for the function y(x) + h(x) to continue to satisfy the boundary condition (8), we must impose h(a) = h(b) = 0 The corresponding increment of the functional (9) is given by
J = J [ y + h] J [ y ] = F ( x, y + h, y '+ h ') dx F ( x, y, y ') dx
a a b b

(10)

= [ F ( x, y + h, y '+ h ') F ( x, y, y ') ] dx

b a

Using Taylors theorem, it follows that

J = [ Fy ( x, y, y ')h + Fy ' ( x, y, y ')h '] dx +
a b

(11)

where the subscripts denote partial derivatives with respect to the corresponding arguments, and the dots denote terms of order higher than 1 relative to h and h. The integral in the right-hand side of (11) represents the principal linear part of the increment J , hence the variation of J[y] is given by

J = Fy ( x, y, y ')h + Fy ' ( x, y, y ')h ' dx

a

By Theorem 1, a necessary condition for J[y] to have an extreme for y = y(x) is that

J = ( Fy h + Fy ' h ') dx = 0
b a

(12)

for all admissible h. According to Lemma 2, (12) implies that 7

Fy

d Fy ' = 0 dx

(13)

The equation in (13) is known as the Euler-Lagrange equation. Thus we have proved
Theorem 2 Let J[y] be a functional of the form
J [ y ] = F ( x, y, y ') dx
a b

defined on the set of functions which have continuous first derivatives in [a, b] and satisfy the boundary conditions y(a) = A, y(b) = B. Then a necessary condition for J[y] to have an extremum for a given y(x) is that y(x) satisfies the Euler-Lagrange equation Fy
1.6

d Fy ' = 0 dx

Euler-Lagrange Equation: The Two-Variable Case

In this section we consider functionals depending on functions of two variables that assume the form of
J [u ] = F ( x, y, u , u x , u y ) dx dy
R

(14)

where R is some closed region and ux, uy are partial derivatives of u = u(x, y). Similar to Lemma 1 (for the one-variable case), we have
Lemma 5 If ( x, y ) is a fixed function which is continuous in a closed region R, and if the integral

( x, y) h( x, y) dx dy
R

vanishes for every function h(x, y) which has continuous first and second derivatives in R and equals zero on the boundary of region R, then ( x, y ) = 0 every in R. The proof of the above lemma is similar in spirit to that of Lemma 1 and the interested reader is referred to [1]. In order to apply the necessary condition for an extremum of the functional (14), i.e., J = 0 , we must first calculate the variation J . Let h(x, y) be an arbitrary function which has continuous first and second derivative and vanishes in the boundary of R. Then if u(x, y) belongs to the domain of definition, i.e., u(x, y) and its first and second derivatives are continuous in R and u(x, y) takes given values on boundary , so does u(x, y) + h(x, y). It follows that

J = J [u + h] J [u ] = F ( x, y, u + h, u x + hx , u y + hy ) F ( x, y, u , u x , u y ) dx dy R = Fu h + Fux hx + Fu y hy dx dy + R The integral in the last equality represents the principal linear part of the increment J , hence the variation of J[u] is given by

J = Fu h + Fu hx + Fu hy dx dy . R
x y

By using Greens theorem

we can write

Q P dx dy = ( P dx + Q dy ) x y

( F h + F h ) dx dy
ux x uy y

= Fux h + ( Fu y h) dx dy Fux + Fu y h dx dy R x R x y y

= Fux h dy Fu y h dx Fux + Fu y h dx dy R x y

= Fux + Fu y h dx dy R x y where the last equality is obtained by the fact that h(x, y) assumes zero value on boundary . Consequently, the variation of the functional J[u] becomes

J = Fu
R

Fux Fu y h( x, y ) dx dy x y

(15)

Thus the condition J = 0 implies that the integral in (15) vanishes for any h(x, y) satisfying the zero boundary condition. By Lemma 5, this leads to the second-order partial differential equation, again known as the Euler-Lagrange equation:

Fu

Fux Fu y = 0 x y

(16)

2. Functions of Bounded Variation 2.1 Bounded Variation Functions of One Variable

Let u(x) be a function defined over [a, b] on which it always assumes finite values. For a set grid points a = x0 < x1 < < xn = b, we construct the sum

Vu ( x0 ,

, xn ) = | u ( xi ) u ( xi 1 ) |
i =1

which is called the variation of u(x) with respect to grid set (x0, , xn). If there a constant M such that
x0 ,, xn

sup Vu ( x0 ,

, xn ) M <

regardless of the number of grid points, then u(x) is called a function of bounded variation (BV), and (u ) = sup Vu ( x0 ,
a x0 ,, xn b

, xn )

is called the total variation (TV) of u(x) on [a, b]. In what follows, u(x) is BV is taken to mean that u(x) is a function of BV.
Example 1 Let u(x) be a function defined on [a, b] that satisfies the Lipschitz condition, namely, there exists a constant M such that, for any x and x on [a, b], | u ( x) u ( x ') | M | x x ' | holds. It follows that

Vu ( x0 ,
b

, xn ) = | u ( xi ) u ( xi 1 ) | M | xi xi 1 | = M (b a)
i =1 i =1

which implies that (u ) M (b a ) , hence u(x) is BV. In addition, if we use Lagranges

a

formula u ( x1 ) u ( x2 ) = u '( ) ( x1 x2 ) for some [ x1 , x2 ] , then we conclude that if u(x) has bounded first-order derivative on [a, b], then u(x) is BV.
Example 2 A continuous function is not necessarily BV. For example, function u(x) defined by

10

1 x sin u ( x) = x 0 is continuous on [0, 1]. If we take

for 0 < x 1 for x = 0

xn =

1 n +

n = 0,1, , m

together with points 0 and 1 as the grid points, then the associated variation (see the figure below) can be estimated as

Vu (0, xm ,

, x0 ,1) xi sin
i 1

1 1 1 m 1 xi 1 sin xi xi 1 i =1 i

Hence sup Vu (0, xm ,

m

, x0 ,1)

i
i =1

and u(x) is not BV.

Continuous Function in Example 2 0.7

0.6

0.5

0.4

0.3

u(x)

0.2

0.1 x1 0 x2 0.1 x0

0.2

0.1

0.2

0.3 x

0.4

0.5

0.6

0.7

11

Some properties and characterization of BV functions are listed below without proof. If u(x) is monotonic on [a, b], then u(x) is BV and (u ) =| u (a ) u (b) | .
a b

If u(x) is BV, then it is a bounded function, i.e., there exists a constant M such that | u ( x) | M for x [a, b]. If u(x) is BV and (u ) = 0 , then u(x) on [a, b] is a constant.
a b

If u(x) and v(x) are BV on [a, b] and and are constants, then u ( x) + v( x) is BV. If u(x) and v(x) are BV on [a, b], then u(x) v(x) is BV. If u(x) is BV on [a, b] and c is any real value with a < c < b, then
(u ) = (u ) + (u )
a a c b c b

Conversely, if u(x) is BV on [a, c] and [c, b], then u(x) is also BV on [a, b]. If u(x) is BV on [a, b], then (u ) is a monotonically increasing function of x on [a, b].
a x

Jordans Decomposition Theorem If u(x) is defined on [a, b] and is BV, then u(x) can be expressed is the difference of two monotonically increasing functions, e.g., u ( x) = ( x) ( x) with ( x) = 1x 1x (u ) + u ( x) , ( x) = (u ) u ( x) a a 2 2

If u(x) is BV on [a, b], then it has at most a countable infinite number of points on [a, b], where u(x) is discontinuous and all discontinuous points are of first kind; u(x) is Riemannintegralable; u(x) has finite derivative almost everywhere on [a, b]; and its derivative u(x) is Lesbeque-integralable on [a, b]. If u(x) is BV on [a, b], then

d x (u ) =| u '( x) | dx a holds almost everywhere on [a, b].

(17)

12

2.2

Bounded Variation Functions of Two Variables

x

From (17), we can write d (u ) =| u '( x) | dx , hence the total variation (TV) of u(x) can be
a

computed as
(u ) = d (u ) = | u '( x) | dx
a a a a b b x b

(18)

The expression of TV in (18) provides a natural way to extend the concept of BV from the onevariable case to the two-variable case.
Definition A function u(x, y) defined in a compact region R is said to be a function of bounded variation (BV) if it has a finite total variation (TV) which is defined by
R (u ) =
2 2 u x + u y dx dy

(19)
2 2 u x + u y is just the

where u x = u ( x, y ) / x, u y = u ( x, y ) / y. Note that the integrand in (19), i.e.

Euclid norm of the gradient of u(x, y), the TV can also be written as
R (u ) = u ( x, y ) 2 dx dy
R

(20)

3. Image De-Noising by TV Minimization 3.1 Oshers Model for Images

Given a noisy image which is represented by its light intensity u0(x, y) over a bounded open set R in the two-dimensional vector space, we want to obtain a decomposition u0(x, y) = u(x, y) + n(x, y) (21)

where u(x, y) represents the true image and n(x, y) is the noise with zero mean and standard deviation 2 . The de-noising problem at hand is to reconstruct u(x, y) from u0(x, y). The model in (21) has been known as the Osher model [5] since the variational optimization method for image de-noising [2]. In the case of image de-blurring, model (21) is modified to u0(x, y) = (Au)(x, y) + n(x, y) where A is a compact operator on L2(R) to model the de-blurring process [3]. The main difference between the methods proposed in [2][3] with many well known earlier attempts is that here images are regarded as two-variable functions of bounded variation and the (22)

13

de-noising and de-blurring problems are treated by minimizing the total variation of the image subject to some variational constraints derived from the model used and the statistics of the noise component. It is this BV framework that allows analysis and processing of piecewise smooth functions with jumps as seen in virtually all natural and synthetic images and is proven proper for many basic image processing tasks.
3.2 Problem Formulation

The TV minimization problem for image de-noising can ce formulated as

minimize R (u ) =

2 2 u x + u y dx dy

subject to: constraints involving u0 ( x, y )

(23)

It follows from model (21) that

and

u ( x, y ) dx dy = u0 ( x, y ) dx dy
R

(24)

1 2 2 R [u( x, y) u0 ( x, y)] dx dy = 2
3.3 A Solution Procedure

(25)

By defining the Lagragian of the problem (23), where the constraints are given in (24) and (25), as the functional 2 1 2 L(u, 1 , 2 ) = u x + u y + 1 (u u0 ) + 2 (u u0 ) 2 2 dx dy R 2 (26)

and applying the Euler-Lagrange equation (16) to functional (26), it can be concluded that the solution of problem (23) (26) must satisfy the second-order partial differential equation ux 2 2 x u x + u y subject to boundary condition u ( x, y ) = 0 on the boundary = R n (27b) u y + 2 2 y u x + u y 1 2 (u u0 ) = 0

(27a)

14

where n denotes the normal along the boundary of region R, which is a reasonable requirement for images defined on a rectangular region and undergone a symmetric extension along the boundary of R. Another advantage of imposing (27b) is that the Lagrange multiplier 1 can now be dropped from the equation because a solution of ux 2 2 x u x + u y u y + 2 2 y u x + u y (u u0 ) = 0

(28a)

u ( x, y ) = 0 on = R n automatically satisfies

(28b)

(u u0 ) dx dy = 0 , i.e., constraint (24).

In [2], the solution procedure for problem (28) uses a nonlinear parabolic equation with time as an evolution parameter. Numerically this is equivalent to the gradient descent method. In doing so, we need to solve ux u = 2 2 t x u x + u y u y + 2 2 y u x + u y (u u0 ) for t > 0, ( x, y ) R

(29a)

u(x, y, 0) given u ( x, y ) = 0 on = R n 1 2 2 R [u( x, y, t ) u0 ( x, y)] dx dy 2

(29b) (29c)

(29d)

where the initial value in (29b) is typically taken to be the original noisy image u0(x, y). As time t increases, the solution of the (29) converges to its steady state i.e. becomes time invariant thus the limiting solution of (29) satisfies (28), a de-noised version of the image. Notice that the Lagrange multiplier in (29a) is time-dependent and must be evaluated in order to keep the time-evolution of the solution u(x, y, t) going forward. By multiplying (29a) by (u u0), using (29d), integrating by parts over region R and using (29c) , and assuming that the steady state has been reached, we obtain a formula for updating the Lagrange multiplier (t ) as
(u ) u (u ) u 2 2 (t ) = 2 u x + u y 0 x x + 0 y y 2 2 2 2 R ux + u y 2 ux + u y 1 dx dy

(30)

15

3.4

A Difference Scheme for Solving (29), (30)

Given below is a difference scheme proposed in [2] for computing a numerical solution of problem (28) by solving (30) on an evolving time grids. For the sake of notation simplicity, in the difference scheme a square region R = [0, 1] [0, 1] has been assumed. The scheme can be applied to non-square images after some straightforward modifications. xi = ih, y j = jh, for i, j = 0,1,..., N , with Nh = 1 tn = nt ,
n i, j

for n = 0,1,......

u = u ( xi , y j , tn ) ui0, j = u0 ( xi , y j ) For i, j = 0,1,..., N 1, do uin, +1 = uin, j j

x + uin, j t x + h ( x u n ) 2 + (m( y u n , y u n )) 2 1/ 2 + i, j i, j + i, j y + uin, j t y + h ( y u n ) 2 + (m( x u n , x u n )) 2 1/ 2 + i, j i, j + i, j t n ( uin, j u0 (ih, jh) )

with
n n n n u0, j = u1, j , u N , j = u N 1, j , uin,0 = uin,1 , and uin, N = uin, N 1.

x ui , j = (ui 1, j ui , j ) y ui , j = (ui , j 1 ui , j )

sgn a + sgn b m(a, b) = min mod(a, b) = min(| a |,| b |) 2 where n is computed based on (30) as
h = 2 2 i j
n

( u ) + ( u )
x n 2 + i, j

y n 2 + i, j

x x ( + ui0, j )( + uin, j )

( u ) + ( u )
x n 2 + i, j

y n 2 + i, j

( u )( u ) ( u ) + ( u )
y 0 + i, j y n + i, j x n 2 + i, j

y n 2 + i, j

A step size restriction is imposed for numerical stability of the scheme: t c for some constant c h2 (31)

16

3.5

MATLAB Code and Results

Listed below are several MATLAB functions written by Guy Gilboa of UCLA that can be used to demonstrate the Rudin-Osher-Fatemi algorithm [2]. The main routine named demo_tv.m is given in the first m-file which calls for the second and third MATLAB routines. 1. File Name: demo_tv.m
%%% TV demo %% %%% By Guy Gilboa %%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Based on: [ROF92] L. Rudin, S. Osher, E. Fatemi, % "Nonlinear Total Variation based noise removal algorithms", % Physica D 60 259-268,1992. %%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% Simple TV denoising %%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%% I = imread('camera256.tif'); % load image I = double(I(11:138,11:138)); % cut a piece, convert to double % params iter = 80; % dt=0.2; eps=1; %%% Add noise std_n = 20; % Gaussian noise standard deviation In = randn(size(I))*std_n; % White Gaussian noise I0 = I + In; % noisy input image % show original and noisy images close all figure(1); imshow(uint8(I)); title('Original') figure(2); imshow(uint8(I0)); title('Noisy image') % denoise image by using tv for some iterations J = tv(I0,iter); figure(3); imshow(uint8(J)); title('Denoised image') %%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% TV denoising with (scalar) data fidelity term %%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Here we assume the noise vaiance is known. %% Therefore we can compute lambda (instead of fixing %% iter). TV therfore can run automatically quite well. %% The process is slower. J = I0; % params ep_J = 0.01; % minimum mean change in image J lam = 0; J_old = 0; iter = 10; dt = 0.2; eps = 1;

17

var_n = std_n^2; % noise variance i = 0; while (mean(mean(abs(J - J_old))) > ep_J), % iterate until convergence J_old = J; J = tv(J,iter,dt,eps,lam,I0); % scalar lam lam = calc_lam(J,I0,var_n,eps); % update lambda (fidelity term) end % for i figure(4); imshow(uint8(J)); title('Denoised image with lambda')

2. File Name: tv.m

function J = tv(I,iter,dt,ep,lam,I0,C) %% Private function: tv (by Guy Gilboa). %% Total Variation denoising. %% Example: J = tv(I,iter,dt,ep,lam,I0) %% Input: I - image (double array gray level 1-256), %% iter - num of iterations, %% dt - time step [0.2], %% ep - epsilon (of gradient regularization) [1], %% lam - fidelity term lambda [0], %% I0 - input (noisy) image [I0=I] %% (default values are in []) %% Output: evolved image if ~exist('ep') ep = 1; end if ~exist('dt') dt = ep/5; % dt below the CFL bound end if ~exist('lam') lam = 0; end if ~exist('I0') I0 = I; end if ~exist('C') C = 0; end [ny,nx] = size(I); ep2 = ep^2; for i=1:iter, %% do iterations % estimate derivatives I_x = (I(:,[2:nx nx])-I(:,[1 1:nx-1]))/2; I_y = (I([2:ny ny],:)-I([1 1:ny-1],:))/2; I_xx = I(:,[2:nx nx])+I(:,[1 1:nx-1])-2*I; I_yy = I([2:ny ny],:)+I([1 1:ny-1],:)-2*I; Dp = I([2:ny ny],[2:nx nx])+I([1 1:ny-1],[1 1:nx-1]); Dm = I([1 1:ny-1],[2:nx nx])+I([2:ny ny],[1 1:nx-1]);

18

I_xy = (Dp-Dm)/4; % compute flow Num = I_xx.*(ep2+I_y.^2)-2*I_x.*I_y.*I_xy+I_yy.*(ep2+I_x.^2); Den = (ep2+I_x.^2+I_y.^2).^(3/2); I_t = Num./Den + lam.*(I0-I+C); I = I+dt*I_t; %% evolve image by dt end % for i %% return image %J = I*Imean/mean(mean(I)); % normalize to original mean J = I;

3. File Name: calc_lam.m

function lam = calc_lam(I,I0,var_n,ep) %% private function: calc_lam (by Guy Gilboa). %% calculate scalar fidelity term of the Total-Variation %% input: current image, original noisy image, noise variance, epsilon %% output: lambda %% example: lam=calc_lam(I,I0,var_n,ep) if ~exist('ep') % good for 256 gray-level ep = 1; end [ny,nx] = size(I); ep2 = ep^2; % estimate derivatives I_x = (I(:,[2:nx nx])-I(:,[1 1:nx-1]))/2; I_y = (I([2:ny ny],:)-I([1 1:ny-1],:))/2; I_xx = I(:,[2:nx nx])+I(:,[1 1:nx-1])-2*I; I_yy = I([2:ny ny],:)+I([1 1:ny-1],:)-2*I; Dp = I([2:ny ny],[2:nx nx])+I([1 1:ny-1],[1 1:nx-1]); Dm = I([1 1:ny-1],[2:nx nx])+I([2:ny ny],[1 1:nx-1]); I_xy = (Dp-Dm)/4; % compute numerator and denomenator Num = I_xx.*(ep2+I_y.^2)-2*I_x.*I_y.*I_xy+I_yy.*(ep2+I_x.^2); Den = (ep2+I_x.^2+I_y.^2).^(3/2); Div = Num./Den; %%% fidelity term lam = mean(mean(Div.*(I-I0)))./var_n;

The above MATLAB code was tested by applying it to a small part of a test image named cameraman. The image has a size of 128 by 128 and was taken from the tif file camera256(11:138,11:138) and was contaminated by a Gaissian white noise with zero mean and standard deviation = 20. The results in term of the original, noisy, and de-noised images are depicted in the following plots. The peak-signal-to-noise ratio (PSNR) was improved from 22.0647 dB to 29.5268 dB.

19

20

For comparison purposes, Donohos well-known wavelet de-noising algorithm using Daubechies wavelets of length 2, 4, and 6 were also applied to the same test image. The best result was achieved by the D2 (Haar) wavelet that yielded PSNR = 27.0670dB. The plot below displays the de-noised image by wavelet (on the left) and de-noised image by TV (on the right) minimization. It is observed that the TV-minimization based algorithm yields improved de-noising results in terms of PSNR as well as visual examination.

4. Image De-Blurring by TV Minimization

In [3], the image de-blurring problem is investigated in a similar variational optimization framework. Here the image model is given by (22), i.e., u0(x, y) = (Au)(x, y) + n(x, y) for (x, y) in R (32)

where A is typically a convolutional type integral operator that can be used for modeling several common blurring processes such as averaging, Gaussian low-pass, Laplacian of Gaussian, and motion blurs. The variational optimization problem in this case can be formulated as
minimize R (u ) = subject to:
R 2 2 u x + u y dx dy 0

u ( x, y) dx dy = u ( x, y) dx dy [ Au ( x, y) u ( x, y)] dx dy =
R 2 R 0

(33)

The Euler-Lagrange equation corresponding to problem (33) is given by

21

ux 2 2 x u x + u y

u y + 2 2 y u x + u y

A ( Au u0 ) = 0

(34a)

u ( x, y ) = 0 on = R n

(34b)

where A* denotes the adjoint operator for operator A. In [3], it is proposed that (34) be solved by calculating the steady-state solution of the parabolic partial differential equation ux u = 2 2 t x u x + u y u y + 2 2 y u x + u y A ( Au u0 ) for t > 0, ( x, y ) R

(35a)

u ( x, y ) = 0 on = R n

(35b)

where, using a technique similar to that for the de-noising case, the time-varying Lagrange multiplier = (t ) is found to be ux R x u 2 + u 2 x y u y + 2 2 y u x + u y
R

(t ) =

A ( Au u0 ) dx dy 2 A ( Au u0 ) dxdy

(36)

22