NATIONAL UNIVERSITY OF SINGAPORE

Department of Mathematics
Semester I (2016/2017) MA5243 Advanced Mathematical Programming
Assignment 2

1. Give a survey about quasi-Newton methods for solving a) nonlinear equations; and b)
nonlinear optimization problems. In this survey, you need to state (1) the history of the
development of quasi-Newton methods; (2) main convergence theorems, in particular on
the BFGS method (no proof is needed); and (3) the source of original references (papers).
The purpose of this survey is to give you an opportunity to learn how to conduct research
on a topic that you are not familiar with. The total length of this survey is limited up to
ten pages.
2. Suppose that f : <n < is a twice continuously differentiable convex function. Consider the following modified Newtons method for minimizing f (x):
Algorithm 1.1 (Newtons Method)
Step 0. Given x0 <n , (0, 1), , (0, 1/2). k := 0.
Step 1. Solve


f (xk ) + 2 f (xk ) + ckf (xk )kI d = 0 with c > 0

approximately to get an approximate solution dk such that


kf (xk ) + 2 f (xk ) + ckf (xk )kI dk k k kf (xk )k ,

where k := min{, kf (xk )k}. If dk does not satisfy
hf (xk ), dk i k kdk k2 ,
let dk = f (xk ).
Step 2. Let mk be the smallest nonnegative integer m such that
f (xk + m dk ) f (xk ) m f (xk )T dk .
Set tk := mk and xk+1 := xk + tk dk .
Step 3. Replace k by k + 1 and go to Step 1.

(1)

2
(i) Show that Algorithm 1.1 is well defined. That is, to show that if f (xk ) 6= 0 for
each k 0, then Algorithm 1.1 will generate an infinite sequence {xk }. [Hint: You
may first consider the case that = 0]
(ii) Show that any accumulation point of {xk } is a solution to min f (x).
(iii) Suppose that x (if exists) is an accumulation point of {xk } and that 2 f (x ) is positive definite. Then, the whole sequence {xk } converges to x superlinearly (quadratically if f 2 () is locally Lipschitz continuous at x ).
[Deadline: November 1, 2016. ]