Introduction to Lemkes Method (Scheme I)

This algorithm works with a principal rearrangement of the ALCP. q = q0 q = and M 0 d T d M .

Instead of using , , and , we use q0, z0 and w0, respectively. Thus we have w0 = q0 + 0 z0 d Tz 0, z0 0, z0w0 = 0 w = q + dz0 + M z 0, z 0, z Tw = 0.

We assume that the constant column q 0 and the covering vector d > 0. Then the smallest positive z 0 such that q + dz0 0 for all z0 z 0 is given by z 0 = max{qi/di}.
i

Recall the theorem stating that the ALCP always has a solution. The algorithm we are about to study proves this constructively under the assumption that the system of equations w0 w is nondegenerate. = q0 q + 0 d T d M z0 z

Statement of Lemkes Method (Scheme I) Algorithm (Lemke, Scheme I Augmented LCP)

). If q 0, then stop: z = 0 solves Step 0. Initialization. Input ( q, M (q, M ). Otherwise, let z 0 = maxi{qi/di}. Let wr denote the (unique, by the nondegeneracy assumption) component of w that equals zero when z0 = z 0. Pivot wr , z0 . Choose the driving variable to be the complement of wr , namely zr . Step 1. Determination of the blocking variable. Use the minimum ratio test to nd the basic variable that blocks the increase of the driving variable. Stop if w0 is the blocking variable. (Interpret this outcome, if possible.)

Step 2. Pivoting. The driving variable is blocked. If z0 is the blocking variable, then pivot z0, driving variable and stop. A solution to (q, M ) is at hand. If some other variable blocks the driving variable, then pivot blocking variable, driving variable . Return to Step 1 using the complement of the most recent blocking variable as the new driving variable.

Justication of Lemkes Method (Scheme I) Existence of pivot elements

After the rst pivot in Lemkes algorithm, all basic variables (and only basic variables) are eligible to block the driving variable. But why does blocking always occur?

Lemma.

denote a square matrix of the form Let M = M aT B c

in which a 0, < 0, and B is nonsingular. Then the pivotal obtained by using B as the pivot block has at least transform of M one negative entry in its last column.
Proof. The indicated pivotal transform is

aT B 1 aTB 1 c B 1 B 1 c If B 1 c 0, then the hypotheses imply aTB 1 c < 0.

Theorem.

In Lemkes Algorithm, when w0 and z0 are basic, the column of the driving variable contains at least one negative entry.

Proof. Consider the w0 row and the set of rows corresponding to basic

z -variables (including the articial variable, z0). Let be the index set of z -variables (including z0) that are currently basic. Let be the index set of the w-variables that are currently nonbasic. These index sets have the same cardinality, but dier by just one element, r: will contain the index 0, and will contain the index of the most recent blocking variable (which is now nonbasic). The of the lemma corresponds to M . The remaining submatrix B in M corresponds to M r of the current driving variable. column of M This is illustrated in the next panel.

aT B c

1 z0

aT B 1 aTB 1c B 1 B 1 c z zr z

T T w0 q0 0 d drT d

M r M w q d M M r M w q d M
T , aT = 0 d

= drT

B = d M

r. c=M

Choosing the covering vector

To initialize the algorithm, it is enough to choose the column vector d so as to make the n 2 matrix [d, q ] have lexicographically nonnegative rows. Nevertheless, it is customary to let d = e, the vector of ones. The choice of the covering vector can aect the number of iterations of the algorithm.

Justication of Lemkes Method (Continued) Almost complementary paths

Making z0 z 0 and z = 0 yields a set of feasible solutions to the ). ALCP ( q, M These points constitute what is called the primary ray. Points along the primary ray satisfy the conditions z0w0 > 0 and ziwi = 0 for all i = 0. Thus, they are almost complementary in the sense that complementary slackness holds for all but one index. In between almost complementary extreme points, we have almost complementary edges.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Theorem. When applied to a nondegenerate instance of the Aug-

), Lemkes Algorithm terminates in nitely many mented LCP ( q, M steps.

Introduction to Lemkes MethodNew Streamlined Format

w = q + dz0 + M z 0, z0 0, z 0, z Tw = 0. We denote this system by (q, d, M ). A solution of (q, d, M ) with z0 = 0 furnishes a solution to the original LCP (q, M ). The version of Lemkes Scheme I for (q, d, M ) is a simple variant of ). the one for ( q, M

Secondary rays

), we have to Without the constraint w0 0 that is used in ( q, M allow for the possibility that the column of a driving variable may be nonnegative and hence that the variable may be unblocked. Such a situation corresponds to the case where q0 is large and w0 is the blocking variable in the earlier version of Lemkes Algorithm for the full augmented LCP. In the event that no blocking variable is found when a driving variable is increased, a secondary ray is generated. A secondary ray is also an almost complementary edge of FEA(q, d, M ), but it is necessarily unbounded.

Algorithm (LemkeStreamlined Version).

Step 0. Initialization. Input (q, d, M ). If q 0, then stop: z = 0 solves (q, M ). Otherwise, let z 0 = maxi {qi /di }. Let wr denote the (unique, by the nondegeneracy assumption) component of w that equals zero 0 . Pivot wr , z0 . Choose zr as the driving variable. when z0 = z Step 1. Determination of the blocking variable (if any). If the driving variable is blocked, use the minimum ratio test to determine the basic variable that blocks the increase of the driving variable. If the driving variable is unblocked, then stop. (Interpret this outcome, if possible.) Step 2. Pivoting. The driving variable is blocked. If z0 blocks the driving variable, then pivot z0 , driving variable and stop. A solution to (q, M ) is at hand. If some other variable blocks the driving variable, then pivot blocking variable, driving variable . Return to Step 1 using the complement of the most recent blocking variable as the new driving variable.

Miscellaneous Comments on Lemkes Algorithm

(a) The n-vector e whose components are all 1 is often used as the default covering vector. Other choices are possible, however. The choice of the covering vector can drastically aect the outcome of the procedure. (b) After the initial pivot which makes the articial variable z0 basic, there is one (and only one) complementary pair of nonbasic variables, initially, wr and zr are such a pair. In general, the nonbasic complementary variables make up the nonbasic pair. (c) When z0 z 0, all the basic variables are nonnegative and are eligible to block the driving variable. In general, after a pivot occurs, the new driving variable is the complement of the variable that just became nonbasic.

(d) One can implement the algorithm in a revised simplex method fashion. Only two columns are needed to execute the minimum ratio test in Step 1. These are the updated constant column and the updated column of the current driving variable. Thus, if the current basis is known, these updated columns can be determined by solving two systems of linear equations. (e) There are certain classes of matrices for which the nonexistence of a blocking variable implies the infeasibility of the given LCP (q, M ).

Example Using Lemkes Algorithm

1 z0 z1 w1 3 w2 6 w3 1

z2

z3

1 0 1 2 1 2 0 2 1 1 1 0

1 w1 z1 z2 z3 z0 w2 w3 3 9 2 1 0 1 2 1 1 1 2 1 4 2 2

1 w1 w3 z2 z3 z0 3 w2 13 z1 2 1 0 3 2 1 1 1 2 5 8 2 2

Example Continued

1 w1 w3 z0 w2 z3

z2

z1

1 0 1 1 1 5 1 2 3 4 1 1 1 1 1 2 2 2 1 w2 w3 z2 z1

z0 w1 z3

1 0 5 1 3 1 2

1 1 2 3 1 1 2 2

1 4
3 2

1 w2 w3 z2 w1 z3

z0 z1 1 1 1

1 0 1 1 2 1 1 3 1 3 1 0 2 2

Thus, we have found the solution ( z1, z 2, z 3) = (0, 1, 3).

Some Theory Behind Lemkes Algorithm Theorem.

When applied to a nondegenerate instance of (q, d, M ), Lemkes Algorithm (Streamlined) terminates after nitely many steps with either a secondary ray or else a complementary feasible solution of (q, d, M ) and hence with a solution of (q, M ).

Proof. This is just like the proof of niteness in the unstreamlined case.

Termination on a secondary ray corresponds to the situation where the latter algorithm terminates with w0 as the blocking variable.

Consequences of termination on a secondary ray Theorem. If Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then M reverses the sign of some nonzero nonnegative vector; that is )i 0 i = 1, . . . , n z i (M z
for some nonzero nonnegative vector z .

Proof. We are working with the system

w = q + dz0 + M z, w 0 , z0 0 , z 0 .

Suppose the algorithm reaches an almost-complementary extreme point (basic ) at which we have z 0 > 0 and w i z i = 0 for all i > 0. feasible solution) (w, z 0 , z If the driving variable is unblocked, we have termination on a secondary ray. ) of Hence there is a nonzero solution (w, z 0, z w = dz0 + M z, w 0 , z0 0 , z 0 .

Moreover, we have (w + w )i ( z + z )i = 0 for all i = 1, . . . , n and all 0. z 0 , z ) is zero). But z 0 > 0 implies Now z = 0, for otherwise z 0 > 0 (else (w, w > 0 and hence that the secondary ray is the primary ray, which is impossible. For i = 1, . . . , n and all 0, we have i + w i z i + w i z i + 2 w i z i = 0 w i z which implies w i z i = w i z i = w i z i = w i z i = 0 Thus for each i = 1, . . . n, i = z i di z 0 + z i (M z )i . 0=z i w Since z i di z 0 0, we have the asserted result: there exists a nonnegative nonzero vector z such that )i 0 for all i = 1, . . . , n. z i (M z

More Theory of Lemkes Method

Lemkes Algorithm will solve any nondegenerate LCP (q, M ) such that M E.
Theorem.

In particular, Lemkes Algorithm will solve any nondegenerate LCP (q, M ) such that M P or M SCP (strictly copositive).
Proof. It is impossible for a strictly semimonotone matrix to reverse

the sign of a nonzero, nonnegative vector. Recall that a matrix M Rnn is said to be semimonotone if [0 = x 0] [xk > 0, (M x)k 0 for some k ] The class of semimonotone matrices is denoted E0.

Theorem.

Let M Rnn E0. If Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then SOL(0, M ) = {0}.

Proof. From the proof of the theorem on consequences of termination

on a ray, we know that there exists a nonzero nonnegative vector ( z0, z ) such that w = dz 0 + M z 0 and z Tw = 0. Since M E0 we cannot have z 0 > 0, for otherwise, 0 = w = d z 0 + M z where = supp( z ). Hence M z < 0. But now it is clear that z SOL(0, M ).

Results for copositive matrices Corollary.

If M CP and Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then the vector y = z / z satises: (i) 0 = y SOL(0, M ), (ii) q Ty < 0.

Proof. Since SOL(0, M ) is clearly a closed cone, it is clear that (i)

holds. As for (ii), note that for all 0 we have + M ( z + z )) ( z + z )T(q + dz 0). 0 = ( z + z )T(q + dz Dividing through by z + z and letting we obtain y T(q + dz 0) 0. Since y Tdz 0 > 0 we obtain (ii).
Theorem.

If M is copositive and q (SOL(0, M )), then Lemkes Algorithm will compute a solution of the LCP (q, M ) if the problem is nondegenerate.

Proof. This is just a corollary of the corollary.

Constructive proof that CP+ Q0 Theorem. If M CP+ and Lemkes Algorithm applied to a nonde-

generate instance of the problem (q, d, M ) terminates on a secondary ray, then the LCP (q, M ) is infeasible.
Proof. If the algorithm terminates on a secondary ray, we have

0=z Tw =z Tdz 0 + z TM z . Since M is copositive, both terms on the right are nonnegative, hence z = 0. both are zero. But since M CP+, we have (M + M T) We also know that z 0 = 0, hence M z 0 and M Tz 0. The latter is equivalent to z TM 0. Moreover z Tq < 0; thus z 0, z TM 0, z Tq < 0.

These conditions imply that FEA(q, M ) = .

Can Lemkes Algorithm nd global minima of nonconvex QPs? Example. Consider the quadratic program
minimize subject to
1 2 x1 1 2 1 2 1 2 x2 2 x1 + 2 x2

2 x1 + x2 6 x1 + 4x2 6 x1 0 , x 2 0

The problem has a nonempty compact constraint set, hence a global optimum. The KKT conditions give the LCP (q, M ) where 1 1 0 2 1 2 0 1 and M = q= 6 2 1 6 1 4

2 1 1 0 0

4 0 0

4 arbitrarily. Then scale it so that d2 = 1. Choose the covering vector d R++

1 y1 y2 x3 x4
1 2 1 2

z0

x1

x2 0 1

y3

y4

d1 1 1 0

2 1 1 0 0 4 0 0

6 d3 2 1 6 d4 1 4

The pivot to perform is y2 , z0 . It leads to the tableau

1 y1 y2 x3 x4
1 2 1 2

z0

x1

x2 0 1

y3

y4

d1 1 1 0

2 1 1 0 0 4 0 0

6 d3 2 1 6 d4 1 4

The pivot to perform is y2 , z0 . It leads to the tableau 1 y1 z0 x3 x4

1 2 (12 1 2 (12 1 2 (1

y2

x1

x2 d1 1 4 d4

y3

y4

+ d1 )
1 2

d1 1 1 0 1

2 d1 1 4d1 1 d3 d4 4 4d3 4d4

+ d3 ) d3 2 1 d3 + d4 ) d4

The pivot to perform is z0 , x2 . It leads to the tableau

1 y1 z0 x3 x4
1 2 (12 1 2 (12 1 2 (1

y2

x1

x2 d1 1 4 d4

y3

y4

+ d1 )
1 2

d1 1 1 0 1

2 d1 1 4d1 1 d3 d4 4 4d3 4d4

+ d3 ) d3 2 1 d3 + d4 ) d4

The pivot to perform is z0 , x2 . It leads to the tableau 1 y1 x2 x3 x4

1 2 1 2 11 2

y2 0 1 4

x1 1 0 1

z0 d1 1 4 + d4

y3 2 1 4

y4 1 4 16

1 4

1 2 1 + d3

1 y1 x2 x3 x4
1 2 1 2 11 2

y2 0 1 4

x1 1 0 1

z0 d1 1 4 + d4

y3 2 1 4

y4 1 4 16

1 4

1 2 1 + d3

The solution of the KKT conditions found is (x1 , x2 , y3 , y4 ) = (0, 1 2 , 0, 0). 1 The point (x1 , x2 ) = (0, 2 ) is a local minimum. The global minimum is (x1 , x2 ) = (3, 0). It cannot be reached by Lemkes Algorithm, no matter what covering vector is used.