An Augmented Lagrange Multiplier

Based Method for Mixed Integer

B. K. Kannan Discrete Continuous Optimization
Graduate Assistant.
Department ot Mechanical Engineering,
University of Toledo, and Its Applications to Mechanical
Toledo, OH 43606-3390
Design
S. N. Kramer
Professor. An algorithm for solving nonlinear optimization problems involving discrete, integer,
Department of Mechanical Engineering, zero-one, and continuous variables is presented. The augmented Lagrange multiplier
University of Toledo, method combined with Powell's method and Fletcher and Reeves Conjugate Gradient
Toledo, OH 43606-3390 method are used to solve the optimization problem where penalties are imposed on
Fellow ASME. the constraints for integer/discrete violations. The use of zero-one variables as a
tool for conceptual design optimization is also described with an example. Several
case studies have been presented to illustrate the practical use of this algorithm. The
results obtained are compared with those obtained by the Branch and Bound al-
gorithm. Also, a comparison is made between the use of Powell's method (zeroth
order) and the Conjugate Gradient method (first order) in the solution of these
mixed variable optimization problems.

Introduction
In nonlinear optimization problems, the design variables are
generally assumed to be continuous quantities. In many cases,
this is a valid assumption. However, very often integer vari-
ables and discrete variables occur in engineering design. Ex-
amples of these types of variables are the number of gear teeth,
the value of a gear module, the specifications of rolled steel
members, the number of bars in a truss, nominal sizes of shafts
and other parts, the number of coils of a spring, the number
of screws or rivets in a structure, etc. The presence of discrete
and integer variables along with continuous variables adds to
the complexity of the optimization problem. Since such situ-
ations often occur in engineering design, an efficient method
to solve this type of problems is of considerable use in optimal
mechanical design.
Also, conceptual design optimization problems can be solved
using a type of integer variables called "zero-one variables"
(commonly called "binary" variables) which can take on values
of either zero or one as the name suggests. Whenever there is
a choice to be made from a list of alternatives, zero-one vari-
ables can be used to choose the best alternative. More details
on the use of zero-one variables are discussed later.
In the linear optimization domain, solution methods have
been developed for integer and discrete problems. However,
in nonlinear optimization problems, no such parallel can be
drawn. An initial approach is to assume all variables to be
continuous and then round the result up or down to the nearest
integer or discrete value. However, this approach often leads
to incorrect results. Fu, Fenton, and Cleghorn (1989) have
given an example as to why this "common sense" approach
cannot be followed. In Fig. 1, the contour map of an uncon-
strained optimization problem is given, showing the continuous
and discrete optimum values. The continuous optimum so-
lution, indicated by point "A", is at (1.55, 1.4) with an ob-
jective function value of 0.0. The optimum solution to the
discrete problem with discrete values as multiples of 0.5 is
indicated by point "B" at (0.5, 0.5) where the value of the
objective function is 18.0. If the continuous optimum were
rounded to (1.5, 1.5), the objective function value would be
40.0 and the amount of error would be quite high.
In the field of nonlinear mixed discrete integer continuous
programming (MDICP), a limited amount of research has been
reported. Gisvold and Moe (1972) developed a method for
nonlinear mixed integer programming problems by modifying
the penalty function approach. Here, discrete requirements are

Contributed by the Design Automation Committee for publication in the

JOURNAL OF MECHANICAL DESIGN. Manuscript received Feb. 1993; revised June
1993. Associate Technical Editor: S. Azarm. Fig. 1 Contours of an unconstrained discrete optimization problem

Journal of Mechanical Design JUNE 1994, Vol. 1 1 6 / 4 0 5

Copyright © 1994 by ASME
Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms
 treated as explicit constraints and an objective function is con- can take on only a particular set of values, the selection of
structed penalizing deviations from the discrete values. The which depends entirely on the design variable. This type of
difficulty with this method is that the penalty parameters are variable may be either equally or unequally spaced. In addition,
difficult to set a priori and the resulting solution depends too there is another type of variable called the zero-one variable,
much on the initial penalty parameter chosen. which takes on values of zero or one. This variable can be
For mixed discrete programming problems, Cha and Mayne used as a tool in conceptual design optimization, wherein a
(1988) suggested a promising method of coupling a sequential single option must be chosen from a multiple set of options.
quadratic programming (SQP) method with a local search The variable associated with the option selected is given a value
strategy. A continuous SQP algorithm moves the search quickly of "one" while the others are given values of "zero". Discrete
in the vicinity of the optimum, and a local search based on and zero-one variables are special cases of integer variables.
the gradient information of the objective finds the best discrete In the method presented here, the integer/discrete violations
solution in this neighborhood. Fox and Liebman (1981) used are expressed as explicit equality constraints which are added
the "complex" algorithm to solve integer/discrete program- to the constraints presented in the problem. These constraints
ming problems. Numerical tests on fifteen problems showed are expressed as:
that this method gave better results than both the discrete
penalty algorithm and an algorithm where the continuous min- qj(l-qj) =0.0 (9)
imum was rounded. However, the disadvantage of using this where qj is the normalized integer/discrete variable which can
method is that convergence is not assured and even if it con- be expressed as:
verges, there is no guarantee that the solution is the global <10)
optimum point.
Gupta and Ravindran (1983) described a Branch and Bound
strategy for mixed discrete integer programming problems.
*-<^t
where s'j and sj are the nearest feasible upper and lower standard
Here, the computational effort was affected by rules for se- values. Clearly, from Eq. (9), qj must be equal to zero or one
lecting nodes for expansion and variables for branching. Re- in order to satisfy the integer/discrete variable constraint. This
cently, Sandgren (1988) proposed an algorithm which used the is similar to what was done by Fu, Fenton, and Cleghorn
Branch and Bound method in conjunction with an exterior (1989). In that work, Eq. (9), instead of being expressed as an
penalty function method and quadratic programming method. equality constraint, was taken as a penalty term and directly
Several engineering design problems were solved in that work included in the objective function.
to show the effectiveness of the method. The same engineering It is important to note that the equality constraints listed in
design problems used there have been solved in this paper and Eq. (9) are nondifferentiable with respect to X and the shape
the results here are compared with those obtained by Sandgren. of these contours is a series of parabolic "bumps" having
The work presented here is intended to show a general method zeros at the nodal points. This is discussed later in the paper.
for solving mixed discrete, integer, zero-one, continuous op- In the case of integer variables, s'j and sj- are the nearest
timization problems using the augmented Lagrange multiplier consecutive integers bounding the variable's current value.
method combined with Powell's zeroth order method (1964) Thus, the denominator is always one in the case of integer
and alternatively Fletcher and Reeves Conjugate Gradient variables. In the case of zero-one variables, the upper and
method (1963). The power and versatility of this method are lower bounds are one and zero. So, Eq. (10) becomes qj = Xj.
For unequally spaced discrete variables, the discrete variables
illustrated with the solution of several engineering design prob-
lems (Kannan, 1993). are stored as a vector and the two values of the s/s nearest to
the variable's value are chosen as the upper and lower bounds.
The objective function along with the constraints (original
Mathematical Description and the imposed constraints due to integer/discrete violations)
Any engineering design optimization problem, in general, are solved using augmented Lagrange multiplier method com-
can be expressed as: bined with a Powell's zeroth order method or alternatively a
Minimize: F(X) (1) first order Fletcher and Reeves conjugate gradient method.
These are well known methods of optimization and can be
Subject to: found in most optimization texts. For example, Vanderplatts
g/X)<0.0 j=l,2,3, ...,t (2) (1986), Reklaitis, Ravindran, and Ragsdell (1983), Beveridge
and Schecter (1970).
hkQL) = 0.0 k=\, 2, 3, . . . , m (3) The reason for using Powell's method or Conjugate Gradient
and X;<x;<Af (4) method in place of Quasi-Newton methods like DFP or BFGS
X=[xux2,x3, ...,x„]T is that computational efficiency would be lost due to the need
for calculating the Hessian approximation. This is especially
The following are the sets of values that different types of difficult for large size problems.
variables can assume:
Xj = {0, 1, 2 ...) ... for integer variables (5)
x, = (0, 1) ... for zero-one variables Zero-One Variables
(6)
Zero-one variables (Sandgren, 1988) represent a choice in
Xi=(xn, xi2, xa ...) . . . for discrete variables (7) the decision tree of design variables. By this way, we can
investigate different qualitative alternatives. For example, these
*i€(-°°, °°) ••• for continuous variables (8) variables can be used to decide on the material needed for a
In the above formulation, FQL) is the objective function to be particular component from a list of alternative materials. A
minimized. The design variables, xh may take on different sets zero-one variable is assigned to each of the alternatives and
of values depending on their type. Continuous variables may the variable corresponding to the selected material is assigned
take on an infinite number of values encompassing the whole a value of "one" while the others are assigned values of "zero."
of the positive and negative domain. Integer variables may These zero-one variables must sum to one so that only one of
take on only positive integer values. In rare cases where integer the variables can attain the value of one. Mathematically, the
variables can take on negative values, a suitable adjustment optimization problem involving zero-one variables, x,, can be
(shift or linear transformation) can be made. Discrete variables expressed as:

4 0 6 / V o l . 116, JUNE 1994 Transactions of the ASME

Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms

 Minimize: F(X) = XiFl(X) + x2F2(X) + ... + xrFr(X) (11)
Subject to: xt+x2 + x3 +.. .+xr=l (12)
xigu(X)sO /=1, r;j=l,...,t (13)
x,hik(X) = 0 / = 1 , . . . , > ; k=l, ..., m
(14)
Equation (13) can be alternatively written as:
gu(X)<0 + H(l-x,) (13a)
The advantage of this alternative is linearity. However, in this
paper this form is not used.
The constraints hik and gy are the equality and inequality
constraints for the /th design option. The advantage of being
able to optimize different design alternatives simultaneously o I a
in an optimization problem outweighs the disadvantages due Fig. 2 Contour plot of Rosenbrock's function
to the inclusion of the additional design variables and con-
straints.

*,<X) = max U-(X), (20)

2r,
Augmented Lagrange Multiplier Method
We now minimize A(X, X, rp) as a pseudo-objective function.
The classical approaches to solving constrained optimization After each unconstrained minimization, the rp's are increased
problems by the use of penalty functions, such as exterior and by multiplying by a suitable factor. The X/s are updated as
interior penalty functions, can be significantly improved by follows.
including Lagrange multipliers. Indeed, Powell (1981) notes For the inequality constraints:
that the use of Sequential Unconstrained Minimization Tech-
niques (SUMT) which do not include Lagrange multipliers is 2v
obsolete as a practical optimization tool. The Augmented La- \ r i ^ + 2rp max «/(X), fory'=l, (21)
grange Multiplier Method, also called Primal-Dual Method
2r„
uses Lagrange multipliers along with penalty functions and For the equality constraints:
hence provides an efficient and reliable tool for design opti- \"kX] = \pk+l + 2rphk{X) for k=l, ... m (22)
mization. The advantages of this method are: (1) the de-
pendency of the algorithm on the choice of penalty parameters Equation (19) has continuous first derivatives with respect to
is greatly reduced; (2) The Lagrange multipliers can be started X, but discontinuous second derivatives at gj(X) = -\j/2rp.
at any arbitrary value (normally zero or one) and updated by Thus, second order methods should be used with caution when
a specific recursive relationship; (3) the optimum is reached solving the unconstrained minimization subproblem.
without the need for the penalty parameter, rp, to be increased
to infinity; (4) Attainment of exact boundary values of gy(X) Illustrative Example
= 0 and hk(X) = 0 are possible; and (5) the starting point Rosenbrock's Test Function. In order to demonstrate the
may be either feasible or infeasible. augmented Lagrange multiplier method algorithm for discrete
The Augmented Lagrange Multiplier method for general variables, consider Rosenbrock's test function but with the
optimization problems (Vanderplatts, 1986) involving equality independent variables restricted to discrete values.
and inequality constraints given by Eqs. (1) to (3) are restated The objective function to be minimized is:
here for convenience.
F(X)=l00(x2~x2l)2 + (\-xl)2 (23)
Minimize: F(X) (15)
Let us specify that variables x} and x2 must be discrete integer
Subject to: §/(X)sO j=\, ... t (16) multiples of 0.55. A contour plot of the function along with
hk(X) = 0 k=l, ... m (17) the discrete points are shown in Fig. 2. The continuous opti-
mum solution for this problem is X* = [1, l ] r with the ob-
The augmented Lagrangian is written as: jective function value F(X*) = 0 shown as point A in Fig. 2.
To find the discrete optimum solution for the above prob-
A(X, X, Z, rp)=F(X)+Yl[\j(gj(X) + ZJ)+rp(gj(X) + Z])2] lem, we proceed as follows. Since there are two discrete vari-
y= i ables, two constraints must be imposed for discreteness
violations. They are given as follows:
+ ^]{\k+lhka) + rP[hk(X)]2) (18)
hk(X) = qk(l-qk)=Q.Q k=l,2 (24)
Here, the X/s are the Lagrange multipliers and the rps are the where
penalty parameters. In order to create the desired augmented
Lagrangian, it has been necessary to add " m " nonnegative xk - i n t xk
Qk = (25)
slack variables, Z2. However the above equation can be shown "0.55 '""\0.55j
to be mathematically equivalent to:
where int(r) gives the integer value of T. The augmented
2 Lagrangian is written as:
A(X, X, rp)=F(X)+^Ti [\jtj(X) + rpltj(X)} ]
j=i
A(X, X, r„) = 100(x 2 -x?) 2 + (1-Xl)2
2
+ J]i^k+MX) + rp[hk(X)] } (19) 2
+ J](\khk(X) + rp[hk(X)]2) (26)
where

Journal of Mechanical Design JUNE 1994, Vol. 116/407

Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms

 Table 1 Gear train design

Type of Starting values Optimum Optimum

Design Variable of (he Solution Solution
Variables (Disc/InVCont) variables [Kantian ' TSandgren]
& Kramer]
*1 ( Td ) Integer 18 13 18
*2 ( n ) Integer 15 15 22
*3 ( Ta ) Integer 33 33 45
*4 ( Tf ) Integer 41 41 60

Fig. 3 Compound gear train

F(X) 3.0556e-03 2.1246e-08 5.7123e-06
Gear Ratio 0.1996 0.1441 0.1466
% Error 38.3 % 0.11% 1.65%
where hk(K) are given by Eq. (24). Now, this augmented La-
grangian function is optimized using either a zeroth order
method or a first order method to locate the discrete optimum.
The problem was solved using Powell's zeroth order method Table 2 Effect of varying the starting values
and again using the Fletcher and Reeves first order method. Starting Values Result
The starting penalty parameter value was chosen as rp — 0.001 Trial x
l -<2 *3 xA *1 "2 *3 *4 F(X)
and was updated after each zeroth order search by multiplying
1 20 15 33 41 12 16 31 43 5.920e-08
by a factor of 1.1. The starting X's were taken as zero and
2 20 10 30 40 17 13 45 33 2.063e-05
updated as given by Eq. (22).
3 10 10 10 10 15 15 60 24 1.433e-04
The optimum result obtained was X* = [1.65, 2.75] with 4 25 15 25 35 12 12 26 37 2.926e-05
objective function value F(X*) = 0.4981 (see point B on Fig. 20 20 30 40 12 14 30 40 1.831e-05
5
2). In comparison, the discrete optimum obtained by using the 6 20 12 20 30 12 13 29 37 1.226e-06
Branch and Bound method was X* = [1.1, 1.1] with the
objective function value F(X*) = 1.2200 (see point C on Fig.
2). Clearly, the method presented here arrived at a better so-
lution to this problem. r'h K L
JL r '•
It is important to note that the solutions obtained may not
be "optimal" in a strict mathematical sense because optimality
has not been proved. It can be said that the results found are
the best ones produced by this method with the stopping criteria
used. When comparing the results from the proposed method
with those obtained using other methods, the results found Fig. 4 Center and end section of pressure vessel

here are better.

The choice of the initial penalty parameter was found not
to affect the final result, as compared to the large effect this
parameter had in many classical penalty function methods. method with Powell's method. The starting value for rp used
Also, the solution did not differ using Powell's method versus in the problem was 0.0001 and the multiplication factor used
Fletcher and Reeves method. However, since finite difference was 1.2. The starting values of Lagrange multipliers used was
methods were used to find the gradients in the first order zero. The starting values for the variables and the optimum
search, difficulties were encountered before arriving at the solution obtained are shown in Table 1 below.
solution and the flexibility in the choice of penalty parameters As can be seen from Table 1, the optimum found by Aug-
was somewhat reduced when using Fletcher and Reeves method. mented Lagrange Multiplier based method is a better result
than the one obtained by Sandgren using Branch and Bound
Case Studies in Mechanical Design method. The percentage error has been reduced to nearly zero.
Since the answer was close to the starting point used by Sand-
In this section, five mechanical design problems involving gren, six additional starting points were tried and six different
mixed discrete, integer, zero-one, and continuous variables are results were obtained. Two results were better than the answer
presented. Some of these have been solved by Sandgren (1988) obtained by Sandgren. The results are listed in Table 2.
using the Branch and Bound method. These examples are based
on the design of mechanical components and show a wide Case Study 2: Design of a Pressure Vessel (Sandgren,
variety of integer and discrete applications that occur in prac- 1988). A cylindrical pressure vessel is capped at both ends
tice. by hemispherical heads as shown in Fig. 4. The total cost,
including the cost of material, cost of forming and welding,
Case Study 1: Design of a Gear Train (Sandgren, 1988). The is to be minimized. The design variables are Ts and Th, the
compound gear train shown in Fig. 3 is to be designed such thicknesses of the shell and head, and R and L, the inner radius
that the gear ratio is as close as possible to 1/6.931 (or and length of the cylindrical section. These variables are de-
0.1442793). The number of teeth for each gear should lie be- noted by Xu x2, x-i, and x4, respectively, and units for each are
tween 12 and 60. Since the number of teeth is to be an integer, inches. The variables are such that R and L are continuous,
the variables must be integers. while Xs a n d Th are integer multiples of 0.0625 inch, the avail-
The problem can be mathematically stated as: able thicknesses of rolled steel plates.
27 This problem is mathematically formulated as follows.
Minimize: ^(X)= — — - - 7 - 7 } = 1 ^ , - ~ ^ ( ( )
(6.931 TaTf) (6.931 x3x,j Minimize: F(X) = 0.6224x1;c:3x(
Subject to: 12<x,<60 ; = 1 , 2, 3, 4 (28)
+ 1.7781x^ + 3.1661x1*4+ 19.84x?x3 (30)
and x/'s should be integers. (29)
Subject to: gi(X)= - X , + 0.0193A: 3 <0 (31)
where X = [x„ x2, x3, x4]T= [Td, Tb, Ta, Tf]T
The problem was solved using Augmented Lagrange Multiplier g 2 ( X ) = - x 2 + 0.00954jt3<0 (32)

4 0 8 / V o l . 116, JUNE 1994 Transactions of the ASME

Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms

 Table 3 Pressure vessel design Table 5 Spring design problem
Type of Starting values Optimum Optimum Type of Starting values Optimum Optimum
Design Variable of the Solution Solution Design Variable of tile Solution Solution
Variables (Disc/Int/Cont) variables [Kannnn [Sandgren] Variables (Disc/Int/ConL) variables [Kannan [Sandgren]
and Kramerl and Kramerl
xi ( Ts ) Discrete 1.25 1.125 1.125 JTI(N) Integer 10 7 10
xi ( n ) Discrete 0.625 0.625 0.625 jr2(d) Discrete 0.375 0.283 0.283
*3(R) Continuous 50.0 58.291 47.700 *3(D) Continuous 2.000 1.329 1.180
« ( D Continuous 120 43.690 117.701 Design Constraints
Design Constraints «l(X) 64574.812 -734.000 54.309
*l(X) . 0.285 -2.925e-04 0.204 «2(X) 6.685 -9.540 8.818
*2<» 0.148 -0.069 0.170 «3<X> 0.175 -0.083 0.082
S3(X) 170079. -0.138 0.000 S4(X> 1.000 -1.390 1.819
84 (X) 120.000 -196.225 122.299 S5<X) 2.333 -1.700 1.172
F(X) 9588.986 7198.200 8129.800 «6<X) 5.155 -5.400 5.464
* In Sandgren's work, inequality constraints are satisfied when they are non-negative. 87 (X) 0.000 -8.290 0.000
«8(X) 0.719 0.000 0.000
F(X) 8.327 2.365 2.799
Table 4 Possible wire diameters for ASTM A228 (inches) * In Sandgren's work, inequality constraints are satisfied when they are non-negative.

0.0090 0.0095 0.0104 0.0118 0.0128 0.0132

0.0140 0.0150 0.0162 0.0173 0.0180 0.0200
0.0230 0.0250 0.0280 0.0320 0.0350 0.0410
0.0470 0.0540 0.0630 0.0720 0.0800 0.0920 This problem is formulated as follows.
0.1050 0.1200 0.1350 0.1480 0.1620 0.1770 Minimize: F(X) = 0.25 ir2Dd2 (N+2) (36)
0.1920 0.2070 0.2250 0.2440 0.2630 0.2830 Subject to: g, (X) = - S+ 8KPmaxD/(ird3)<0 (37)
0.3070 0.3310 0.3620 0.3940 0.4375 0.5000
ZPnajfN
&>(X)= -lmax + - -+l.05(N+2)d<0
Gd*
(38)
g3 ( X) = - TTJ^X, - - 7T^3 + 1,296,000 < 0 (33) g3(X)=-d + dmi„<0 (39)
g4(X)=-D,mx + D + d<0 (40)
g 4 (X) = x , - 2 4 0 < 0 (34) g5(X)=-C+3<0 (41)

X[ and x2 are integer multiples of 0.0625 (35) g6(X)=-8pm + 8<0 (42)

This problem was solved using the augmented Lagrange mul- 8PnwxD5N
g7(X)=-//+ -+ {P,n i)/K
tiplier algorithm combined with Powell's method. The starting Gd"
values for the variables and the optimum solution obtained as -1.05(yV+2)tf<0 (43)
well as the results obtained by Sandgren using Branch and
Bound method are shown in Table 3. gi(X) = {Pmax-Pload)/K-bw<Q (44)
The starting value for rp used in the problem was 0.0001
and the multiplication factor used was 1.8. The starting values In the above equations,
of Lagrange multipliers used were zero. S is the maximum allowable shear stress,
In this problem, the starting values used by Sandgren were
very close to the optimum and these values were used for Pmax is the maximum design load,
comparison purposes. The active constraint was found to be
gi(X) as opposed to gi(X) as found by Sandgren. Even still, dmin is the minimum wire diameter,
the method proposed here produced a better value of objective
function. In addition, seven more starting values were tried Dmm is the maximum outer diameter,
considerably farther from the optimum point and the same
result was obtained. The starting values tried were: (2.5, 2.5, 5pm is the maximum deflection under preload,
50.0, 120.0), (0.0,0.0,50.0, 120.0), (0.0,0.0, 100.0, 120.0), K is the spring correction factor
(5.0,0.0, 100.0, 120.0), (5.0,0.0,200.0, 120.0), (10.0, 1.0, = {AC- 1)/(4C-4) + .615/C, and (45)
200.0,120.0) and (0.0,0.0,200.0,200.0). In this formulation,
the inequality constraints are satisfied when they are negative.
In Sandgren's work, they were satisfied when they were po- C is the spring index = D/d. (46)
sitive.
The values of all the parameters can be found in Sandgren
Case Study 3: Spring Design Problem (Siddall, 1982; Sand- (1988) and the derivation of the equations used in formulating
gren, 1988). This is a problem involving discrete, integer, and ' the constraints can be found in Siddall (1982). The starting
continuous variables. The objective is to design a helical point and the final optimal solution along with comparison to
compression spring with the least volume. The spring is to be Sandgren's result are shown in Table 5. The starting value for
manufactured from music wire spring steel ASTM A228. This rp used in the problem was 0.0001 and the multiplication factor
means that the wire diameter can assume only discrete values used was 1.1. The starting values of Lagrange multipliers used
shown below in Table 4. The design variables are D (winding were zero.
diameter), d (wire diameter), and N (number of spring coils). As seen from Table 5, a better result has been obtained using
D is a continuous variable, cf is a discrete variable, and N is the Augmented Lagrange Multiplier algorithm compared to
an integer variable. Sandgren's Branch and Bound algorithm.

Journal of Mechanical Design JUNE 1994, Vol. 116/409

Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms

 H H
Fig. 5 Schematic for welded beam problem Fig. 6 Pseudo-objective function contours for different penalties

The objective function to be minimized is:

Table 6 Welded beam design
F(X) = x1(l.2xsx9) +x2(xu) +xi(x?jx9)
Type of Starting values Optimum Optimum
Design Variable of the Solution Solution
+ (0.0481 lx4 + 0.06*5 + 0.015X6)(XKPC8) (L+xg) (47)
Variables (Disc/Int/Conl.) variables [Kannan [Sandgren] Subject to:
and Kramer]
hl(X)=xi+x2 + xi-1.0 = 0 (48)
'1 zero-one 0.3 0 0
'2 zero-one 0.3 0 0 h2(X) = x4 + x5 + x6-\.0 = Q (49)
*3 zero-one 0.3 1 1 gl (X) = - ^allowable + 6FL/xi04 ^0 (50)
x4 zero-one 0.3 0 0
zero-one 0.3 0 0
g 2 ( X ) = - . P c + 6000<0 (51)
'5
"6 zero-one 0.3 1 1 ft(X)=-5max + 4FL 3 /i?x 1(> ^<0 (52)
'1 continuous 5.0 0.320 0.319
where 6max is the maximum deflection and Pc is the computed
"a continuous 5.0 14.109 14.105
buckling load and is a function of the beam dimensions and
continuous 5.0 5.721 5.651
"9
material properties. More details including the derivation of
•VI0 continuous 5.0 0.321 0.320
Pc can be found in Reklaitis et al. (1983). The inequality con-
'11 continuous 5.0 4.991 5.000
straints apply to each material, making the total number of
Design Constraints for Aluminum Qrfi = 1)
inequality constraints equal to nine. The problem is trans-
g|(X) 7083.471 -7077.861 7072.850 formed by modifying only the inequality constraints such that:
gj(X) -7.971e06 -2548.800 2547.712
g3® -24.992 -24.331 24.426 xigj(X)<0 7=1, 2, 3; i = 4, 5, 6 (53)
F(X) 67.969 1.970 1.971 The starting point and the final optimal solution along with
' In Sandgren's work, inequality constraints are satisfied when they are non-negative. comparison to Sandgren's result are shown in Table 6. The
starting value for rp used in the problem was 0.0001 and the
multiplication factor used was 1.5. The starting values of La-
grange multipliers used were zero. An alternate method, of
course, would be solving nine (3 x 3) smaller optimization
Case Study 4: Welded Beam Problem (Reklaitis et al., 1983; problems and selecting the solution with the smallest value.
Sandgren, 1988). This is a problem involving zero-one vari- The constraints shown are for aluminum only since the other
ables and continuous variables. Consider the design of a beam constraints, Eq. (53), are all zero. The answers obtained by
which is to be rigidly attached to a fixed support and is sub- the proposed method and those obtained by Sandgren are
jected to a load as shown in Fig. 5. Three methods of attach- essentially identical and agree that the welded aluminum beam
ment (brazed, bolted, and welded connections) and three is the best.
materials (steel, brass, and aluminum) are considered in the
design. The proper choices of materials and attachment are to Discussion
be made which minimizes cost while satisfying a limit on the
allowable bending stress, the buckling load, and the allowable In addition to the starting values of the independent vari-
deflection at the load application point. The various materials ables, the factors which influence the resulting solution are the
have different elastic and shear moduli and a different cost. starting penalty value, rp, the factor by which this penalty value
is increased, 7, and the starting values of the Lagrange mul-
The design variable vector is defined as tipliers, \j. Let us consider each of these.
X= [xu x2, ..., x6, h, t, p, b, D]
Effect of Starting Penalty Value. One of the main advan-
where tages of using an augmented Lagrange multiplier method based
X\ = zero-one variable for brazed connections solution is that the dependence of the final solution on the
starting penalty value is greatly reduced. Previously, penalty
x2 = zero-one variable for bolted connections function based methods developed by Gisvold and Moe (1972)
x3 = zero-one variable for welded connections produced results that were extremely dependent on the initial
X) = zero-one variable for steel penalty parameter value and this was a major drawback be-
cause each starting penalty value led to different results. All
Xs = zero-one variable for brass of the examples given above were tried with at least six different
x6 = zero-one variable for aluminum starting penalty parameter values and the same result was ob-
tained each time. For example, in the gear train design problem,
x1 = h = thickness of weld (continuous variable) the values used were: (0.01, 0.001, 0.005, 0.0001, 0.0005,
x% = t = width of beam (continuous variable) 0.00001). The only criterion for choosing the initial penalty
x9=p = length of connection (continuous variable) parameter is that it should be selected as a low value and
gradually increased. The reason for this is that, as we increase
Xw = b = thickness of beam (continuous variable) the starting penalty parameter value, the augmented objective
X\ i =D = diameter of bolts (continuous variable) function will have more "bumps" as shown in Fig. 6. As the

410 / V o l . 116, JUNE 1994 Transactions of the ASME

Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms

 size of these "bumps" increases due to larger values of the
penalty parameter, the search tends to "fall" into the nearest
local minimum. Since it is very tough to get the search to come
out of the local minimum, the search tends to get stuck at that
point, making the location of the true discrete optimum very
difficult.
Multiplication Factor for Penalty Values. The multipli-
cation factor for incrementing the penalty parameter should
be chosen to be between 1 and 2. This will ensure that the
penalty parameter is increased gradually. If the multiplication
factor is too high, the size' of the above described bumps in-
creases at a very fast rate and hence arriving at the discrete
optimum becomes very difficult. Also, in some cases, the search
converges to a local optimum if the multiplication factor is
too high. This is because of the same reason as why the starting
penalty value should not be very high. Even if the starting
penalty value is very small, if the multiplication factor is very
high, the size of the bumps increases at an enormous rate,
making the search fall into a local optimum. In the case studies,
the values of the multiplication factor were taken to be between
1 and 2.
Starting Values of Lagrange Multipliers. In the augmented
Lagrange multiplier method, the X's are not taken as inde-
pendent variables in order to keep the number of variables
from getting too large. Since the optimum values of X's are
not known in advance, initial values are assumed and these
are updated after each search is completed. It is recommended
to begin with X<- = 0 when the gradients of the objective
function and the constraints are not calculated (i. e., when using
a zeroth order method). As an alternative, Vanderplatts (1986)
recommends that \k can be taken to be 1.0 if Vhk(X) • VF(X)
< 0 and X* = -1.0 if Vft*(X) • VF(X) > 0. This has the
numerical advantage that it establishes whether X* is to be
increased or decreased depending on the signs of the gradients
of hk(\) and F(X).
Scaling the Constraints. In common engineering design
optimization problems, the constraints are often vastly dif-
ferent in magnitude. Therefore, some constraints dominate the
optimization process. Also, multiplying all the constraints by
the same penalty parameter leads to numerical ill conditioning.
This was observed in all but one of the case study problems
discussed. Therefore, the constraints had to be scaled. The
scale factor for a constraint was taken as the ratio of the
gradient of the objective function to the gradient of that con-
straint at the starting point. This simple step improved the
convergence of the problems to a great extent.
Advantages of Zero-One Variables. The use of zero-one
variables for conceptual optimization problems is more effi-
cient when we have numerous choices for each variable. This
leads to a very large number of combinations. For example,
if we have "n" choices for variable x\,"m" choices for variable
Xi and "p" choices for variable x3, the number of individual
optimization problems to be solved is "nmp." If n - m = p
= 10, then we will have to solve 1000 optimization problems
to see which gives the best result. Instead, the conceptual op-
timization using zero-one variables can accomplish this task
as one optimization problem and hence is much more efficient.
Hardware Details. The algorithm was implemented using
a GATEWAY 2000 "486" PC. For Rosenbrock's function,
the CPU time used to arrive at the solution was 8.63 sec. For
the case studies 1,2, 3, and 4, the total CPU time used were
Journal of Mechanical Design
Downloaded From: http://mechanicaldesign.asmedigitalcollection.asme.org/ on 05/25/2015 Terms of Use: http://asme.org/terms
1.42, 2.61, 2.27, and 7.93 seconds respectively including print-
ing results at each iteration.
Conclusions
From the examples presented above, it can be seen that the
method of solving mixed discrete integer continuous optimi-
zation problems using the augmented Lagrange multiplier
method proved very efficient and accurate. Even fairly large
problems were accurately solved. The iterations in the aug-
mented Lagrange multiplier method require very little storage
space as there is nothing which needs to be stored in memory
to be used later. The augmented Lagrange multiplier method
combined with zeroth order search was found to be better than
with first order search methods since the use of finite difference
techniques in approximating the derivatives was found to give
considerable problems in arriving at the discrete optimum so-
lution. In some smaller problems, however, the derivative was
found analytically so that first order methods were very useful
for these problems. For large size problems, analytical deter-
mination of the derivatives is often not feasible so that first
order methods are not recommended. The zeroth order search
technique used (Powell's method), was successful in finding
the solution for all of the case studies conducted. The welded
beam problem stands as a good example of how this method
can be used to solve conceptual design optimization problems
using zero-one variables.
References
Beveridge, G. S., and Seheeter, R. S., 1970, Optimization: Theory and Prac-
tice, McGraw-Hill, New York.
Cha, J. C., and Mayne, R. W., 1987, "Optimization with Discrete Variables
via Recursive Quadratic Programming; Part I: Concepts and Definitions; Part
II: Algorithms and Results," Proceedings of the 1987 ASME Design Automation
Conference, Boston, MA, Vol. 1, pp. 7-22.
Cha, J. C , and Mayne, R. W., 1988, "The Symmetric Rank One Formula
and its Application in Discrete Nonlinear Optimization," Proceedings of the
1988 ASME Design Technology Conference, Kissimmee, FL, pp. 95-105.
Davydov, E.G., and Sigal, I., 1972, "Application of Penalty Function Method
in Integer Programming Problems," Engineering Cybernetics, Vol. 10, No. 1,
pp. 21-24.
Fletcher, R., and Reeves, C. M., 1963, "Function Minimization by Conjugate
Gradients," Computer Journal, Vol. 7, No. 2, pp. 149-154.
Fox, D. B., and Liebman, J. S., 1981, "A Discrete Nonlinear Simplex Method
for Optimized Engineering Design," Engineering Optimization, Vol. 5, pp. 129-
149.
Fu, J. F., Fenton, R. G., and Cleghorn, W. L., 1989, "Nonlinear Mixed
Integer-Discrete-Continuous Programming and its Application to Engineering
Design Problems," Proceedings of the 1989 ASME Design Automation Con-
ference, Montreal, Canada, Vol. 2, pp. 59-65.
Gisvold, K. M., and Moe, J., 1972, "A Method for Nonlinear Mixed Integer
Programming and its Application to Design Problems," ASME Journal of
Engineering for Industry, Vol. 94, pp. 353-364.
Gupta, O. K., and Ravindran, A., 1983, "Nonlinear Integer Programming
and Discrete Optimization," ASME JOURNAL OF MECHANISMS, TRANSMISSIONS
AND AUTOMATION IN DESIGN, Vol. 105, pp. 160-164.
Kannan, B. K., 1993, "An Augmented Lagrange Multiplier Based Method
for Mixed Integer Discrete Continuous Optimization and its Application to
Mechanical Design," Masters Thesis, Department of Mechanical Engineering,
University of Toledo.
Powell, M. J. D., 1964, "An Efficient Method for Finding the Minimum of
a Function of Several Variables without Calculating Derivatives," Computer
Journal, Vol. 7, No. 4, pp. 303-307.
Powell, M. J. D., 1981, "Optimization Algorithms in 1979," Committee on
Algorithms Newsletter, No. 5, Mathematical Programming Society, pp. 2-16.
Reklaitis, G. V., Ravindran, A., and Ragsdel!, K. M., 1983, Engineering
Optimization, Methods and Applications, John Wiley and Sons, New York.
Sandgren, E., 1988, "Nonlinear Integer and Discrete Programming in Me-
chanical Design," Proceedings of the ASME Design Technology Conference,
Kissimmee, FL, pp. 95-105.
Siddall, J. N., 1982, Optimal Engineering Design: Principles and Applications,
Marcel Dekker, New York.
Vanderplatts, G. N., 1986, Numerical Optimization Techniques for Engi-
neering Design: with Applications, McGraw-Hill, New York.
JUNE 1994, Vol. 116/411