A C++ Class Library for Mathematical Programming

Soren S. Nielsen Management Science and Information Systems University of Texas Austin, TX 78712 nielsen@guldfaxe.bus.utexas.edu June 1993 Revised, June 1994
Abstract
We present a library of C++ classes for writing mathematical optimization models in C++. The library de nes classes to represent the variables and constraints of models, and also de nes overloaded operators on these classes which results in a natural syntax for model de nition. The system requires programming in C++ for its use, and is hence most suitable for the advanced programmer/modeler. For such a user, however, it provides some advantages over standard modeling systems. First, the system preserves all the advantages of working with a programming language, such as e ciency, exibility and openness. Second, C++ allows users to extend and specialize existing data types. As an example of this, we show how a user could de ne a specialized network model type with nodes and arcs. E cient data structures for storing and manipulating sparse arrays are introduced, the concept of variable aliasing is discussed, and a number of related, future research topics are presented.

Key Words: Modeling Languages, Object-oriented Programming, C++.

To appear, The Impact of Emerging Technologies on Computer Science and Operations Research, A. Sofer and S.G. Nash (ed), Kluwer, 1995.

1 Introduction
Mathematical programming systems, for instance GAMS 1992] or AMPL 1993], facilitate the formulation and solution of mathematical optimization models by providing high-level modeling abstractions such as variables and equations, and languages for e ciently manipulating and combining these objects, and by providing automatic interaction with optimization software. However, formulating and solving a mathematical model is often only a part of the solution process. In realistic applications an optimization system is (or should be) an integrated part of a larger decision support system, since data typically need to be processed by other parts of the system before and after the optimization step: Storing and retrieving data from data bases, reacting to live data feeds, presenting results graphically and interacting with other \black box" components. When models are used operationally, the process of data collection, model solution and result processing needs to be automated, and cannot be carried out entirely within the modeling system. Although modeling languages are developing in response to these issues, and increasingly are beginning to incorporate programming languages constructs, they are generally di cult to integrate, and modelers are still often forced to escape the modeling system in favor of languages such as C or FORTRAN. At the same time as modeling languages are developing, so are programming languages. A modern, object-oriented language such as C++ (Ellis and Stroustrup 1990]). gives the programmer the full exibility and e ciency of any programming language, but at the same time allows the de nition of \abstract data types", through classes, which can be used to represent high-level objects. The idea naturally arises to de ne modeling abstractions in the programming language, and hence taylor the programming language towards modeling applications, rather than approaching the capabilities of programming languages by extending the modeling systems. We present here a library of C++ classes which de nes and implements modeling abstractions. By suitably rede ning (overloading) standard C++ operators, a syntax very similar to that of GAMS or AMPL can be used in the de nition of expressions and constraints. Our aim is to show that the exibility and capabilities of a modeling system can be approximated using programming language constructs, thus combining the notational convenience of modeling abstractions o ered by modeling systems with the openness, e ciency and exibility of a programming language. To this end, we present several examples of the use of the system. We assume that the reader is somewhat familiar with C, but assume no prior knowledge of C++. While the C programming language, Kernigan and Ritchie 1978], is becoming widely used to write optimization software (e.g., CPLEX, Bixby 1992], LOQO, Vanderbei and Carpenter 1993]), C++ is not widely used in the OR community. Birchenhall 1992] uses C++ to de ne a matrix library for econometrics. This library could be very useful as a supplement to, or integrated with, the present class library, since it incorporates routines for matrix decomposition and for solving systems of equations. 2

However, in contrast with the mathematical programming library, it applies dense data structures throughout, and is consequently less suited for large-scale applications. The paper is organized as follows. We rst give a small example LP model. We then discuss aspects of the C++ implementation of the class library, and discuss advantages and disadvantages of using C++ for modeling. Section 4 discusses the addition of vectors and associated operations to the library to allow an e cient, algebraic notation for vector operations. Section 5 introduces the concept of aliasing, and in Section 6 we give some future research topics.

2 A Small Example Model

In order to introduce our ideas, we show here a small LP model formulated in GAMS, Figure 1. We also show in Figure 2 how this model can be formulated in C++, using the mathematical programming (MP) library. The model is from AMPL 1993], where its formulation is found on page 5. The two representations declare and de ne model components in the same order. The C++ model begins by including the MP library de nitions. Then the model variables, constraints and the model itself are declared. These data types (MP variable, MP constraint and MP model) are de ned as classes in the MP library. We next de ne the constraints of the model. The statement
Time = (1/200.0)*X_B + (1/140.0)*X_C <= 40.0;

illustrates an example of operator overloading: Here, the <= operator is de ned to accept two parameters (of type MP expression), and return an object of type MP constraint, which is then assigned to the object Time. The right-hand side of the <= operator is the constant 40.0, while the left-hand side is constructed by the overloaded + and * operators, which take operands of numeric type or type MP variable and return objects of type MP expression. The e ect of executing this statement is to build a tree-structure which represents the constraint, and store a reference to the tree structure in Time. Note that the standard uses of <=, + or * on numerical data are not lost. C++ is a strongly typed language, and the precise interpretation of operators is determined at compile-time by the types of the operands. After de ning the remaining constraints, it only remains to de ne which constraints are part of the model. This is done by calling the subject to function of the model object with a list of constraint objects. We then de ne the model's objective, which is another object of type MP expression, and ask for it to be solved and the objective function maximized. The system is presently connected to only one LP solver (LOQO, Vanderbei and Carpenter 1993]), but the interface is su ciently general that new solvers can be added quite easily, as long as they can be called as subroutines from the C++ system. 3

* Small LP example. Ref: AMPL 1993, p. 5] variable profit; positive variables X_B, X_C; equations Time, B_limit, C_limit, prof_def; Time B_limit C_limit prof_def .. .. .. .. (1/200)*X_B + (1/140)*X_C =L= 40; X_B =L= 6000; X_C =L= 4000; profit =e= 25*X_B + 30*X_C;

model m /Time, B_limit, C_limit, prof_def/; solve m maximizing profit using lp;

Figure 1: A small LP model in GAMS

// C++ LP model in 2 variables. Ref: AMPL #include "MP_library.h" void main() { MP_variable X_B, X_C; MP_constraint Time, B_limit, C_limit; Time = B_limit = C_limit =

1993, p. 5].

(1/200.0)*X_B + (1/140.0)*X_C <= 40; X_B <= 6000; X_C <= 4000;

MP_model model; model.subject_to( Time, B_limit, C_limit ); model.objective = 25*X_B + 30*X_C; model.maximize(); }

Figure 2: The same model in C++ using the Mathematical Programming class library. 4

3 Structure and Use of the Class Library

The C++ programming language grew out of the popular C language, under the in uence of languages like Simula and Smalltalk. While C++ contains C as a proper subset, there is a distinctly di erent avor to the language. The primary di erence between the two languages is that C++ has classes. Classes are used to de ne and implement new data types and the operations allowed on them. An often-used example is that of de ning a complex-number type as a class containing two oating point numbers (for the real and imaginary part), and then de ning the arithmetic operators (+, -, *, /) and various functions (sin, cos, log) on them, as well as input and output functions and conversion functions from real numbers. Once such a class is de ned, complex numbers can be used in a program just like the built-in arithmetic data types. In this sense C++ is an extensible language. Although C++ is not a declarative language, programming in C++ naturally focuses attention on the declaration of suitable data types and their associated behavior and operations (i.e., classes and their interfaces), and only secondarily on their implementation, whereas in C one tends to focus more on the imperative language constructs, i.e., executable statements. Developing class libraries in C++ can be considerably more complex than programming in many other languages, but useful paradigms and idioms are developing to facilitate this; see, e.g., Coplien 1992]. On the other hand, using a well-designed class library can be quite simple, as our examples indicate. To represent a variable in a mathematical model we de ne an MP variable class. Objects, or variables, of type MP variable have attributes that de ne the type of the variable (continuous or discrete), and oating point elds level, dual, upper and lower, which represent the level and dual values, and bounds of the variable. By default, the level and dual values are 0, and the bounds are 0 and \In nity", respectively, but these can be reset. For instance, x.set upper(200); sets the upper bound of the variable x to 200. There are also functions to x a variable, temporarily changing its bounds to a common value, and to \un x" it. MP variables can be aliased with each other, which allows multiple variable names to refer to the same actual MP variable object; examples are given in Section 5. An MP variable is a special case of an object of class MP expression. Basic arithmetic operators are de ned on objects of type MP expression, such as + and *. These operators again return an object of class MP expression, which is a treestructure representing the operation and its operands. An MP expression can also take the form of a numerical constant, so that numbers can be mixed with variables in expressions. MP expressions are used to specify objective values and constraints. A constraint is represented by an object of class MP constraint. The relational operators <=, 5

3.1 The MP Classes

are de ned to take two objects of type MP expression and return an . Examples of this were seen in Figure 2. Finally, a complete optimization model consisting of an objective function and a constraint set is represented by the class MP model. The primary operations on an MP model are inclusion of constraints and an objective function, and the functions maximize(), minimize() and solve(). The solve() function does not use any objective expression, but just nds any feasible point. These functions interface with standard LP or NLP solvers, although only an LP solver is presently available.
== >= MP constraint

and

The inclusion of these data types into a C++ program can be used to approximate some of the facilities for modeling o ered by dedicated modeling languages. Having to work within C++ carries some costs with it. For example, it is signi cantly harder to learn C++ than, for instance, GAMS or AMPL, and while a modeling system can give reasonable error messages when mistakes are made | because it has some built-in knowledge about the semantics of modeling | the error messages from C++ compilers are usually quite cryptic. Run-time errors such as divisions with zero, which modeling systems catch gracefully, typically aborts the C++ run with hardly any explanation. Hence, the MP library is not suitable for the casual modeler who needs to solve only a few, relatively simple models. However, when the need arises for solving complex models, especially with structures which cannot easily be expressed algebraically, or whose solution requires interaction with the operating environment, modeling within C++ brings a number of advantages. First, the programmer has all the exibility of a programming language, including the ability to write subroutines to manipulate models or model components (variables etc.) or data, to interface with the operating system, for instance to read and write data or present graphics, or to call external \black-box" subroutines. Second, although every run of a model involves compilation and linking, which may be slow compared to the speed with which a modeling system processes a model le, the resulting executable program may run at signi cantly higher speeds, especially if the modeling system uses interpreted code. This can make a signi cant di erence in cases where the same model is executed repeatedly on di erent problem instances, e.g., as part of an operational system, or where the data processing within the model (or model setup) is computationally intensive. The use of algebraic modeling languages tends to favor a concise and compact model representation, where the model formulation can (and should) be independent of the actual problem instance to be solved, such as the number of variables and constraints, and the actual data values. The model formulation is also independent of which solver will ultimately solve the problem. These highly desirable properties of a modeling system, as summarized by Fourer 1983], are presently only partly supported by the C++ class library, and to some 6

3.2 Modeling in C++

extend are a matter of programming style which must be applied consciously. Independence between the formal model formulation | algebraic or not | and a speci c problem instance's dimension and data can in simple cases be achieved in the C++ system by keeping input routines and model speci cation separate, and using dynamic arrays, as illustrated in Figure 4. In the general case, however, we recognize that much more powerful concepts and methods are needed. For example, it would be desirable to have all data instances, variables etc. indexed by sets, the size and structure of which could in turn be determined at run-time, for each problem instance. There is no concept of set in C++ and all arrays are indexed, using integers, from 0 to some upper bound. The design of exible classes to represent sets, preferably sparse and of multiple dimensions, which could then be used to index multidimensional arrays of model components, is under current investigation. At a minimum, it should be possible to use sparse integer sets for indexing, at least in one dimension, and to use vector operations such as inner products to approximate a natural, algebraic notation. Steps towards this goal will be presented in Section 4. Independence between model speci cation and solver, on the other hand, presents no serious problem. As illustrated in our examples, the solution of a model requires the call of one of the optimization functions. These can in turn interface with any available solver suitable for the problem at hand. The solver can either be external (as in GAMS), in which case a problem representation needs to be written to some common area of memory or the le system, and a system call be issued to invoke the solver, or it can be a subroutine which is linked into the system. The latter method was used to connect the class library to the LOQO solver by Vanderbei and Carpenter 1993], for LPs, and to a specialized zero-one minimax solver, which is currently being developed for the purpose of solving mixed-binary linear programs through a generic C++ implementation of Benders decomposition (See next Section). One of the principal advantages of C++ over standard modeling languages is that it allows the use of existing classes and objects as building blocks to create more general or specialized structures. We illustrate this exibility with two examples: First, we show how the MP library classes can be specialized to create an environment for the speci cation and solution of network models. Second, we illustrate how the capabilities of the library can be extended by outlining how a general implementation of a decomposition algorithm can be written in a problem-independent way, using the build-in solvers as building blocks. The rst example shows the specialization of the MP classes to network modeling. The customization code is shown in Figure 3. We de ne classes to represent nodes and arcs of the network using the existing constraint and variable classes, respectively. The class Network node is de ned as a sub-class of the existing class MP constraint, another class Network arc as a sub-class of MP variable, and 7

3.3 Extending and Specializing the Library

//

Defining network data types (nodes, arcs) using existing classes.

#include "MP_library.h" class Network_node: public MP_constraint { public: Network_node() // Constraints are equality constraints by default: { constraint_type = eq; }; }; class Network_arc: public MP_variable { public: void connects(Network_node& from_node, Network_node& to_node) { // Add this arc to balance constraints of from- and to-node. // The expression '*this' refers to the arc object for which // the 'connects' function was called. from_node.rhs += *this; to_node.lhs += *this; }; }; class Network_model: public MP_model { };

Figure 3: models.

network.h

: De ning classes to represent the nodes and arcs of network

// Define and solve a generic network model. #include "network.h" void main() { int i, num_nodes, num_arcs, *from_node, *to_node; double *supply, *cost; read_network(num_nodes, num_arcs, from_node, to_node, supply, cost); array<Network_node> array<Network_arc> Network_model net; node(num_nodes); arc(num_arcs);

// Define network topology: for (i = 1; i<= num_arcs; i++) arc i].connects( node from_node i]], node to_node i]] ); // Add supplies to right-hand sides of nodes (demands are negative): for (i = 1; i<= num_nodes; i++) node i].rhs += supply i]; // Set up the objective function: for (i = 1; i<= num_arcs; i++) net.objective += cost i]*arc i]; // The model consists of all the nodes: for (i = 1; i<= num_nodes; i++) net.subject_to( node i] ); // Finally, solve the model: net.minimize(); }

Figure 4: Example of the use of the simple network classes to express a generic network model in terms of nodes and arcs. The routine read network initializes its parameters with the network size, topology and other data; its de nition is not shown. The declarations array< ... > declare dynamic arrays of the type speci ed between the brackets. Note that this speci cation is completely independent of the speci c problem instance to be solved, and of the solver used. 9

nally a class Network model as a sub-class of MP model. New classes de ned in this way inherit all the properties of their parent classes, and can be used wherever the parent classes can. For instance, Network nodes are (specializations of) constraints, and have constraint types and duals associated with them, Network arcs are variables which can be used in ow conservation constraints of arcs, and a Network model automatically has an objective function and the functions maximize(), minimize() and solve(), and accepts Network nodes as constraints. The de nition of Network node states that the default constraint type is equality. A Network arc has a function connects which is called to specify the nodes to which the arc is incident, and which modi es the balance constraints of the two nodes to take the arc's ow into account. This is done by adding the arc variable to the constraints' right-hand and left-hand sides, rhs and lhs. The class Network model in this example adds nothing to the de nition of MP model, but could di er, for instance in having a network solver as the default solver. As a result of these de nitions, the user is now able to think in terms of nodes and arcs. The network topology needs to be de ned to the system, but the fact that network models implicitly have balance constraints for each node is captured once and for all in the de nition of the Network node class, so no (explicit) constraints are needed. For completeness, an example of the use of these network classes is given in Figure 4, which implements a generic model for pure network problems. While this de nition of network modeling concepts may be too simplistic for realistic use, it illustrates the point that C++ allows extensive customization and reuse of existing class libraries to meet speci c users' needs. We mention that the language AMPL also has specialized facilities for handling network models, but they are an integral part of the language, not a user-de ned extension. Our second example shows how one can use subroutines in C++ to encapsulate and generalize common operations, just as in any other programming language. We will outline how one could write a routine to implement a decomposition algorithm in a quite general fashion. The user should be able to set up a model | for instance, a mixed-integer model | and then call this decomposition routine with the model as a parameter to have it solved. This should be contrasted with, for instance, GAMS. In order to solve a model by decomposition in GAMS, the modeler needs to specify the complete algorithm for his/her speci c problem, by setting up master and subproblem de nitions, and implement an explicit loop which updates these problems appropriately, while checking for convergence. Until the latest version of GAMS, 2.25, this was further complicated by the lack of a suitable looping facility. Usually, the structure of the underlying model is obscured by this rewriting process, and if it changes, only an expert (usually the original modeler) can change the model. The problem clearly is the di culty in separating the model which is solved, from the algorithm | as expressed within the modeling system | for solving it. We outline in Figure 5 how a general (i.e., model-independent) implementation 10

#include "mathprog.h" // Define Benders Decomposition Routine: void Benders(MP_model& model) { // Informal outline of algorithm: MP_model Master, Sub; // Here we know nothing about the size or structure of 'model'. // Get that information by looping through the constraints. for current_constraint = each constraint in model do { if (only integer variables in current_constraint) Master.subject_to( current_constraint ); // Add to master else Sub.subject_to( current_constraint ); // Add to subproblem } while (not converged) { Master.minimize(); // Uses binary min-max routine Fix integer variables in Sub at solution to Master; Sub.minimize(); // Uses LP solver Master.subject_to( new_cut ); } } void main() { MP_variable ... ; MP_binary_variable ... ; MP_constraint ... ; MP_model my_model; // Set up model as usual, then instead of my_model.minimize(), do: Benders( my_model ); }

Figure 5: Illustration of the capability in C++ to write general routines which work with class objects as parameters. This example outlines the de nition and use of a general routine for solving mixed-integer models by decomposition. 11

of Benders decomposition can be written. The Figure shows the clear separation between the model speci cation in the main program, and the application of the decomposition algorithm to solve it. Hence, the decomposition routine is immune to changes in the model, and vice-versa. The decomposition routine could easily be equipped with optional parameters to indicate which master and subproblem solvers the user would like to use. A recently developed implementation of Benders decomposition within the C++ system, for use in solving mixed-binary LPs, uses the LOQO solver to solve the LP subproblems, and a specialized binary min-max solver to solve the master problems. The actual implementation, which is documented in a forthcoming working paper, is too long for this paper, but follows the above outline closely.

4 Algebraic Notation and Sparse Arrays

One of the primary advantages of modeling languages is the use of algebraic notation, which provides a very compact way to express operations on vectors and matrices, such as multiplication or addition. In order to introduce such convenient notations in C++ we rst need to de ne a suitable notion of vectors and matrices, with elements consisting of numbers, variables, etc. While C++ allows the de nition of arrays1 of any type, there are no operators associated with them (except indexation), and it is not possible to de ne operators on standard arrays in C++. Instead, it is necessary to de ne classes to implement vector objects. Since these classes ideally should allow the de nition of vectors of arbitrary type, they are naturally de ned as template classes, i.e., classes which take type parameters (see, e.g., Ellis and Stroustrup 1990]). We describe here how vectors are implemented in the C++ class library, and the simple operations we have de ned, and how this leads to a convenient notation for simple vector expressions, not unlike that o ered by some modeling languages. These classes and operators could then be used as building blocks for de ning matrix objects, but for this paper we have limited ourselves to vectors. A vector, as an abstract object, is a way to store objects by integer indices, so they can later be retrieved. The simplest implementation of a vector uses xed bounds, and allocates space for each element with index within the bounds. For use with mathematical programs, such dense data structures are insu cient: There are many cases where only very few of the potential elements of a vector are actually used, and to allocate space for all the potential elements within the index space would be extremely wasteful. Also, subsequent operations on such vectors could be executed much faster if it were known in advance which ones were unused. Hence, an implementation of sparse vectors is needed. We implement sparse arrays as a template class whose primary operator is the
We use the terms array and vector interchangeably, although array tends to be a programming term, and vector a mathematical term.
1

12

indexation operator, operator ]. The underlying data structure is a balanced, binary tree, in which space is allocated only to elements which are referenced through the indexation operator. Initially, the tree is empty, and it grows in a balanced way whenever a new element of the vector object is referenced (There is also a way to remove elements, in which case the tree shrinks while staying balanced). The particular variant of tree we use is the AVL-tree, Adelson-Velskii and Landis 1962]. These trees allow access (including insertion and deletion) to elements in logarithmic time. We have modi ed the implementation of AVL trees from Wirth 1976] to allow sequential access to all elements of the sparse vector, which is needed to e ciently implement arithmetic operators on such classes. (Section 4.1). Of course, since the use of sparse data structures for small, or dense, arrays is necessarily wasteful, the implementation should ideally be able to switch automatically between a dense and a sparse representation. We have not implemented this facility. Sparse arrays of double precision numbers and MP variables are declared using the types sparse array and MP var array2, respectively. We have de ned the operator * for inner product on these types, and illustrate in Figure 6 their use. Of course, additional operators, such as addition or subtraction, should also be de ned for a complete system. The model shown is concerned with constructing a portfolio of stocks, such that the weighed average portfolio (\beta") is equal to 1, and the expected portfolio return is maximized. The inner product is used in the de nition of the meet beta constraint, and in the objective. Another facility of algebraic notation is to write summations and similar operations compactly. Although we have not yet incorporated such operators into the system, we show (also in Figure 6) how this could be done, using the summation operator as an example. The resulting summation operator is used in the de nition of the sum x constraint. This code, which is explained below, illustrates that user-de ned operations can relatively easily be incorporated into models, which is in contrast with most modeling languages.

4.1 Iterators

The use of sparse vectors is indispensable in mathematical programming, but raises certain problems when we try to introduce operations on such objects. To calculate, for instance, the inner product between two sparse vectors, we need to access only the previously referenced elements (assuming that the non-referenced elements correspond to zeros). Without wanting to access, or spend time skipping past, elements which are not present, how can this be done?

Sparse vector<MP variable>, where Sparse vector is

These types are typedef's for the template de nitions Sparse vector<double> and the template vector class.

13

//

Small portfolio model

#include "MP_library.h" MP_exp& sum(MP_var_array& x) // Define a summation function { MP_exp& result = *new MP_exp; // To accumulate result Iterator X(x); // To iterate through x while (X.more()) result += x X.current()]; return result; } void main() { sparse_array exp_ret, beta; double target_beta = 1.0; MP_var_array x; int Num_stocks; read_data(Num_stocks, exp_ret, beta); for (int i = 1; i <= Num_stocks; i++) x i].set_upper( 0.25 ); MP_model portfolio; MP_constraint meet_beta = sum_x = ( beta * x == target_beta ), ( sum(x) == 1 );

portfolio.subject_to(meet_beta, sum_x); portfolio.maximize( exp_ret * x ); }

Figure 6: An example of the use of sparse arrays. The multiplication operator * on arrays is the usual inner product. The routine read data initializes Num stocks, exp ret and beta; its de nition is not shown. Nothing in the model depends on the number of stocks in the universe. 14

double operator*(sparse_array& x, sparse_array& y) { AVL_iterator X(x), Y(y); Bool x_more = X.more(), y_more = Y.more(); double z; while (x_more && y_more) if (!x_more || X.current() > Y.current()) { y_more = Y.more(); } else if (!y_more || Y.current() > X.current()) { x_more = X.more(); } else { z += x X.current()] * y Y.current()]; x_more = X.more(); y_more = Y.more(); } return z; }

Figure 7: Implementing inner products using iterators to access only the allocated elements of sparse arrays without knowledge of their internal representation. To illustrate, consider this simple implementation of the inner product of the real arrays x and y:
long i; double product = 0; for (i = 1; i <= Limit; i++) product += x i] * y i]; return product;

This code is ne for dense vectors, but in the case of sparse vectors su ers from two problems: First, each and every element of both x and y between 1 and Limit are accessed (and hence, in our implementation, allocated space), and the multiplications then executed. Second, this code requires knowledge of the bounds (dimension) of the vectors, even though it is often natural to think of sparse vectors as having no particular bounds at all (being \boundless"). A way around these problems is provided by iterators. Iterators allow accessing only the elements which have been allocated, without knowledge of the internal implementation of the array. An example of the use of iterators is given in Figure 6. The declaration Iterator X(x) associates the iterator X with the sparse array x. 15

Subsequent calls of iterator function X.more() returns True if there are more elements allocated in x, and in this case X.current() returns the index of the next element. This solution allows accessing only the elements needed, and is completely independent of the dimensions of the vectors. As a somewhat more involved example, we show in Figure 7 the implementation of the inner product operator for sparse arrays, which is de ned in the MP library. The routine uses two iterators to iterate through the two vectors, matching up allocated elements. The code shown is for the case of double precision vectors, but is identical for vectors with elements of any type, and is implemented as a template function.

5 Variable Aliasing
In many models it is convenient to be able to refer to the same variable by multiple names. Under the MP library, this can be achieved by aliasing variables with each other. Aliasing does not appear to have a direct counterpart in GAMS or AMPL3 (although AMPL has a \de ned variable" feature). The concept of having multiple names for the same object is supported by C++ in the form of reference variables. In the code fragment
int i,k; int &j = i; // Now j is the same integer variable as i ... j = 3; if (i != 3) Error();

j i

is de ned to be a synonym (alias) for i. However, j will always be an alias for ; there is no way later to make j an alias for k. For MP variables we need more exibility. Aliases are arranged with the is alias of() function:
MP_variable x,y; x.is_alias_of(y); // Now x is the same MP variable as y ... x.set_upper(10.3); if (y.upper() != 10.3) Error();

After the call of is alias of(), the names x and y refer to the same variable, and the two constraints are equivalent. We give an example of the use of aliasing from stochastic programming.
Our use of the term \alias" does not have anything to do with the semantics of the GAMS ALIAS statement.
3

x + y >= 10; x + x >= 10;

16

array<MP_var_array> x(Num_scen); array<MP_constraint> flow_conservation(Num_scen); ... MP_model Two_stage_stochastic; // Either impose non-anticipativity constraints --MP_constraint non_anticipativity Num_scen]; for (i = 1; i <= Num_var; i++) if (is_first_stage(i)) for (scen = 2; scen <= Num_scen; scen++) Two_stage_stochastic.subject_to( x scen] i] == x 1] i] ); // --- or, alternatively, alias the first-stage variables: for (i = 1; i <= Num_var; i++) if (is_first_stage(i)) for (scen = 2; scen <= Num_scen; scen++) x scen] i].is_alias_for( x 1] i] );

Figure 8: Code fragments showing a split-variable formulation of a two-stage, stochastic program. Some of the variables x are rst-stage, the rest are second-stage. Either non-anticipativity constraints or aliasing can be used to enforce equality among rst-stage variables. The declaration array<MP var array> x Num stock] invokes a template class array, and declares x to be a (dense) array of sparse arrays of variables, i.e., a two-dimensional array. A two-stage, stochastic programming model can be viewed as a sequence of models, called scenario sub-problems, which usually have the same structure, but where the data may di er among scenarios. The scenario sub-problems are not completely separate, but share some subset of the variables, called the rst-stage variables. The rst-stage variables, which represent decisions to be made up-front, before realizing one of several possible future events (scenarios), are logically identical across all scenario sub-problems, since the rst-stage decision must be made without foresight. A common way to model two-stage, stochastic programs is by variable-splitting, i.e., giving each scenario sub-problem its own copy of the rst-stage variables, and then adding (non-anticipativity) constraints imposing equality among the separate copies of the rst-stage variables. Figure 8 shows a stochastic model in the split-variable formulation. This formulation is useful because it allows a uniform treatment of rst- and second-stage variables within each scenario, as in the gure, where x consists of both rst- and second-stage 17

5.1 Formulation of Stochastic Programs

variables. Aliasing provides a way to achieve the same notational convenience without having to introduce multiple copies of the rst-stage variables or non-anticipativity constraints, as also shown in the Figure. The variables names x 1] i], x 1] i], ..., x Num scen] i] are aliased such that they for each i refer to the same variable across scenarios. In e ect, we still use the convenient split-variable formulation, but the model implementation contains no redundancy. More generally, variable aliasing can be used as a modeling tool for connecting separate (sub-)models within a larger system of models. Separate models, for instance for managing inventories of raw material, production, and shipping, could be maintained independently, but by aliasing key variables be solved as a large, combined model. Geo rion 1990] gives another example where a transshipment model is hierarchically decomposed into two transportation models. This is just one approach to distributed modeling and model management; for a survey on these topics see Krishnan 1993].

6 Extensions
We discuss now several extensions to the MP library, which are the topics of current research.

Multi-dimensional Arrays: Although the MP library allows the use of (spar-

se) arrays of numbers and variables, it does not support matrices and higherdimensional arrays. Such objects can be declared using regular C++ arrays, but still no operations (such as matrix multiplication) are available. For advanced use of the library, one needs a general array structure, which allows for arbitrary dimensions, and which provides the natural operations for 1- and 2-dimensional arrays, i.e., vectors and matrices. Index Sets: Both GAMS and AMPL have rich facilities for de ning one- or multidimensional sets and subsets thereof, and for de ning data and variables indexed by such sets. Again, exible set structures are indispensable for anything but the most trivial kinds of models. It appears that such index structures could be de ned using C++ classes. Managing (possibly multidimensional) sparse sets is very similar to managing sparse arrays, so set classes could be based on the machinery for multidimensional arrays. The indexing operation needs to be modi ed to accept indices of set class objects, and operators using sets, such as summations or products over sets, need to be de ned. Type Information and Dimensional Analysis: Automatic consistency checking and type or unit conversions help manage large models or the integration of several models. Strongly typed programming languages such as Pascal or C++ associate a type with each datum or variable in a program, and 18

enforce type consistency. ASCEND (Piela et al. 1991]) is an example of a modeling system which allows type or dimensional information to become part of a model. Type information helps the system guard against inconsistencies, for instance adding apples and oranges, or provide automatic conversion, for instance between meters and yards. It also helps enforce domain constraints, for instance that a node-arc incidence matrix must be indexed by pairs of numbers of type \node". In C++ type information could be represented by objects which could optionally be associated with data elements and variables. The operators de ned on MP library objects, such as + and *, could then be extented to type-check (at run-time) the consistency of operand types, and possibly provide some automatic conversions. Nonlinear Programming: The syntax introduced allows for writing nonlinear expression as part of constraints or in the objective function. Since the class library has complete control over the expressions generated, symbolic differentiation routines could be developed which would provide exact derivatives to a non-linear optimizer. At the same time, a derivative (or gradient) function could be made available at the user level, so that a modeler could write a nonlinear expression, and have its gradient and Hessian generated automatically. This would aid in the development of prototype non-linear algorithms.

7 Conclusion
We have presented a library of classes and de nitions which allow the convenient formulation of simple mathematical optimization models within the high-level programming language C++. Operators on the mathematical programming objects were de ned which led to a natural, modeling language-like syntax for de ning expressions, constraints and models. We have also presented data structures for storing and managing large, sparse, one-dimensional arrays of data, variables and constraints. Finally, we have mentioned several topics for future research in the areas of generalized array structures, index sets, type information, and non-linear programming.

References
1] G.M. Adelson-Velskii and E.M. Landis. Doklady akademia nauk SSSR. Soviet Math (English Translation), 3:1259{1263, 1962. 2] C.R. Birchenhall. A draft guide to MatClass { a matrix class for C++. Version 1.0D. Technical report, University of Manchester, Oxford Road, Manchester M13 9PL, UK, May 1992. 19

3] R.E. Bixby. Implementing the Simplex method: The initial basis. ORSA Journal on Computing, 4(3):267{284, 1992. 4] A. Brooke, D. Kendrick, and A. Meeraus. GAMS: A User's Guide. The Scienti c Press, 2 edition, 1992. 5] J.O. Coplien. Advanced C++ Programming Styles and Idioms. Addison-Wesley, 1992. 6] M.A. Ellis and B. Stroustrup. The Annotated C++ Reference Manual. AddisonWesley, Reading, MA, 1990. 7] R. Fourer. Modeling languages versus matrix generators for linear programming. ACM Transactions on Mathematical Software, 9(2):143{183, June 1983. 8] R. Fourer, D.M. Gay, and B.W. Kernigan. AMPL: A Modeling Language for Mathematical Programming. Scienti c Press, 1993. 9] A. Geo rion. Reusing structured models via model integration. Technical report, University of California, Los Angeles, 1990. Working Paper No. 362. 10] B.W. Kernigan and D.M. Ritchie. The C Programming Language. Prentice-Hall Software Series, 1978. 11] R. Krishnan. Model management: Survey, future research directions and a bibliography. ORSA CSTS Newsletter, Spring 1993. 12] K.M. Westerberg P.C. Piela, T.G. Epperly and A.W. Westerberg. ASCEND: An object-oriented computer environment for modeling and analysis: The modeling language. Computers in Chem. Eng., 15(1):53{72, 1991. 13] R.J. Vanderbei and T.J. Carpenter. Symmetric inde nite systems for interior point methods. Mathematical Programming, 58:1{32, 1993. 14] N. Wirth. Algorithms + Data Structures = Programs. Prentice-Hall Series in Automatic Computation, 1976.

20