Anda di halaman 1dari 21

Int. J. Industrial and Systems Engineering, Vol. 10, No.

1, 2012

Initiative mapping methodology for system-ofsystems applications


Shannon R. Bowling*
Department of Electrical Engineering Technology,
Bluefield State University,
219 Rock Street Dickason Hall 230,
Bluefield, WV 24701, USA
E-mail: srbbsu@gmail.com
E-mail: sbowling@odu.edu
*Corresponding author

Ghaith Rabadi, Mahmoud T. Khasawneh,


Nevan E.N. Shearer and Jun Zhang
Department of Engineering Management and Systems Engineering,
Old Dominion University,
241 Kaufman Hall, Norfolk, VA 23529, USA
Fax: +1 757 683 5640
E-mail: grabdi@odu.edu
E-mail: mkhasawn@odu.edu
E-mail: nesheare@odu.edu
E-mail: jxzhang@odu.edu
Abstract: In this paper, we propose developing a generic approach to handle
different system-of-systems applications, including (1) a framework to map or
classify initiatives based on their contribution to different capabilities and
(2) introduce an optimisation-based methodology that selects combinations of
initiatives that will maximise the contribution to the capabilities, and ultimately
maximises the yield or benefit of the entire system. The approach will focus on
using subject matter experts to obtain rankings of capabilities. The optimisation
of initiative selection will utilise binary integer programming and desirability
mapping to ascertain the overall effectiveness of the combined initiatives. After
conducting multiple simulation runs at various budget constraints, the heavy
hitters are dominant and the big losers time and time again are not chosen.
The resulting selection is a subset of the total number of initiatives, and can be
subsequently evaluated in greater detail than trying to evaluate the entire
selection and all of the possible combinations.
Keywords: initiative mapping; desirability function; system-of-systems
engineering; Monte-Carlo simulation; optimisation; MCDM; multi-criteria
decision-making.
Reference to this paper should be made as follows: Bowling, S.R., Rabadi, G.,
Khasawneh, M.T., Shearer, N.E.N. and Zhang, J. (2012) Initiative mapping
methodology for system-of-systems applications, Int. J. Industrial and Systems
Engineering, Vol. 10, No. 1, pp.7595.

Copyright 2012 Inderscience Enterprises Ltd.

75

76

S.R. Bowling et al.


Biographical notes: Shannon R. Bowling is an Assistant Professor in the
Department of Electrical Engineering Technology at Bluefield State University.
He received his PhD in Industrial Engineering with a focus in Human Factors
Engineering from Clemson University, SC, in August 2003; his MS in
Engineering Technology with an emphasis in Quality Management (2000) from
East Tennessee State University, TN and BSc in Electrical Engineering
Technology (1998) from Bluefield State College, WV. He has also a graduate
minor in Experimental Statistics from Clemson University. He teaches topics
related to modelling and simulation including operations research, optimisation
methods, systems dynamics and agent-based modelling. In addition, he teaches
course relating to electronics, power systems and digital circuits. He has
various papers published and/or submitted in the area of modelling and
simulation. He is a member of the Institute of Industrial Engineers (IIE), the
American Society for Engineering Management (ASEM) and the American
Society of Engineering Education (ASEE).
Ghaith Rabadi is an Associate Professor in the Department of Engineering
Management and Systems Engineering at Old Dominion University. He
received his PhD in Industrial Engineering from the University of Central
Florida in 1999. He has an MS and BSc in Industrial Engineering from the
University of Central Florida and the University of Jordan in 1996 and 1992,
respectively. His research interests include operations research, scheduling,
simulation modelling and analysis, applied optimisation, supply chain
management and logistics.
Mahmoud T. Khasawneh is currently pursuing a PhD in Engineering
Management in the Department of Engineering Management and Systems
Engineering at Old Dominion University. He received his ME in Systems
Engineering from Old Dominion University in 2009. He received a BS in
Management Information Systems (MIS) from Hashemite University in Jordan
in 2007. His research interests are focused on modelling and simulation (M&S)
in healthcare. His research aims to utilise M&S as a decision-making
framework cut costs and improves healthcare delivery. He also has an interest
in using agent-directed simulations as a tool to explore social science and
engineering theories.
Nevan E.N. Shearer is a PhD candidate in Engineering Management at the
Department of Engineering Management and Systems Engineering. He
received a Master of Engineering Management from Old Dominion University
in 2007. He has a Bachelors degree in Mechanical Engineering Technology
from Old Dominion University in 2005. His research interests include response
surface methodology and applied statistical methods. He also has an interest in
using operations research to solve optimisation problems and complex
problems.
Jun Zhang is a PhD candidate in the Department of Engineering Management
and Systems Engineering at Old Dominion University. He received his MS in
Management Science and Engineering from Nanchang University, China in
2007. His research interests are focused on modelling and simulation, decisionmaking and operations research. His research focus is on using metaheuristics,
especially genetic algorithms to solve optimisation problems. He is also
interested in using evidence theory for decision-making in an uncertain
situation.

Initiative mapping methodology for system-of-systems applications

77

Introduction

The motivation behind this paper is to help implement large-scale systems of systems. In
applying large-scale systems of systems, there are several objectives that need to be met
and even more ways to achieve those objectives. This paper proposes a methodological
way to evaluate those proposed systems (or initiatives) and gives direction to finding the
best combination of initiatives to apply to reach the objectives, thus maximising the
desirability of the system as a whole. An initiative, in reference to this paper, can either
be the implementation of a policy, system or system of systems. The methodology
focuses on using experts to rank capabilities of each initiative or system and with each
initiative having a known cost associated with it. In addition to the way systems are
referred to as initiatives, the capabilities being evaluated are akin to the objectives of the
system. The ultimate objective is to select appropriate initiatives to maximise the
individual capabilities in such a way that maximises the capability of the system as a
whole, all within our constraints like the amount of money available for investment.

Background research

Cook et al. (1988) utilised a technique for ranking projects based on Roys (1968)
concordance model, and used it along with a tournament ranking method developed by
Ali et al. (1986). Roys model requires that each project be assigned a rating relative to
each criterion, and that criterion be given weights reflecting their relative importance.
Project i is then evaluated relative to project i1 in terms of a concordance and a
discordance index. Butler et al. (2001) developed a ranking and selection procedure that
used a simulator (to generate statistics on cost, duration and crews utilisation) and
combines multiple attribute utility theory (MAUT) with statistical ranking and selection
to select the best configuration of a set of possible configurations using the indifferencezone approach. Ahire and Rana (1995) use a multi-criteria decision-making (MCDM)
approach for selecting total quality management (TQM) pilot projects. Their MCDM
project selection method uses elements of the analytical hierarchy process (AHP) to help
determine the project rankings.
Lee and Kim (2000, p.367) use analytic network process (ANP) combined with goal
programming (GP) for information system (IS) project selection. The ANP is used prior
to the GP to set priorities among multi-criteria and trade-off among objectives. The
resulting information is then used in the GP formulation for project selection. Badri
(2001, p.27) uses a combined AHP-GP model for the selection of the best set of
quality control instruments for customer data collection purposes. AHP is used for the
weighting of five sets of quality measures; next, the weights are utilised in the GP model
for the final selection of elements incorporated into the quality control system being
designed. Derringer and Suich (1980) use a desirability approach to simultaneous
optimisation of multiple response variables. The desirability approach, originally
proposed Harrington (1965) can be adapted for multi-criteria optimisation in many
situations. The desirability approach is a major aspect of the initiative mapping process
developed in this paper.
Chen and Cheng (2009) presented an MCDM approach for the selection of IS
projects. Their method was founded on the fuzzy measure and the fuzzy integral. Rao
(2008) suggested an improved compromise ranking method for multi-attribute material

78

S.R. Bowling et al.

selection problems. The improvement in the method was implemented by introducing the
AHP for weight assignment of importance of attributes and by also using ranked value
judgement on a fuzzy conversion scale for attributes that have qualitative values. Gabriel
et al. (2006) suggested a project selection model that integrates multi-objective
optimisation, Monte Carlo simulation and the AHP. The formulation of the competing
objectives for project selection was carried out using probability distributions to describe
costs. Henriksen and Palocsay (2008) addressed the problem of scoring and ranking
potential R&D projects. The manner in which their method is implemented is by
establishing the set of attributes against which the proposed projects will be evaluated,
and then assigning each project a rating on the evaluation scale. The results are then
combined to get the ordinal scale score using mathematical computations. Huang (2007)
proposed a method that incorporates random fuzzy uncertainty into project selection.
Huang selected investment expenditures and annual net cash flows as random fuzzy
variables. A hybrid algorithm that integrates both genetic algorithm and fuzzy simulation
was designed to solve the proposed models and obtain results. Halouani et al. (2007)
introduced a new method that can integrate both quantitative and qualitative information
in a context of uncertainty. Their method was a response to the typical multi-criteria
group decision-making (MCGDM) techniques which, in their opinion, discriminates in
favour of quantitative information.
Vrana and Aly (2010) suggest the decision-making process dealing with assessing
industrial technologies has inherent problems with being ill structured, subjective and
vague. They approach this problem by using a hierarchical fuzzy decision-making
process that addresses the subjectivity of the problem as well as the quantitative aspect.
Komljenovic and Kecojevic (2009) use a combination of the coefficient of technical level
(CTL) and the AHP to make their selections of materials handling equipment. The
methodology they have developed, however, is specific to the selection of materials
handling equipment and cannot be easily adapted for choosing multiple systems based on
multiple criteria. Baykasoglu et al. (2009) assess several different multi-criteria decision
support (MCDS) methods for making contractor selections. Even though no new
methodologies are developed in their paper, the authors give direction to the advantages
and disadvantages of several MCDS methods.

Initiative mapping methodology

The purpose of this paper is to develop a generic approach to handle different system-ofsystems applications, including
1

a framework to map or classify initiatives based on their contribution to different


capabilities

introduce an optimisation-based methodology that selects combinations of initiatives


that will maximise the contribution to capabilities, and ultimately maximise the yield
or benefit.

The approach will focus on using subject-matter experts (SMEs) to obtain rankings of
capabilities. The optimisation of initiative selection will utilise binary integer

Initiative mapping methodology for system-of-systems applications

79

programming and desirability mapping to ascertain the overall effectiveness of the


combined initiatives. The proposed method is described below in a nine-step process.
Stage 1: Initiative ranking
In this stage, we have a set of initiatives, each of which is evaluated according to specific
capabilities. Several SMEs will rank each initiatives capabilities on a scale from 1 to 5
(with 5 being the strongest and 1 weakest). In many cases, there is difficulty in gathering
data for different capabilities of initiatives. By using expert rankings, we can overcome
this problem and develop a robust basis for ranking the various initiatives. Moreover,
having a diverse group of experts specialising in different fields related to the subject will
offer more reliability in the analysis. Figure 1 illustrates the proposed format for initiative
ranking. In denotes each initiative, rm denotes the ranking of each capability and En
denotes each expert.
Stage 2: Average ranking
After having SMEs rank each initiatives capabilities, they will be averaged and compiled
into a single matrix. Furthermore, the cost for each initiative will be estimated based on
quotes or SME opinion. This methodology will be implemented as a way to incorporate
the feedback of all the SMEs, as it is of utmost importance to combine their expertise into
one measure that can be evaluated objectively. Despite taking the average for SMEs
rankings, a lower bound and an upper bound will be created at a later stage of this
analysis for sampling purposes. This also allows for the incorporation of uncertainty
associated with the SME estimates. Figure 2 provides an illustration of the rankings
averages in addition to the estimated cost associated with each initiative.
Figure 1

Initiative ranking

80

S.R. Bowling et al.

Figure 2

Average ranking

Stage 3: Ranking conversion


Having averaged the rankings, the next step will be to convert those averages into a
number within the range between 3.66 and 3.66 to induce variability (uncertainty) into
the capabilities of each initiative. Consequently, this is the total range of where the
logistic function comprises 95%, having a range of 7.32. The logistic function was
chosen because of its simplicity and similarity to the cumulative distribution function of
the normal distribution. In fact, logistic functions with the appropriate parameters show
no significant difference to the cumulative normal function (Bowling et al., 2009). The
total range is then divided by five in order to give the range of each interval (7.32/5
1.464) associated with the five possible ranking scores provided in Stage 1. Now, we can
easily convert between average rankings and its probability associated with the logistic
function. The proof below shows the relationship between the logistic function and the
ranking conversion:

1
1  e x
1
0.975
1  e x
x 3.66
d

and
1
0.025
1  e x
x 3.66
The range from 3.66 to 3.66 represents a 95% confidence of the logistics function. Since
the total range is divided into five intervals (7.32/5 1.464) and we know the midpoints
of the outer intervals, a simple equation can be used to make the ranking conversion. The
following equation uses the aforementioned approach to convert the expert rankings into
associated values on the logistics curve.

4.392  1.464 u average rank

(1)

Initiative mapping methodology for system-of-systems applications

81

where x is the midpoint for each range. The intercept is 4.392. This equation is used to
calculate the midpoint of each average ranking. For example, if the average ranking of a
capability is one, then the converted ranking is centered at:

4.392  1.464 u 1

which is equal to 2.928. Figure 3 shows the logistic curve, which is used to convert the
rankings into desirabilities. Figure 4 shows the matrix of the ranking conversion.
Stage 4: Bounding
Because we know the total range for each ranking is 1.464, the range associated with
each midpoint is x r (1.464/2). For example, the previous center-point of 2.928 will be
given a range of 2.928 r 0.732 or [3.66, 2.196]. This is the range that the sampling
will be drawn from if the average ranking is one. Figure 5 shows the matrix that displays
the lower bounds and upper bounds associated with each midpoint calculated in the
previous step. The lower bound and upper bound are created to allow multiple
optimisations using Monte-Carlo simulation.
Stage 5: Run sampling
Now that we have established a lower and upper bound for each ranking, we can use a
uniform distribution to draw random values from within these bounds. The uniform
distribution was chosen to create more variance in the samples, as it is the distribution of
maximum ignorance. These random samples will be used to simulate multiple runs.
Monte-Carlo simulation will be used to create initiative/capability matrices for which
optimisations will be performed. Figure 6 demonstrates the matrices used in each
iteration.
Figure 3

Logistic curve (see online version for colours)

82

S.R. Bowling et al.

Figure 4

Ranking conversion

Figure 5

Bounding matrix

Figure 6

Run sampling

Initiative mapping methodology for system-of-systems applications

83

Stage 6: Desirability mapping


The run sample values from Stage 5 are now converted into individual desirabilities
(Harrington, 1965) using the following logistic function:
1
1  ex

(2)

where x is the random number sampled between the lower and upper bound. Desirability
values will be between two extremes, which are zero and one. A desirability of zero
represents a completely undesirable value, while a desirability of one will represent a
completely desirable or ideal response value. Figure 7 displays the mapping of individual
desirabilities for each initiative.
The individual desirabilities are then combined using the geometric mean
(Harrington, 1965), which gives the overall desirability DT :
DT

1/ k

d1 Yi u d2 (Y2 ) u" u dk Yk

(3)

where k denotes the number of responses. Notice that if any response Yi is completely
undesirable di (Yi ) 0 , then the overall desirability is zero.
Next, we face the challenge of creating a theoretical framework for combining
multiple initiatives into one measure.
Stage 7: Aggregate via union
Using the concepts of reliability engineering (Lewis, 1996), the proposed approach of
using desirability is analogous to a system of two or more subsystems working in parallel
or in probability theory, the union of two or more events. Similarly, it is assumed that
multiple initiatives working in parallel will increase the reliability of the system. The total
reliability, or in this case, desirability can never exceed one, and the results
asymptotically approach one indicating the law of diminishing return. Let us consider the
following example:
P(A and B)

P(A)  P(B)  P(A or B), which can be extended to:

P(A and B)

P(A)  P(B)  P(A) u P(B).

P(A and B) 0.33  0.58  0.33 u 0.58 0.719, we can see that 0.719 greater than either
0.33 or 0.58, representing two is better than one.
We notice that the more systems (i.e. initiatives) we have, the number of terms in the
equation above will grow exponentially, which will make it an extremely difficult
mathematical process. Therefore, we propose a simplification of the approach mentioned
above, which, guided by reliability engineering is as follows (Lewis, 1996):
P (A and B)

1  [(1  P(A)) u (1  P(B))]

P (A and B)

1  [(1  0.33) u (1  0.58)]

P (A and B)

0.719

By adopting this approach, we can eliminate the tedious task of evaluating combinations
of events. Figure 8 presents an overview of the aggregate via union process.

84

S.R. Bowling et al.

Figure 7

Desirability mapping

Figure 8

Aggregate via union

Stage 8: Optimal selection (via binary-integer programming)


Integer programming will be used to maximise the overall desirability by selecting the
best combination of initiatives. Optimisation will be performed for each sampling run.
The objective is to optimise the total desirability of the capabilities of the system. During
each optimisation run, samples are drawn from the established bounds. The samples are
then converted into desirabilities associated with each initiatives contribution to the
various capabilities. Then the optimisation is performed to select the best combination of
initiatives within the budget constraints. The optimisation can be performed several times
so that multiple combinations of initiatives may be selected. The way integer
programming will be used in this situation is as a switch to turn on/off the initiatives to
their desirabilities and estimated costs using binary decision variables. Consequently,
binary-integer optimisation software can optimise the overall desirability by switching
on the best initiatives within our constraints, such as the amount of money available for
investment.

Initiative mapping methodology for system-of-systems applications

85

Below is the model formulation for the binary-integer programme:


Max Z

DT

1/ n

dj

where
N

dj

1

1  x u d
i

ij

i 1

or
N

dj

*d

ij

i 1

subject to:
n

x c d total budgeted cost


i i

i 1

xi

0 or 1

where: N: number of capabilities; n: number of initiatives; ci

cost of initiative i.

Stage 9: Final initiative selection (via union)


After optimising several sampling runs, a final initiative selection will be performed. Due
to the fact that the capabilities will change for each optimisation, several initiatives might
be chosen more than the investment will allow for. However, the heavy hitters should
appear to be dominant after several iterations, and the big losers should time and time
again not be chosen. The resulting selection will be a subset of a total number of
initiatives and be much easier to evaluate in detail than trying to evaluate the entire
selection and all of the possible combinations.
An important point to note is that the overall desirability is a conceptual measure. The
overall desirability or even the individual desirabilities give no indication as to an
absolute value of the effectiveness of the initiative or initiative combinations.

Case example/application

Border security is a very complex problem that overarches multiple domains and systems
that may have multiple objectives in addition to securing the borders. If we, for example,
take the department of justice (DoJ) as one of the systems, its responsibility is to hear
cases and prosecute those who violate the law, including illegal immigrants. However,
DoJs main mission may be focused on other types of criminals more than illegal
immigrants. In many cases, the different systems may have conflicting objectives and
priorities that may or may not improve border security.
In an effort to increase border security, several different types of initiatives can be
considered. The objective of implementing different initiatives is to enhance border
security capabilities, such as deterrence, detection, apprehension, processing and

86

S.R. Bowling et al.

returning illegal aliens as shown in Figure 9. There exist multiple processes, activities
and resources that support one or more of the capabilities listed below. To enhance the
different capabilities, different initiatives are usually funded and implemented. To gain
the most benefits, however, the initiatives should be selected properly, so that their
combined effect maximises the overall security. For example, installing surveillance
technology like cameras improves the detection capability and may also enhance the
deterrence capability. However, increasing the number of cameras to a large number may
lead to information overload that may in return reduce the effectiveness of the detection
capability. As the number of the initiatives becomes larger, the impact on the capabilities
and ultimately on border security becomes more difficult to quantify and predict, and
fuzzier to assess.
This framework can be used in the case of border security to evaluate initiative
selections. Each initiative will be evaluated based on their capabilities, such as
deterrence, detection, apprehension, etc., as well as, a certain cost associated with it. In
this example, 25 initiatives are evaluated based on six capabilities and their
corresponding cost. This example will begin at Step 2: average ranking, where the
various SMEs input have been compiled, after the input from each SME was collected at
Step 1. Table 1 shows the average rankings associated with each initiative.
Having averaged the rankings, the next step is to convert those averages into a
number within the range between 3.66 and 3.66 to induce variability (uncertainty) into
the capabilities of each initiative. This conversion was performed using equation 1.
Table 2 displays the rankings conversions.
Now that the rankings have been converted, bounds can be introduced for each
capability to facilitate sampling. The lower bound and upper bound are created to allow
multiple optimisations, using Monte-Carlo simulation. The bounds were calculated
according to the following formula:

1.464
xr

2
where x is the converted average ranking. Table 3 shows all the calculated bounds.
Figure 9

Security broken into five capabilities in the context of border security


Security

Deter (1)

Detect (2)

Apprehend (3)

Process (4)

Return (5)

Initiative mapping methodology for system-of-systems applications


Table 1

87

Average ranking of border security initiatives

Initiative Capability 1 Capability 2 Capability 3 Capability 4 Capability 5 Capability 6

Cost

4.40

1.85

1.85

1.02

3.68

3.54

$5,386,045

1.16

4.64

4.46

4.02

3.04

4.05

$1,347,396

1.13

1.67

1.34

4.83

1.97

2.53

$8,433,967

1.23

3.86

3.03

4.14

3.51

4.73

$3,039,492

3.45

2.86

4.65

3.82

1.64

4.64

$962,504

4.60

4.97

1.83

2.34

2.33

2.52

$1,616,156

1.17

2.53

3.30

4.24

4.77

2.51

$5,607,858

2.60

3.47

4.04

1.08

3.84

1.09

$2,221,389

2.64

3.85

4.72

4.53

4.25

4.35

$6,043,765

4.65

3.47

2.37

4.76

4.21

1.86

$5,553,082

1.46

1.38

2.16

4.84

3.18

1.33

$5,541,814

3.50

3.02

2.69

4.43

2.22

2.16

$5,186,944

3.95

2.99

4.13

1.43

2.92

4.43

$9,346,001

1.35

4.94

2.87

4.64

2.68

2.51

$1,820,285

4.13

1.42

2.56

1.86

1.79

1.99

$6,562,133

3.56

4.64

1.73

2.44

1.60

1.68

$7,483,652

2.95

3.61

4.30

4.17

4.05

2.53

$9,545,516

3.81

3.38

1.61

3.73

2.03

4.10

$9,938,250

4.50

4.74

3.23

4.51

4.25

3.93

$6,171,513

1.98

1.40

1.73

4.22

1.14

4.01

$9,086,594

4.80

2.24

1.71

1.09

1.05

1.92

$7,236,907

3.92

1.90

4.22

2.19

2.82

1.80

$8,991,699

1.59

1.05

4.74

3.89

2.99

1.31

$4,928,913

4.15

3.59

2.92

3.58

1.73

2.12

$2,028,168

4.82

4.98

4.33

2.37

4.36

4.94

$661,973

Table 2

Ranking conversions

Initiative Capability 1 Capability 2 Capability 3 Capability 4 Capability 5 Capability 6

Cost

2.05

-1.68

-1.69

-2.89

1.00

0.79

$5,386,045

2.70

2.40

2.14

1.49

0.05

1.54

$1,347,396

2.74

1.94

2.43

2.68

1.51

0.68

$8,433,967

2.59

1.27

0.05

1.67

0.74

2.54

$3,039,492

0.65

0.20

2.42

1.20

1.99

2.41

$962,504

2.34

2.88

1.72

0.97

0.99

0.70

$1,616,156

2.67

0.68

0.44

1.81

2.60

0.72

$5,607,858

0.59

0.69

1.52

2.82

1.23

2.79

$2,221,389

0.53

1.25

2.51

2.23

1.84

1.98

$6,043,765

88
Table 2

S.R. Bowling et al.


Ranking conversions (continued)

Initiative Capability 1 Capability 2 Capability 3 Capability 4 Capability 5 Capability 6

Cost

2.42

0.70

0.92

2.58

1.77

1.67

$5,553,082

2.25

2.37

1.23

2.70

0.26

2.45

$5,541,814

0.73

0.03

0.45

2.10

1.15

1.24

$5,186,944

0.12

2.10

$9,346,001

1.39

0.01

1.65

2.29

2.42

2.84

0.19

2.40

0.47

0.72

$1,820,285

1.65

2.31

0.64

1.67

1.77

1.47

$6,562,133

0.81

2.40

1.86

0.82

2.04

1.93

$7,483,652

0.07

0.89

1.90

1.71

1.53

0.69

$9,545,516

1.19

0.55

2.04

1.07

1.42

1.61

$9,938,250

2.19

2.55

0.34

2.21

1.82

1.37

$6,171,513

1.50

2.34

1.86

1.79

2.72

1.48

$9,086,594

2.63

1.11

1.88

2.80

2.85

1.58

$7,236,907

1.35

1.61

1.78

1.19

0.26

1.75

$8,991,699

2.06

2.86

2.55

1.30

0.01

2.47

$4,928,913

0.85

1.86

1.29

$2,028,168

0.92

1.99

2.84

$661,973

1.69

0.87

0.12

2.67

2.89

1.95

Table 3

Bounding

Initiative Capability 1 Capability 2 Capability 3

Capability 4

Capability 5 Capability 6

1.317 2.781 2.415 0.95 2.420 0.956 3.624 2.160 0.269 1.733

0.06 1.520

3.427 1.963 1.665 3.13 1.411 2.875 0.762 2.226 0.681 0.783

0.81 2.275

3.467 2.003 2.675 1.21 3.158 1.694 1.945 3.409 2.246 0.782 1.41 0.050

3.318 1.854 0.533 2.00 0.682 0.782 0.935 2.399 0.008 1.472

0.080 1.384 0.933 0.53 1.688 3.152 0.466 1.930 2.725 1.261 1.68 3.139

1.611 3.075 2.145 3.61 2.449 0.985 1.698 0.234 1.719 0.255 1.43 0.034

3.404 1.940 1.416 0.05 0.296 1.168 1.082 2.546 1.864 3.328 1.45 0.017

1.325 0.139 0.045 1.42 0.786 2.250 3.547 2.083 0.496 1.960 3.52 2.058

1.265 0.199 0.515 1.98 1.782 3.246 1.503 2.967 1.103 2.567

1.80 3.269

1.25 2.712

1.690 3.154 0.037 1.43 1.654 0.190 1.849 3.313 1.038 2.502 2.40 0.933

2.980 1.516 3.102 1.64 1.959 0.495 1.968 3.432 0.469 0.995 3.18 1.716

0.000 1.464 0.699 0.76 1.181 0.283 1.368 2.832 1.880 0.416 1.97 0.504

0.654 2.118 0.740 0.72 0.917 2.381 3.025 1.561 0.851 0.613

1.37 2.832

3.152 1.688 2.108 3.57 0.923 0.541 1.665 3.129 1.205 0.259 1.45 0.016

0.917 2.381 3.038 1.57 1.372 0.092 2.400 0.936 2.501 1.037 2.21 0.742

0.081 1.545 1.666 3.13 2.593 1.129 1.552 0.088 2.775 1.311 2.66 1.200

Initiative mapping methodology for system-of-systems applications


Table 3

89

Bounding (continued)

Initiative Capability 1 Capability 2 Capability 3

Capability 4

Capability 5 Capability 6

0.804 0.660 0.161 1.63 1.169 2.633 0.977 2.441 0.800 2.264 1.42 0.042

0.456 1.920 0.182 1.28 2.771 1.307 0.333 1.797 2.147 0.683 0.87 2.338

1.461 2.925 1.822 3.29 0.391 1.073 1.476 2.940 1.091 2.555

0.64 2.099

2.230 0.766 3.071 1.61 2.593 1.129 1.060 2.524 3.456 1.992 0.75 2.213

1.900 3.364 1.841 0.38 2.617 1.153 3.529 2.065 3.586 2.122 2.31 0.850

0.615 2.079 2.343 0.88 1.052 2.516 1.924 0.460 0.996 0.468 2.48 1.018

2.797 1.333 3.591 2.13 1.822 3.286 0.571 2.035 0.742 0.722 3.21 1.742

0.956 2.420 0.138 1.60 0.853 0.611 0.123 1.587 2.592 1.128 2.02 0.557

1.937 3.401 2.161 3.63 1.216 2.680 1.648 0.184 1.257 2.721

2.10 3.568

After establishing a lower and upper bound for each ranking, a uniform distribution is
used to draw random values from within these bounds. Thirty Monte-Carlo simulations
were conducted for various budget constraints. The budget was incremented from
$500,000 to $5,000,000 at $100,000 increments. For instance, at a budget of $500,000,
thirty runs were performed and the results were reported. The purpose of doing multiple
runs is to create a frequency chart to identify dominant initiatives. An Excel
spreadsheet was set up to perform the desirability mapping and Monte-Carlo simulation
necessary for the optimisation of initiative selection.

Results and discussion

The following charts show the results from the repeated runs. Figures 1015 show the
maximum desirability for each capability based on the budgeted amount for the project. It
can be observed that the maximum desirability for each capability does not always
increase as the budget increases. Figure 12 shows a significant decrease in the desirability
at roughly a one-million dollar investment. This is due to the fact that solving the integer
programming formulation reveals that a compromise should be made in reducing the
desirability of one capability may increase the total desirability of the entire system. This
trade-off continues until a critical capability of the entire system is reached, then the
desirabilities of the individual capabilities begin to level off. Notice, however, in
Figure 16, DT continues to have a constant increase in desirability that asymptotically
approaches one. No significant reduction in desirability is noticed between investment
levels due to the fact that DT is the objective that is being maximised in the integer
programming formulation.
Figures 17 and 18 show the frequency of the selected initiatives for investment levels
of $4M and $5M, respectively. At the $4M investment level, only 11 initiatives
(A,E,F,G,H,I,J,Q,R,S,V) were chosen after 30 runs. Of those 11 initiatives, only six
initiatives (A,E,F,I,Q,S) had a frequency of at least 20 or more. Therefore, when trying to
determine what initiatives should be investigated further for possible funding or adoption,
these six initiatives could be evaluated using a more advanced modelling technique like
systems dynamics modelling. The histogram for the $5M investment level shows similar
results to the $4M investment.

90

S.R. Bowling et al.

Figure 10 Desirability of capability 1 (see online version for colours)

Figure 11 Desirability of capability 2 (see online version for colours)

Figure 12 Desirability of capability 3 (see online version for colours)

Initiative mapping methodology for system-of-systems applications


Figure 13 Desirability of capability 4 (see online version for colours)

Figure 14 Desirability of capability 5 (see online version for colours)

Figure 15 Desirability of capability 6 (see online version for colours)

91

92

S.R. Bowling et al.

Figure 16 Total desirability (see online version for colours)

Figure 17 Initiative selection at $4,000,000 (see online version for colours)


Initiative Selection at $4,000,000
35
30
25
20
15
10
5
0
A B C D E F G H I J K L M N O P Q R S T U V X Y Z

Initiative mapping methodology for system-of-systems applications

93

Figure 18 Initiative selection at $5,000,000 (see online version for colours)


Initiative Selection at $5,000,000
35
30
25
20
15
10
5
0
A B C D E F G H I J K L M N O P Q R S T U V X Y Z

Conclusions

The advantages of the initiative mapping process developed in this paper may be
summarised as follows:
1

provides a quick method for choosing from a large number of initiatives

serves as a multi-attribute/multi-criteria ranking framework

combines a desirability approach with Monte-Carlo simulation

this method is unique, because it takes into account the variability of the subjective
rankings given by the experts.

It should be pointed out that this process is used to screen a large number of proposed
initiatives to further evaluate the efficacy of each initiative and/or combinations of
initiatives on the entire system. This approach capitalises on the knowledge of subjectmatter experts while recognising that humans may not have the ability to assess the
impact of initiatives on complex systems. By creating uncertainties associated with SME
rankings, Monte-Carlo simulation can be used to create ranges of the efficacy of
proposed initiatives and proposed initiative selection alternatives. By clearly determining
what initiatives definitely enhance system performance and what initiatives definitely do
not enhance system performance, system designers are allowed to spend more effort
focusing on modelling the impact of the significant initiatives and not waste time trying
to model initiatives that may not enhance performance of the system.
A possible limitation to this methodology is inherent in the way the SME estimates
are combined. This is due to the fact that they are combined using the arithmetic mean.

94

S.R. Bowling et al.

This assumes that each SME is equally knowledgeable. Therefore, it might appropriate to
investigate alternative combination methods for SME estimates.
Future research will be focused into turning the binary-integer programme that was
developed as part of this effort into a generic excel-based decision support system. The
purpose of this tool would be to assist managers in making decisions about potential
policies and projects in system-of-systems applications. An additional feature that can be
incorporated is the ability to change the amount of uncertainty associated with the SME
rankings.

Acknowledgement
The authors would like to thank the reviewers whose constructive comments significantly
improved this manuscript.

References
Ahire, S.L. and Rana, D.S. (1995) Selection of TQM pilot projects using an MCDM approach,
Int. J. Quality and Reliability Management, Vol. 12, No. 1, pp.6181.
Ali, I., Cook, W.D. and Kress, M. (1986) On the minimum violations ranking of a tournament,
Journal of Management Science, Vol. 32, No. 6, pp.660672.
Badri, M.A. (2001) A combined AHP-GP model for quality control systems, Int. J. Production
Economics, Vol. 72, pp.2740.
Baykasoglu, A., Ozbay, E., Gogus, M.T. and Oztas, A. (2009) Contractor selection with multi
criteria decision support tools, Int. J. Industrial and Systems Engineering, Vol. 4, No. 2,
pp.174197.
Bowling, S.R., Khasawneh, M.T., Kaewkuekool, S. and Cho, B.R. (2009) A logistic
approximation to the cumulative normal distribution, Int. J. Industrial Engineering and
Management, Vol. 2, No. 1, pp.114127.
Butler, J., Morrice, D.J. and Mullarkey, P.W. (2001) A multiple attribute utility theory approach to
ranking and selection, Journal of Management Science, Vol. 47, No. 6, pp.800816.
Chen, C-T. and Cheng, H-L. (2009) A comprehensive model for selecting information system
project under fuzzy environment, Int. J. Project Management, Vol. 27, pp.389399.
Cook, W.D., Golan, I., Kzakov, A. and Kress, M. (1988) A case study of a non-compensatory
approach to ranking transportation projects, Journal of Operations Research Society, Vol. 39,
No. 10, pp.901910.
Derringer, G. and Suich, R. (1980) Simultaneous optimization of several response variables,
Journal of Quality Technology, Vol. 12, pp.214219.
Gabriel, S.A., Kumar, S., Ordonez, J. and Nasserian, A. (2006) A multiobjective optimization
model for project selection with probabilistic considerations, Socio-Economic Planning
Sciences, Vol. 40, No. 4, pp.297313.
Halouani, N., Chabchoub, H. and Martel, J.M. (2007) PROMETHEE-MD-2T method for project
selection, European Journal of Operational Research, Vol. 195, No. 3, pp.841849.
Harrington, E.C. (1965) The desirability function, Industrial Quality Control, Vol. 21,
pp.494498.
Henriksen, A.D.P. and Palocsay, S.W. (2008) An excel-based decision support system for scoring
and ranking proposed R&D projects, Int. J. Information Technology and Decision Making,
Vol. 7, No. 3, pp.529546.

Initiative mapping methodology for system-of-systems applications

95

Huang, X. (2007) Optimal project selection with random fuzzy parameters, Int. J. Production
Economics, Vol. 106, pp.513522.
Komljenovic, D. and Kecojevic, V. (2009) Multi-attribute selection method for materials handling
equipment, Int. J. Industrial and Systems Engineering, Vol. 4, No. 2, pp.151173.
Lee, J.W. and Kim, S.H. (2000) Using analytic network process and goal programming for
interdependent information system project selection, Computer and Operations Research,
Vol. 27, pp.367382.
Lewis, E.E. (1996) Introduction to Reliability Engineering, 2nd edition, New York: Wiley.
Rao, R.V. (2008) A decision making methodology for material selection using an improved
compromise ranking method, Materials and Design, Vol. 29, No. 10, pp.19491954.
Roy, B. (1968) Classement et choix en presence de points de vue multiples (la methode
ELECTRE), Revue Francaise dInformatique et de Recherche Operationnelle, Vol. 8,
pp.5775.
Vrana, I. and Aly, S. (2010) Assessing candidate industrial technologies utilising hierarchical
fuzzy decision making systems, Int. J. Industrial and Systems Engineering, Vol. 6, No. 2,
pp.187206.