This article was downloaded by: [Deakin University Library]

On: 12 August 2015, At: 12:42

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place,
London, SW1P 1WG

Engineering Management Journal

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/uemj20

An Integrative Evaluation Framework for Determining

the Value of Group Decision Support Systems
a b c
Manuel Mora , Gloria Phillips-Wren & Fen Wang
a
Autonomous University of Aguascalientes, Mexico
b
Loyola University Maryland, USA
c
Central Washington University, USA
Published online: 20 Apr 2015.

Click for updates

To cite this article: Manuel Mora, Gloria Phillips-Wren & Fen Wang (2014) An Integrative Evaluation Framework
for Determining the Value of Group Decision Support Systems, Engineering Management Journal, 26:2, 24-38, DOI:
10.1080/10429247.2014.11432008

To link to this article: http://dx.doi.org/10.1080/10429247.2014.11432008

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 An Integrative Evaluation Framework for Determining the
Value of Group Decision Support Systems
Manuel Mora, Autonomous University of Aguascalientes, Mexico
Gloria Phillips-Wren, Loyola University Maryland, USA
Fen Wang, Central Washington University, USA
Abstract: Group-based decision-making situations are relevant
to engineering management practice. To assist group decision
making, specialized computer-based tools called Group Decision
Support Systems (GDSS) have been promoted since the late
1980s. Nevertheless, a review of the extensive research on GDSS
design and evaluation reveals that an integrated framework is
not available to evaluate the decision-making value of a specific
GDSS. This article proposes a GDSS-value evaluation framework
by considering the process of, and outcomes from, decision
making together with the technical capabilities needed to provide
them. The framework is implemented with an Analytic Hierarchy
Downloaded by [Deakin University Library] at 12:42 12 August 2015
Process (AHP) model, and its utilization is illustrated with the
evaluation of three real GDSS applications. This integrative GDSS-
value evaluation framework can be used by engineering managers
as a tool to determine the overall decision-making value from an
existing or projected GDSS with the aim of selecting the most
adequate GDSS for a particular engineering management group-
based decision-making situation.
Keywords: GDSS Evaluation, McGrath’s Model, Analytic
Hierarchy Process (AHP), Engineering Management, Group
Decision Support System
EMJ Focus Areas: Information Technology, Quantitative
Methods and Models, Concurrent Engineering and Teams,
Systems Engineering
A
ccording to Kotnour and Farr (2005), engineering
management is defined by IEEE (1990) as “… the
discipline addressed to making and implementing
decisions for strategic and operational leadership in current
and emerging technologies and their impacts on interrelated
systems”. Similarly Rouse and Compton (2009) establish that
systems engineering and management, a foundation discipline
for engineering management, provides a set of methods and tools
for efficient design and effective use of human-machine systems.
Such systems include management decision-making methods
and decision support systems.
Engineering management decisions can be made by a single
decision maker (Huber, 1981) or by groups or teams (Huber,
1984b). In particular, a decision group is “two or more people
who are jointly responsible for detecting a problem, elaborating
on the nature of the problem, generating possible solutions,
evaluating potential solutions, or formulating strategies for
implementing solutions” (DeSanctis & Gallupe, 1987). A group
faces additional challenges compared to a single decision maker
such as considering the views of multiple people and developing
consensus. For example, group-based decision situations within
engineering management practice in business and governmental
settings include: the selection of cargo handlers for military
Refereed Research Manuscript accepted by Associate Editor Wilbon.
24
operations (Bard & Sousk, 1990), the planning and selection of
organizational actions for natural disasters (Belardo & Harrald,
1992), the value management rating in civil engineering projects
(Sánchez et al., 2005), the evaluation of the conceptual design of
aerospace vehicles (Unal et al., 2005), determination of support
value for management workshops in the construction industry
(Fan, Shen & Kelly, 2008), R&D project expert selection (Sun et
al., 2008), support for multi-facility location problems (Bashiri &
Hosseininezhad, 2009), information technology project selection
(Zandi & Tavana, 2010), and development of new electronic
products (Lu et al., 2011).
In all of these examples, adequate group decision-making
processes are needed to achieve expected outcomes and avoid
wasting valuable organizational, economic, and materials
resources resulting from incorrect group decisions. Poor
quality organizational group decisions can result in negative
organizational outcomes such as financial loses, corporate image
deterioration, competitive weakness, cancellation of contracts,
reduction of shared market, and possible damages to third
parties (e.g., ecological disasters) (Huber, 1984a). Consequently,
group-based decision-making situations are relevant concerns
for engineering management practice along with the concurrent
design, evaluation, and selection of tools that support efficient
and effective group decision-making processes.
One of the tools promoted in the information systems area
since the 1980s (Huber, 1984b; DeSanctis & Gallupe, 1985)
to aid decision making for groups is Group Decision Support
Systems (GDSS). In the case of a single decision maker, Decision
Support Systems (DSS) provide the expected support (Huber,
1981). GDSS have been defined as, “interactive computer-based
systems that combine communication, computer, and decision
technologies to support unstructured problem formulation
and solution in group decision-related meetings” (DeSanctis
& Gallupe, 1987). The primary purpose of GDSS is to improve
the efficiency and effectiveness of a decision team by providing
collaborative support, such as interactive retrieval, sharing,
and use of information between decision group participants
and between group participants and the system (Huber, 1984b;
DeSanctis & Gallupe, 1987).
GDSS are developed under the premise that the collective
intelligence of a team outperforms an individual in complex
decisional tasks when the group is effectively supported (Huber,
1984b). With adequate support, negative group effects such as
monolithic rather than creative groupthink, dominance of high
status participants, null participation of low status stakeholders,
group miscommunication, and scarcity of time available for
sufficient exploration of alternatives are minimized (Kraemer &
King, 1986). In contrast, with adequate support, positive effects
such as team creativity, knowledge co-creation, process efficiency,
team learning, team satisfaction, and better decision quality are
expected (Limayem, Banerjee & Ma, 2006).
Engineering Management Journal Vol. 26 No. 2 June 2014
 Decision making by groups, however, is more difficult
to support than by individuals, through computerized tools.
For example, a GDSS has multiple-users/multiple-models
while a DSS has a single-user/single-model (Sage, 1991). A
GDSS faces group performance/satisfaction trade-offs that are
not present in a DSS (DeSanctis & Gallupe, 1987). A GDSS is
framed for an implicit political-competitive decision-making
process model, while a rational-alike model is commonly used
in a DSS (Huber, 1981). A GDSS must allow the co-existence
of different perceptions of data, context, and uncertainties that
are not present in a DSS (Barkhi, 2001-2002; French, 2007). A
GDSS requires treating group coordination, sense-making and
convergent understanding of the problem that is minimized
or lacking for an individual decision maker using a DSS (Gray
et al., 2011). A GDSS may accommodate different structures
based on the physical location of participants (same room or
different room), the size of decision group (small or large), and
the concurrency of the session (synchronous or asynchronous)
(DeSanctis & Gallupe, 1987). An additional difficulty for GDSS is
the lack of a standardized and systematic group-based decision-
Downloaded by [Deakin University Library] at 12:42 12 August 2015
making process (McGrath, 1984; French 2007). In contrast, for
individual decision makers using a DSS, there are several well-
recognized decision-making process models, primarily based on
Simon’s seminal process of three phases: Intelligence, Design and
Choice (Simon, 1961, 1997).
Due to these factors, the evaluation tasks of GDSS design and
GDSS value are more difficult than for DSS, possibly explaining
the few integrative design and value evaluation frameworks
proposed for GDSS (Huber, 1984b; Jelassi & Beauclair, 1987; Gray
& Mandviwalla, 1999). In contrast, non-integrative GDSS design
and value evaluation frameworks are more abundant in the
literature (Limayem et al., 2006). These studies have been primarily
focused on decision-making value rather than on design issues
by proposing a variety of group decision performance metrics
(Stevens & Finlay, 1996; Fjermestad & Hiltz, 1998-99, 2000-01).
Performance metrics have been grouped as metrics regarding
process efficiency, process consensus, process satisfaction,
process effectiveness, and system usability (Fjermestad & Hiltz,
1998-99, 2000-01). Other authors (Stevens & Finlay, 1996) have
also grouped the performance metrics as substantive outcomes
(individual and organizational) and process performance
indicators; however, the literature has not provided an integrative
GDSS design and value evaluation framework that includes both
GDSS user and designer concerns. Furthermore, a standardized
group-based decision-making process model derived from
the main GDSS evaluation frameworks has not been reported.
With this situation, GDSS designers are faced with disperse
literature to select suitable design building-blocks to be included
in the projected GDSS. Similarly, GDSS users/implementers
interested in determining the expected value provided by the
system are faced with partial viewpoints that do not link design
characteristics with process-outcomes issues.
In contrast, several design and value frameworks have
been reported for DSS (Sprague, 1980; Sage, 1981; Huber, 1981;
Phillips-Wren, Hahn & Forgionne, 2004; Mora et al., 2005;
Phillips-Wren et al., 2009). In particular, the last two frameworks
are integrative by combining the criteria from a user viewpoint
(Forgionne, 1999; 2000; Phillips-Wren et al., 2004) with the
criteria from a design engineering viewpoint (Mora et al.,
2005). The user’s viewpoint concerns the expected or achieved
impacts on the decision-making process and outcomes. A design
engineering viewpoint concerns the services, the architectural
Engineering Management Journal Vol. 26 No. 2
capabilities, and the computational mechanisms projected or
existent in the system. With an integrative view, DSS designers
and users can evaluate the designs and value of two or more
projected or existent DSS applications to determine their best
architectural design (an à priori evaluation), or their provided
value (an à posteriori evaluation), through the evaluation of their
impacts on decision-making process and outcomes.
In this article, an integrative GDSS value evaluation
framework is proposed that considers the concerns of both GDSS
users and designers. Outcomes from a group-based decision
can be evaluated similarly to individual decisions through a
single effectiveness criterion, e.g., achievement of the goal or
through multiple criteria, e.g., group performance, organizational
effectiveness, and organizational efficiency (Stevens & Finlay,
1996). The group-based decision-making process can be also
evaluated by objective metrics, e.g., average time for reaching
consensus; number of generated feasible alternatives, among
others; and socio-political considerations, e.g., members’
satisfaction and support of the group decision, among others
(Turoff & Hiltz, 1982). In turn, design engineering issues can
be associated with GDSS services, architectural capabilities,
and computational mechanisms similar to DSS design
engineering issues.
This article proposes a GDSS-value evaluation framework
by integrating the dispersed foundational research on GDSS
design and evaluation. A conceptual design research method is
used (March & Smith, 1995; Hevner et al., 2004). The model is
implemented with an Analytic Hierarchy Process (AHP) model
(Saaty, 1990; 1994), and its utilization is illustrated with evaluation
of three real GDSS applications. This integrative framework can
be used by engineering managers as a tool to compare the overall
value of existing or projected GDSS with the ultimate aim of
selecting the most adequate GDSS for the decision problem. Also,
engineering managers coordinating a GDSS development project
can use this framework to make design decisions.
The remainder of this article continues with a brief
description of the conceptual design research method. Next, we
report the integrative GDSS value evaluation framework and
illustrative applications. Finally, this article presents a discussion
of implications for GDSS designers and users, recommendations
for further research, and limitations of this study.
Research Methodology
This research uses a conceptual design research method (March
& Smith, 1995; Hevner et al., 2004) based on three criteria: (1) it
provides a framework to elaborate a new conceptual or physical
artifact through a systematic research process; (2) it is adequate
to treat the complex conceptual components to be analyzed
and synthesized; and (3) it responds to recommendations and
critiques from GDSS research pioneers (Gray & Mandviwalla,
1999) on the need to conduct GDSS research with an engineering
perspective. A conceptual design research method provides
principles that can be used to organize existing knowledge on
group decision-making processes and their associated GDSS
design and evaluation frameworks.
In a few words, conceptual design research is a research
method useful for building/designing and evaluating non-trivial,
non-naturally available, and non-existent artifacts needed for
human beings (March & Smith, 1995). Design research artifacts
can be constructs, models, methods, or system instantiations
(Hevner et al., 2004). Build activity responds to the general inquiry:
is it feasible to build an artifact X by using a design approach Y?
June 2014 25
 Evaluate activity responds to the inquiry: does the artifact X fulfill
the design range of a set M of expected metrics? The conceptual
design research steps used in this research were: knowledge gap
identification, methodological knowledge selection, conceptual
design, design data collecting, and analysis and synthesis. In the
knowledge gap identification step, the research objectives, that
guide the research, the knowledge gaps that motivate the research,
and the conceptual units of study, e.g. selected research articles
to be analyzed, are defined. In the methodological knowledge
selection step, an exploratory or full design purpose is confirmed,
and the type of design method (heuristic, analytic, or axiomatic)
is defined. In the conceptual design step, the core designed artifact
is elaborated through an iterative heuristic, analytic, or axiomatic
design method. In the design data collecting step, initial validity
evaluations are conducted (comparisons with related competitive
designs, face validity tests, conceptual what-if analysis). Finally,
in the analysis and synthesis step, the contributions from the
conceptual designed artifact are derived. Through analysis
(direct insights) and synthesis (emergent insights) of findings the
designed conceptual artifact is realized.
Downloaded by [Deakin University Library] at 12:42 12 August 2015
In this research, the five aforementioned steps have been used
to elaborate the integrative GDSS-value evaluation framework.
This framework includes, as an intermediate result, a standardized
group-based decision-making process model. An additional
product of this integrative framework is its implementation with
the analytic hierarchy process (AHP) (Saaty, 1990; 1994). AHP is
a multiple attribute decision method that provides a quantitative
evaluation of a set of alternatives using a value structure defined
by the user (Saaty, 1990; 1994). The value structure is a hierarchy
with a single goal at the top level, with one or more sub-levels
containing criteria (that might be weighted) as determined by the
user. Hence, the structure reflects the judgment of the user as to
the goal, sub-goals, and criteria of importance, along with their
relative importance. The contributions of each criterion in each
level toward the criteria in the next upper level are computed on
comparative paired judgments between alternatives entered by
the user at the lowest level. Alternatives (in our case alternative
GDSS applications) are compared using qualitative judgments
that are mapped to quantitative data. Each comparative
assessment uses a ratio scale from 1 to 9 where the values 1, 3, 5,
7 and 9 correspond to equal, moderate, strong, very strong and
extreme importance for an alternative X versus an alternative Y,
based on its contribution to the specific upper level criterion Z
on the lowest level. The values 2, 4, 6 and 8 are for intermediate
values. A consistency ratio (CR) is calculated for judgments
across all pairwise comparisons of alternatives (Alonso & Lamata,
2006). It is expected that this value will be small (no greater than
0.10). If the CR value is less than 0.10, adequate consistency is
achieved between comparisons. If the CR value is greater than
0.10, pairwise comparisons need to be adjusted. Several AHP
evaluation frameworks have been reported for DSS (Phillips-
Wren et al., 2004; 2009) as well as for other type of systems (Ngai
& Chan, 2005).
Elaboration of an Integrative GDSS-Value Evaluation
Framework
A Standardized Group-based Decision-Making Process
Exhibit 1 shows the 20 GDSS research articles that were selected as
units of study. These studies were selected based on Pervan’s study
(1998) by any of the following criteria: (1) the paper is widely
cited, e.g., at least 50 scholarly citations using Google scholar;
(2) the paper summarizes GDSS research focused on design
26
issues as reported by leaders in the field; and/or 3) the paper
reports innovative GDSS design issues. The papers are grouped
additionally by GDSS foundations, GDSS with a political-
negotiation view, GDSS survey studies, and GDSS innovations.
Space limits preclude reporting a detailed analysis of these
20 articles (available upon request to the authors); however,
these 20 selected papers provide comprehensive and primary
literature useful to elaborate an integrative GDSS value evaluation
framework. The analysis of these core units of study produced
four categories of design parameters: DP.1 GDSS process, DP.2
GDSS functions, DP.3 GDSS mechanisms, and DP.4 GDSS
attributes. Three core design effects were also identified as
follows: transformation of a GDSS process, transformation of
GDSS capabilities, and transformation of both GDSS process and
capabilities. Exhibit 2 shows the list of the four identified design
parameters (DP.1 to DP.4) with their respective parameter values
and their identified core design effects.
By applying an iterative heuristic design process, the two
first authors developed the artifact [ART.1]: a high level group-
based decision-making model. This iterative design applied the
design parameters DP.1 and DP.2 several times on a previously
reported generic decision-making process for individual decision
makers identified here as artifact [ART.0] (Mora et al., 2005).
Each transformation was analyzed to be logically congruent by
the first two authors. The third author of this article realized an
additional verification for its logical congruence. Next, an overall
agreement was realized by the three authors. An additional
external validation of an earlier version of the artifact [ART.1] was
presented to several recognized GDSS international researchers.
We collected recommendations and critiques from six of the nine
contacted GDSS experts. Their expert opinions are included in
this article.
Exhibit 3 reports the results of the second design iterative
transformation. The most critical values considered were: {v3,
v6, v8, and v4} for design parameter DP.1, and {v4 and v1}
for design parameter DP.2. The first parameter values account
for a shared group-based decision-making process model,
from a group problem recognition activity to a group decision
activity. The second set of parameter values accounts for core
functionalities expected in a GDSS. In particular, the concept
of a thinkLet (Briggs, de Vreede & Nunamaker, 2003) is used
to provide structure and guidelines on how the group decision-
making process can be conducted and supported by a GDSS. A
thinkLet (Briggs et al., 2003) is a script based on best practices
of well-trained facilitators in the use of a technology (tool)
that has the following characteristics: a specific goal, a set of
steps to be followed for reaching these goals, and a specified
configuration of the technology (e.g., in the case of GDSS,
a list of specific capabilities). These authors (idem, 2003)
identified five types of thinkLets from GDSS facilitators: (1) for
divergence of concepts (moving from few to more concepts);
(2) for convergence of concepts (moving from more to few or
one unique concept); (3) for organization of concepts (moving
from a not well-understood situation to a better understood
situation); (4) for evaluation of concepts (moving from not
known and understood consequences of actions to known
and better understood consequences); and (5) for building of
consensus (moving from low to high or minimal expected level
of agreement). From this conceptual artifact [ART.1], e.g., the
high-level group-based decision-making process, a subsequent
[ART.2] conceptual artifact emerged after further analysis on
the structure of phases, activities, and their logical interactions.
Engineering Management Journal Vol. 26 No. 2 June 2014
 Exhibit 1. Seminal GDSS Research Studies Used to Develop an Integrative Group Decision-Making Process

ID GDSS Study Article Selection Criteria

i ii iii
Studies on GDSS Foundations
1 Sage (1981) 200+ citations YES -
2 Huber (1984b) 400+ citations - -
3 DeSanctis and Gallupe (1985) 200+ citations - -
4 DeSanctis et al. (2008) - YES -
Studies on GDSS Political Viewpoints
5 McGrath and Kravitz (1982) 100+ citations - -
6 McGrath (1984) 2,000+ citations - -
7 Straus and McGrath (1994) 500+ citations - -
8 Straus (1999) 50+ citations YES -
9 DeSanctis and Gallupe (1987) 1,800+ citations - -
Studies on GDSS Surveys
Downloaded by [Deakin University Library] at 12:42 12 August 2015

10 Fjermestad and Hiltz (1998-1999) 600+ citations YES -

11 Fjermestad and Hiltz (2000-2001) 200+ citations YES
12 Dennis et al. (1998) 600+ citations YES -
13 Nunamaker et al. (1996-97) 500+ citations YES -
14 Sage (1991) - YES -
Studies on GDSS Design Issues
15 Gray and Mandviwalla (1999) - YES YES
16 Briggs, de Vreede and Nunamaker (2003) 300+ citations - YES
17 He, Song and Zhao (2003) - - YES
18 Aken (2006) - - YES
19 Wu, Zhou and Yu (2008) - - YES
20 Rosanty, Dahlan and Hussin (2012) - - YES

This conceptual artifact [ART.2] is summarized in Exhibit 4. Its
description proceeds as follows:
• GROUP AGENDA ELABORATION PHASE: This phase
and the intelligence phase of an individual decision-making
process share a purpose of finding the main elements that
a decision team accepts for analysis in the next phase. In
groups, the next phase could involve different paths. Two
sets of thinkLets are posed: (1) thinkLets for guiding the
group to the acknowledgement of a problematic situation
that must be addressed by the group, and (2) thinkLets for
guiding the group to a highly valued set of relevant issues to
be considered in the decision problem (i.e., goals, solutions,
pieces of solutions, restrictions, resources, and so on).
• GROUP NEGOTIATION PHASE: In this phase, a GDSS must
provide suitable functionalities so that inherent discussions
caused by conflicts of cognition (i.e. differences in mindsets
of decision makers, or of interest/motive, i.e. differences
of the affectation and benefits of involved parties) can be
mediated effectively and efficiently. Resolving a conflict
should be a highly valued outcome, as well as a definition
of decision schemes to be evaluated. For conflict resolution,
we pose two sets of thinkLets: (1) thinkLets for the group
to organize relevant issues previously identified, and (2)
thinkLets for conducting required negotiations within the
group. This phase is proposed to be realized before the group
performs group evaluations, but it may also be realized after
Engineering Management Journal Vol. 26 No. 2
such an activity, especially when conflicts and disagreements
still exist or emerge in the evaluation phase.
• GROUP EVALUATION PHASE: In this phase, a GDSS assists
in finding the right answer to analytic problems, i.e. McGrath’s
intellective tasks, or the best team-preferred course of action
when there is not a unique, correct response, i.e. McGrath’s
preference tasks. Thus, an effective and efficient GDSS must
provide procedures to solve (optimize) or resolve (satisfy)
a group decision situation. In this phase, an initial, highly
valued outcome is either an optimal analytic response or a
non-optimal but acceptable solution. We suggest two types
of thinkLets: (1) thinkLets for achieving the establishment
of group preferences, and (2) thinkLets for supporting the
group toward an analytic solution. After this phase, group
negotiations might still be required. In this case, the previous
phase and this one must be repeated until a satisfactory set
of solutions can be reached within timely and economically-
based restrictions.
• IMPLEMENTATION AND KNOWLEDGE PRESERVATION
PHASE: In this phase (taken from extended decision-making
processes for individuals), groups can develop competitive
or non-competitive realization actions (McGrath, 1984). In
the first case, implementation acts are the co-responsibility
of the team, and the availability of resources is assigned by
organizational contest. In the second case, implementation
acts are also the co-responsibility of a team, but resources
June 2014 27
 Exhibit 2. GDSS Design Parameters

DESIGN PARAMETER TYPE OF DESIGN EFFECT

DP.1 GDSS Process On GDSS Process
Values for DP.1
v1 = {Formulation (determination of needs and restrictions, determination of objectives and metrics, determination of courses of action);
Analysis (determination of impacts, adjustment of parameters); Interpretation (evaluation of courses of action, agreements and commitments
on selected decision(s))}
v2 = {Problem Sensing, Problem Exploration, Problem Definition, Criterion and Constraint Identification, Proposal Generation, Proposal
Evaluation, Choice, Implementation, and Performance Evaluation}
v3 = {idea generation, participation, decision making, conflict management, influence and process management}
v4 = {Generate Phase (task 1: planning, task 2: creativity actions); (ii) Choose Phase (task 3: intellective actions; task 4: preference actions); (iii)
Negotiate Phase (task 5: cognitive-conflict actions; task 6: mixed-motive actions); and (iv) Execute Phase (task 7: competitive task; task 8: non-
competitive tasks)}
v5 = {Generate Phase (task 1: planning, task 2: creativity actions); Choose Phase (task 3: intellective actions; task 4: preference actions);
Negotiate Phase (task 5: cognitive-conflict actions; task 6: mixed-motive actions) }
v6 = {session planning; idea generation; idea organization; evaluation; issue exploration; knowledge preservation}
Downloaded by [Deakin University Library] at 12:42 12 August 2015

v7 = {generate basic alternatives, evaluate the alternatives regarding decision criteria, assess uncertainty on individual preferences, aggregate
individual preferences for a group outcome, ends when the consensus is high (a minimal value is prefixed) or start negotiations and return to
previous steps}
v8 = {Identification (recognition, diagnosis), Development (design OR search and screen), and Selection (judgment OR analysis OR
bargaining)}
v9 = {gathering of group opinions on a specific decision problem, analysis of consistency of opinions, test the agreed assembly rule,
assembling group decision when rule is satisfied and ends, or return to previous steps in negative case}
v10 = {problem definition and data collection, repeat AHP performance evaluation until a satisfactory consistency index, conduct fuzzy
evaluation of alternatives, get overall ranking of alternatives, aggregate preferences using Social Choice methods, report final outcome}
DESIGN PARAMETER TYPE OF DESIGN EFFECT
DP.2 GDSS Functions On GDSS Capabilities and Processes
Values for DP.2
v1 = {information retrieval/generation, information sharing, information use}
v2 = {information communication, information deliberation, information access, negotiation and coordination}
v3 = {information retrieval, sharing and negotiation, and use and judgment tasks}
v4 = {thinkLets ( divergence, convergence, organization, evaluation, consensus)}
DESIGN PARAMETER TYPE OF DESIGN EFFECT
DP.3 GDSS Mechanisms On GDSS Capabilities
Values for DP.3
v1 = {agenda, ideas gathering/brainstorming, idea evaluation, decision aids, meeting thoughts, and utilities}
v2 = {level 1, level 2, level 3}
v3 = {electronic board system, voting, ranking, issue organizer, idea generator, agenda, split/divide screen, brain-writing, group outliner,
MCDM, AHP, topic commenter, group writer, questions}
v4 = { brainstorming, delphi, nominal group, voting, alternative evaluator }
v5 = {collaborative writing, electronic polling (voting), MCDM (weight-based methods)}
v6 = {session director, brain-writing, nominal group technique, Delphi, police formation, topic commenter, assumption surfacing, voting,
alternative evaluator, issue analyzer }
DESIGN PARAMETER TYPE OF DESIGN EFFECT
DP.4 GDSS Attributes On GDSS Capabilities
v1 = { modular, with a consistent user interface, structured, flexible, and with data portability }
v2 = {visual decision making–packaging, new analytic methods based on diagrams and pictorial representations, integration with video and
computer conferencing, and expert system(s) as the n+1st meeting participant}

28 Engineering Management Journal Vol. 26 No. 2 June 2014

 Exhibit 3. A High-Level Group-Decision Making Process

[ART.0] A Generic Decision- [ART.0] + {DP.1, DP.2 } = [ART.1] [ART.1] + {DP.1, DP.2} = [ART.2]
Making Process Iteration 1 Iteration 2
Problem Problem PROBLEM RECOGNITION
THINKLETS

retrieval and
Detection Sensing

Information
Intelligence

planning
GROUP AGENDA
Data Problem
ELABORATION
Gathering Exploration IDEAS GENERATION
PHASE
Problem Formulation Problem THINKLETS
Identification
Model Classification Criterion and IDEAS ORGANIZATION
Constraint Identification THINKLETS
Information
sharing and
negotiation GROUP
Design

Model Proposal NEGOTIATION

Building Generation PHASE IDEAS NEGOTIATION
Model - THINKLETS
Validation
Evaluation Proposal
Evaluation IDEAS PREFERENCES
Information

judgment
Downloaded by [Deakin University Library] at 12:42 12 August 2015

GROUP
use and

THINKLETS
Choice

Sensitivity -
EVALUATION
Analysis
PHASE
Selection Choice IDEAS EVALUATIONS
THINKLETS
Results Decision
Implementation

Presentation Implementation IMPLEMENTATION

THINKLETS
Task -
-

Planning
GROUP DECISION
Task - IMPLEMENTATION
Monitoring PHASE
Outcome-Process Performance
Learning

Analysis Evaluation KNOWLEDGE PRESERVATION

THINKLETS
-

Outcome-Process -
Synthesis
Group Group Session Planning GROUP GROUP SESSION PLANNING
Coordination Group Session Control COORDINATION GROUP SESSION GUIDANCE
- -
PHASE
Group Session Closing GROUP SESSION CLOSING

are already assigned. Both actions are mapped in this
research with implementation actions. Additionally, we
consider that a GDSS must support the notion of decision
tracking via learning tasks. Consequently, we pose the
following thinkLets: (1) thinkLets for supporting the group
on monitoring implemented decisions, and (2) thinkLets
for helping the group preserve and re-use the gained group
decision knowledge.
• GROUP COORDINATION PHASE: This phase is added to
the four initial phases reported in Exhibit 5. In contrast to a
linear decision-making process for individuals, in the group
decision-making process model, a group control activity
is suggested. This central task is permanent from the start
to the end of the group decision-making process. While
some group decisions may require only one iteration from
phase I to IV, more complex and conflictive group decision
situations might demand several iterations. In this phase,
three activities were identified: one for planning the group
session, one for carrying the control over the four group
decision phases, and one more for formally closing the
group session. Furthermore, this phase is responsible for the
distribution of selected and agreed tasks by the group and
the provision of GDSS tools for supporting the information
access, communication, deliberation and integration. Hence,
this integrative high-level group decision-making process
poses the consideration of two types of tasks: four core
activities for making a group decision and one coordination
activity. It is expected that both types of activities can be
supported by information technology mechanisms included
in the GDSS.
Analysis of GDSS Mechanisms
The analysis of the DP.3 GDSS mechanisms identified eleven
categories of mechanisms. Most GDSS mechanisms are reported
for Level 1 (communication tasks) rather than Level 2 (core
evaluation and choice tasks). This suggests a bias of GDSS studies
toward utilization for communication and negotiation rather than
structured or semi-structured decision process, as seen for single
users. Exhibit 5 maps the 11 categories of GDSS mechanisms
to the specific GDSS phases and activities for transforming the
artifact [ART.2] to the final artifact [ART.3]. It is important to
note the following issues: (1) all phases and activities need

Engineering Management Journal Vol. 26 No. 2 June 2014 29

 Exhibit 4. A Standardized Group-Based Decision-Making Process
Downloaded by [Deakin University Library] at 12:42 12 August 2015
support for retrieval, communication, and negotiation purposes;
(2) evaluation and judgment support is required only for some
activities; (3) choice and evaluation activities are reported in
two mutually exclusive paths (either group evaluation task for
situations, where an analytical model can be elaborated and
agreed upon, or a group preferences task for more conflictive
situations, where simple voting or ranking mechanism is a better
method); and (4) new mechanisms for tracking implemented
decisions and preserving gained knowledge are also suggested.
The Integrative GDSS-Value Evaluation Framework and its
Implementation on the AHP Model
In Exhibit 6, the Integrative GDSS-Value Evaluation Framework
is shown. This framework includes the group-based decision-
making process reported in Exhibit 4. This framework is
implemented by using an Analytic Hierarchy Process model
with five levels that permit a multi-attribute evaluation of GDSS
(Exhibit 6). In the first level there is a single decision goal:
the GDSS overall provided value. In the second level (GDSS
contributions) there are four criteria: process satisfaction,
process performance, outcome satisfaction, and outcome
performance. In the third level (GDSS phases) there are four
criteria: agenda, negotiation, evaluation, and implementation.
In the fourth level there are three criteria: information retrieval,
communication and negotiation services, and judgment and
evaluation services. Finally, in the fifth level (GDSS mechanisms)
there are four criteria: mechanisms A to mechanisms D.
Mechanisms A include agenda writer, idea generator, and idea
organizer. They can be considered mechanisms for information
retrieval and communication or mechanisms for the Agenda
phase. Mechanisms B include electronic board systems, shared
editor, NGT tool, and topic commenter. They can be considered
30
mechanisms for organization or negotiation, for the Negotiation
phase. Mechanisms C include Delphi and voting-ranking tools.
They are mechanisms focused on judgment tasks. Thus, they
are labeled as Mechanisms for Judgment. Lastly, mechanisms D
include MADM (multiple attribute decision making)/AHP and
simulation tools. They are focused on quantitative evaluations,
and they are labeled as mechanisms for Evaluation phase. These
five levels constitute the value hierarchy in an AHP model. The
additional last level accounts for the specific alternatives to be
evaluated: the GDSS tools.
The Analytic Hierarchy Process (AHP) is a multiple attribute
decision-making method widely discussed in the decision making
literature (Saaty, 1994; Gass, 2005). AHP provides a systematic
method for achieving a quantitative comparison of two or more
alternatives qualitatively evaluated on a set or net of criteria levels
(Saaty, 1994). Furthermore, using a computer-based tool, changes
in preferences (weights) on criteria and evaluated alternatives (e.g.
sensitivity analysis) can be easily conducted to observe effects when
some change is required. AHP can be considered a hybrid method
regarding the level of evaluation objectivity. While AHP provides
an objective and systematic quantitative calculation process to
evaluate alternatives, a subjective or inter-subjective process is
used to establish the preferences (weights on criteria). It implies
that the final overall evaluation of the same set of alternatives
being evaluated can produce different results when different users
employ AHP due to differences in their preferences. Rather than a
weakness, it is an inherent characteristic of multi-attribute decision
making process methods to express the opinions of the evaluator.
All multiple attribute decision-making methods (MADM) have
this characteristic in contrast with objective multi-goal decision
making methods (MODM), such as linear programming or
optimization techniques (Yoon & Hwang, 1995).
Engineering Management Journal Vol. 26 No. 2 June 2014
 Exhibit 5. Mechanisms for Information Retrieval, Communication and Negotiation, Evaluation and Judgment in GDSS

[ART.2] + { DP.3, DP.4 } = [ART.3]

GDSS Mechanisms for Information Retrieval, Communication and GDSS Mechanisms for Evaluation
Negotiation GDSS Support Services and Judgment GDSS Support
Services
GDSS MECH A GDSS MECH B GDSS MECH C GDSS MECH D

IDEA GENERATOR …

ELECTRONIC BOARD
AGENDA WRITER …

SIMULATION TOOLS
TOPIC COMMENTER
SHARED EDITOR …

MADM/AHP TOOLS
PHASE

IDEA ORGANIZER /

TECHNIQUE TOOL
NOMINAL GROUP
ACTIVITY

VOTING TOOL /
CATEGORIZER

RANKING …
SYSTEM

DELPHI
TOOL
GROUP PROBLEM
█ █ █ █ - - - - - - -
RECOGNITION THINKLETS
AGENDA
GROUP

GROUP IDEAS
Downloaded by [Deakin University Library] at 12:42 12 August 2015

GENERATION - █ █ █ - - - - - - -
THINKLETS
GROUP IDEAS
GROUP NEGOT.

ORGANIZATION - - █ █ █ █ █ - - - -
THINKLETS

GROUP IDEAS
- - - █ █ █ █ - - - -
NEGOTIATION THINKLETS

GROUP PREFERNCES
- - - █ - - - █ █ - -
THINKLETS
GROUP
EVAL.

GROUP EVALUATIONS
THINKLETS - - - █ - - - - - █ █

IMPLEMENTATION
- - - █ █ - - - - - █
GROUP IMP.

THINKLETS
& KP.

KNOWLEDGE
PRESERVATION - - - █ █ - - - - - -
THINKLETS

Illustration of the Usefulness of the Integrative GDSS-Value and negotiation is strongly more important that the GDSS
Evaluation Framework service of judgment and evaluation for the phases of agenda,
This illustrative case summarizes the evaluation of the value negotiation, and implementation. In turn, the GDSS service of
provided by three real GDSS applications by using the integrative judgment and evaluation is considered much more important
GDSS-value evaluation framework. The first GDSS application than information retrieval, communication, and negotiation
(GDSS.1 TeamSpirit) is a web-based group decision system GDSS service for the phase of evaluation. In the fifth level, GDSS
designed to support creative problem-solving (Chen et al., 2007). mechanisms A (for Agenda) and B (for information retrieval and
It was used by undergraduate and graduate students in a higher communication) are considered significantly more important
education institution in Taiwan. The second GDSS application than C (for judgment) and D (for evaluation) mechanisms for
(GDSS.2 Promethee) is a system used for industrial site location the GDSS service of information retrieval, communication, and
decisions (Ishizaka & Nemery, 2012). The third GDSS application negotiation. On the other hand, the mechanisms C (for judgment)
(GDSS.3 HR) was used for recruiting personnel (Shih, Huang and D (for evaluation) are considered much more important than
& Shyur, 2005). Exhibit 7 summarizes the main architectural mechanisms A (for Agenda) and B (for information retrieval and
capabilities of these three GDSS applications. communication) the GDSS service of judgment and evaluation.
A set of equal assessment of importance for the second A verbal scale of importance with nine values (from 1 for
(GDSS contributions) and third (GDSS phases) levels is used in equal importance among criterion X and criterion Y regarding
this example. For the fourth (GDSS services) and fifth (GDSS an upper criterion Z, to 9 for extreme importance) was used for
mechanisms) levels, the paired assessment of importance between the paired assessment in the value hierarchy. A similar scale from
criteria considers what specific GDSS services are more useful 1 to 9 was used for assessing the three GDSS applications. AHP
for GDSS phases, and what specific GDSS mechanisms are more generates final weights and final scores in the range from 0.000
useful for GDSS services. In the fourth level, it is assumed that the to 1.000, after the determined calculations of contributions of
specific GDSS service of information retrieval, communication, each evaluated alternative (the three GDSS applications in this

Engineering Management Journal Vol. 26 No. 2 June 2014 31

 Exhibit 6. The Integrative GDSS-Value Evaluation Framework
Downloaded by [Deakin University Library] at 12:42 12 August 2015
Exhibit 7. Architectures of Three GDSS Evaluated with the Design-Evaluation GDSS
GDSS.1 TeamSpirit
Agenda Phase
•• Variable agenda support (manage
agenda, invite participants, view agenda)
•• Idea generation / brainstorming tool
Negotiation Phase
•• Idea consolidation / Idea organization
•• Discussion Forum
•• Information sharing tool
Evaluation Phase
•• Idea evaluation / voting / ranking
•• MADM tools
Implementation Phase
•• Meeting repository
•• Web-based architecture
GDSS Mechanisms assessment
•• GDSS Mechanisms A for Agenda (agenda
writer, idea writer / generator, idea
organizer / categorizer): VERY HIGH
•• GDSS Mechanisms B for Negotiation
(electronic board system, shared editor,
topic commenter, NGP tool): HIGH
•• GDSS Mechanisms D for Judgment
(Delphi tool, ranking and voting tools):
VERY HIGH
•• GDSS Mechanisms E for Evaluation
(MADM and simulation tools): VERY HIGH
32
GDSS.2 Promethee
Agenda Phase
•• Fixed agenda support
•• Brainstorming (for goals, criteria and
alternatives)
•• Weight elicitation
Negotiation Phase
•• GAIA visualization tool
•• Hierarchical clustering of ideas
Evaluation Phase
•• Promethee MADM tool
•• Sensibility Analysis
Implementation Phase
•• Database repository
•• Network-based architecture
GDSS Mechanisms assessment
•• GDSS Mechanisms A for Agenda
(agenda writer, idea writer / generator,
idea organizer / categorizer): HIGH
•• GDSS Mechanisms B for Negotiation
(electronic board system, shared
editor, topic commenter, NGP tool):
VERY HIGH
•• GDSS Mechanisms D for Judgment
(Delphi tool, ranking and voting tools):
HIGH
•• GDSS Mechanisms E for Evaluation
(MADM and simulation tools): HIGH
Engineering Management Journal
GDSS.3 HR
Agenda Phase
•• Fixed agenda support
•• Idea generation (for criteria)
Negotiation Phase
•• Weight elicitation
•• Consensus index on criteria
Evaluation Phase
•• MADM tool (TOPSIS technique)
Implementation Phase
•• Database repository
•• Web-based architecture
GDSS Mechanisms assessment
•• GDSS Mechanisms A for Agenda (agenda
writer, idea writer / generator, idea
organizer / categorizer): MODERATE
•• GDSS Mechanisms B for Negotiation
(electronic board system, shared editor,
topic commenter, NGP tool): LOW
•• GDSS Mechanisms D for Judgment
(Delphi tool, ranking and voting tools):
MODERATE
•• GDSS Mechanisms E for Evaluation
(MADM and simulation tools): MODERATE
Vol. 26 No. 2 June 2014
 case) for each particular criterion in the lowest layer of the value
hierarchy. These contributions are propagated by each upper layer
of criteria to obtain a final score for each evaluated alternative.
Given that the assessment for the value hierarchy was fixed to
be equal in all criteria layers, the CR (consistency ratio) was 0.00
for all criteria matrices. Exhibit 8 reports the final accumulated
weights for the five criteria layers. Exhibit 9 reports the evaluation
matrix for the three GDSS applications. Assigned values are the
paired comparisons based in the initial assessment for each GDSS
tool reported in Exhibit 9. The respective CR values for the four
matrices of the GDSS tool layer are: 0.03, 0.06, 0.03, and 0.03.
Exhibits 10 to 13 highlight the evaluation results for
the three GDSS applications realized by the authors (using
an open access AHP software tool). As can be seen, GDSS.1
(TeamSpirit) received the highest decision value of 0.535,
GDSS.2 (Promethee) received 0.375 and GDSS.3 (HR) 0.090. In

Exhibit 8. Matrices of Accumulated Weights for the Five Criteria Layers

GDSS-AHP CRITERIA LAYER ACCUMULATED WEIGHTS

GDSS VALUE 1.000
Process Process Outcomes Outcomes
satisfaction Performance Satisfaction Performance
GDSS CONTRIBUTIONS 0.250 0.250 0.250 0.250
Agenda Negotiation Evaluation Implementation
GDSS PHASES 0.250 0.250 0.250 0.250
Downloaded by [Deakin University Library] at 12:42 12 August 2015

Information Retrieval, Communication Judgment and Evaluation

and Negotiation
GDSS SERVICES 0.667 0.333
A. Agenda writer, B. Electronic C. Delphi tool, D. MADM and
idea writer / board system, ranking and simulation tools.
generator, idea shared editor, voting tools.
organizer / topic commenter,
categorizer. NGP tool.
GDSS MECHANISMS 0.287 0.287 0.135 0.292

Exhibit 9. Evaluation of the Three GDSS Tools

GDSS Mechanisms A (agenda writer, idea writer / generator, GDSS.1 TeamSpirit GDSS.2 Promethee GDSS.3 HR
idea organizer / categorizer):
GDSS.1 TeamSpirit --- 3 5
GDSS.2 Promethee 1/3 --- 3
GDSS.3 HR 1/5 1/3 ---
CR = 0.03
GDSS Mechanisms B (electronic board system, shared editor, GDSS.1 TeamSpirit GDSS.2 Promethee GDSS.3 HR
topic commenter, NGP tool):
GDSS.1 TeamSpirit --- 1/3 7
GDSS.2 Promethee 3 --- 9
GDSS.3 HR 1/7 1/9 ---
CR = 0.06
GDSS Mechanisms C (Delphi tool, ranking and voting tools) GDSS.1 TeamSpirit GDSS.2 Promethee GDSS.3 HR
GDSS.1 TeamSpirit --- 3 5
GDSS.2 Promethee 1/3 --- 3
GDSS.3 HR 1/5 1/3 ---
CR = 0.03
GDSS Mechanisms D (MADM and simulation tools) GDSS.1 TeamSpirit GDSS.2 Promethee GDSS.3 HR
GDSS.1 TeamSpirit --- 3 5
GDSS.2 Promethee 1/3 --- 3
GDSS.3 HR 1/5 1/3 ---
CR = 0.03

Engineering Management Journal Vol. 26 No. 2 June 2014 33

 Exhibit 10, the contributions to the overall GDSS value from the
lower layer of GDSS mechanisms for each GDSS tool are reported.
For GDSS.1 (TeamSpirit) the contributions to the total value
of 0.535 come mainly from GDSS mechanisms A (for agenda)
and D (for evaluation), with 0.187 and 0.174, respectively. For
GDSS.2 (Promethee) the main contribution for the total value
of 0.375 comes from the mechanism B (for information retrieval
and communication) with a value of 0.192. For the GDSS.3
(HR) with a total value of 0.090, the main contributions come
from mechanisms A and D with the values of 0.030 and 0.028
respectively. In Exhibit 11, the contributions to the overall GDSS
value from the next lower layer of GDSS services for each GDSS
tool are presented. For GDSS.1 (TeamSpirit), the contributions
to the total value of 0.535 are 0.328 and 0.206, respectively, for
information retrieval and communication, and judgment and
evaluation services. For the total value of 0.375 reached by
GDSS.2 (Promethee) tool, the values are 0.282 and 0.092, and for
the total value of 0.090 of GDSS.3 (HR) tool, the values are 0.055
and 0.034. In Exhibit 12, the contributions to the overall GDSS
value from the next layer of GDSS phases for each GDSS tool are
summarized. For GDSS.1 (TeamSpirit) the contributions to the
total value of 0.535 are of 0.149, 0.128, 0.128 and 0.128 for the
Evaluation, Implementation, Negotiation and Agenda phases,
respectively. For the total value of 0.375 in GDSS.2 (Promethee),
the values are 0.075, 0.099, 0.099, and 0.099; and for the total
value of 0.090 in GDSS.3 (HR), the values are 0.024, 0.021, 0.021,
and 0.021, respectively. Finally, in Exhibit 13, the contributions
to the overall GDSS value from the next layer of GDSS process-
outcome contributions for each GDSS tool are reported. For
GDSS.1 (TeamSpirit), the contributions to the total value of

Exhibit 10. Contributions to GDSS Value from GDSS Mechanisms for Each GDSS Tool
Downloaded by [Deakin University Library] at 12:42 12 August 2015

Exhibit 11. Contributions to GDSS Value from GDSS Services for Each GDSS Tool

34 Engineering Management Journal Vol. 26 No. 2 June 2014

 Exhibit 12. Contributions to GDSS Value from GDSS Phases for Each GDSS Tool
Downloaded by [Deakin University Library] at 12:42 12 August 2015
0.535 are 0.133 for the four process-outcome criteria. For the
total value of 0.375 in GDSS.2 (Promethee), the values are 0.093
for the four criteria, and for the total value of 0.090 in GDSS.3
(HR), the values are of 0.022 for the four criteria.
The GDSS AHP evaluation model reveals that the paired
comparison of the architectural design realized in GDSS.1
outperforms the designs of the other two GDSS applications.
GDSS.1 (TeamSpirit) design can be considered moderately better
than the design of GDSS.2 (Promeethe) and notably better than
GDSS.3 (HR). The differences in architectural designs of the
three GDSS applications are not directly identified in the GDSS
process-outcome level. In this level the three GDSS contribute
similar values to the four criteria (but with a different final
value as total). In the remaining levels, differences emerge. In
the GDSS phase level, for GDSS.1 (TeamSpirit), the Evaluation
phase is better supported than others. For GDSS.2 (Promethee),
the Agenda, Negotiation, and Implementation phases are slightly
better supported than the Evaluation phase. For GDSS.3 (HR),
Exhibit 13. Contributions to GDSS Value from GDSS Contributions for each GDSS Tool
Engineering Management Journal Vol. 26 No. 2
the Evaluation phase is also slightly better supported than
the other three GDSS phases. In the GDSS services level for
GDSS.1 (TeamSpirit) information retrieval, communication and
negotiation service receive slightly better support than evaluation
service. In GDSS.2 (Promethee), the former service receives a
strongly better support than the latter one, and for GDSS.3 (HR)
the support is similar for both GDSS services.
This GDSS AHP evaluation model permits further analysis
such as trade-offs, sensitivity analysis, and uncertainty analysis,
and these are suggested for further study. Hence, we must note that
the evaluations for these three GDSS applications are illustrations
based on the preferences of the authors on the established criteria.
Other evaluators might obtain different results.
Conclusions
GDSS, like all decision support technologies, must provide value
to users and to the users’ organizations; however, in order to
achieve this aim, a well-designed GDSS must be developed. In
June 2014 35
 this article, we have provided a comprehensive and integrative
framework that can be used to evaluate the value and design of
GDSS. The evaluation model is multiple criteria, systematic, user-
centered, and semi-formalized. The model has been implemented
by using the AHP method to compare alternative GDSS
applications (already developed) or planned GDSS (designs).
This integrative GDSS-value evaluation framework provides
to the practitioner an integrative group-based decisional process
that can be used as a reference model of tasks to be supported for
a GDSS application under consideration, a five-category taxonomy
of integrated and coherent metrics for group-based decisional
process and outcomes derived from core studies, a four-level AHP
evaluation model useful for users and designers by capturing
impact and design elements, an analytical and objective procedure
to compare GDSS designs and evaluate GDSS applications by
using a multi-criteria model, and a model useful for evaluating
both traditional and intelligence-based GDSS. This article also
contributes to the GDSS literature with a theoretically-based
integration of the accumulated knowledge on group decision-
making processes used in GDSS; with a theoretically-supported
Downloaded by [Deakin University Library] at 12:42 12 August 2015
structure of what is expected by GDSS users; with a theoretically-
supported value structure, e.g. criteria and sub-criteria, to evaluate
GDSS designs and tools; with an objective causal linkage between
the GDSS designs whats (higher level criteria) and hows (lower
level criteria); and with a review of current GDSS applications and
technologies needed to build them. Three real GDSS applications
have been used to illustrate the usefulness of this evaluation
framework. These results should be motivators to advance further
empirical tests for designing GDSS. Given the inherent complexity
of designing a GDSS, we hypothesize that the utilization of a
standardized group decision-making process model and its
derived AHP evaluation model can reduce the complexity of the
GDSS design process and improve the overall quality of a GDSS
application; however, these hypotheses require further empirical
research efforts not pursued in this study.
Engineering management is concerned with a variety of
relevant group-based decision problems occurring in business,
industrial, and governmental settings, as well as the techniques,
approaches, and methods of addressing them. GDSS applications
can be used to support such decision making processes. Thus,
advancing our understanding of how to design better GDSS
and how to objectively evaluate them are worthy research and
professional goals for the engineering management discipline.
This research also has inherent limitations: it used 20 core
papers filtered by criteria based on number of citations and
quality of content useful for this research, but some important
papers might have been omitted; the analysis rounds were limited
to 15 sessions; the three GDSS applications were evaluated from
their respective articles rather than by the utilization of the actual
system; and the authors’ expertise in DSS design and evaluation
could potentially bias our viewpoint on the group-based
decision-making process and the implemented AHP evaluation
model; however, we posit that this integrative GDSS-value
evaluation framework can be used by engineering managers as a
tool to determine the overall decision-making value provided for
existing or projected GDSS, with the ultimate aim of selecting the
most adequate GDSS for a particular engineering management
group-based decision-making situation.
Acknowledgments
The authors thank the reviewers and editors whose comments
and suggestions significantly contributed to the final version of
36
this manuscript. The authors also thank the renowned GDSS
researchers consulted in the early stages of this research for their
critical and useful insights, as well as our academic institutions
for supporting our research projects.
References
Aken, A. (2006). Towards a functional GDSS: A new framework.
AMCIS 2006 Proceedings, Paper 199, pp. 1556-1565.
Alonso, J. & Lamata, T. (2006). Consistency in the analytic
hierarchy process: A new approach. International Journal of
Uncertainty, Fuzziness and Knowledge-Based Systems, 14(4),
445-459.
Bard, J.F., & Sousk, S.F. (1990). A tradeoff analysis for rough
terrain cargo handlers using the AHP: An example of
group decision making. IEEE Transactions on Engineering
Management, 37(3), 222-228.
Barkhi, R. (2001-2002). The effects of decision guidance and
problem modeling on group decision-making. Journal of
Management Information Systems, 18(3), 259-282.
Bashiri, M., & Hosseininezhad, S.J. (2009). A fuzzy group decision
support system for multifacility location problems. The
International Journal of Advanced Manufacturing Technology,
42(5-6), 533-543.
Belardo, S., & Harrald, J. (1992). A framework for the application
of group decision support systems to the problem of planning
for catastrophic events. IEEE Transactions on Engineering
Management, 39(4), 400-411.
Briggs, R., de Vreede, G., & Nunamaker, J. (2003). Collaboration
engineering with ThinkLets to pursue sustained success with
group support systems. Journal of Management Information
Systems, 19(4), 31–64.
Chen, M., Liou, Y., Wang, C., Fan, Y., & Chi, Y. (2007). TeamSpirit:
Design, implementation, and evaluation of a web-based
group decision support system. Decision Support Systems,
43(4), 1186-1202.
Dennis, A., George, J., Jessup, L. & Nunamaker, J. & Vogel, D.
(1998). Information technology to support electronic
meetings. MIS Quarterly, 12(4), 591-624.
DeSanctis, G., & Gallupe, B. A. (1985). GDSS: a brief look at a new
concept in decision support. Proceedings of the Twenty-First
Annual Conference on Computer Personnel Research, pp. 24-28.
DeSanctis, G., & Gallupe, B., (1987). A foundation for the study
of group decision support systems. Management Science,
33(5), 589-609.
DeSanctis, G., Poole, M.S., Zigurs, I., & DeSharnais, G.,
D’Onofrio, M., Gallupe, B., … Shannon, D. (2008). The
Minnesota GDSS research project: Group support systems,
group processes, and outcomes. Journal of the Association for
Information Systems, 9(10), 551-608.
Fan, S., Shen, Q., & Kelly, J. (2008). Using group decision support
system to support value management workshops. Journal of
Computing in Civil Engineering, 22(2), 100-113.
Fjermestad, J., & Hiltz, S.R. (1998-1999). An assessment of group
support systems experimental research: methodology and
results. Journal of Management Information System, 15(3),
7-149.
Fjermestad, J., & Hiltz, S.R. (2000-2001). Group support systems:
A descriptive evaluation of case and field studies. Journal of
Management Information System, 17(3), 115-159.
Forgionne, G. (1999). An AHP model of DSS effectiveness.
European Journal of Information Systems, 8(2), 95-106.
Forgionne, G. (2000). Decision-making support systems
Engineering Management Journal Vol. 26 No. 2 June 2014
 effectiveness: The process to outcome link. Information
Knowledge-Systems Management, 2(2), 169-188.
French, S. (2007). Web-enabled strategic GDSS, e-Democracy
and Arrow’s theorem: A bayesian perspective. Decision
Support Systems, 43(4), 1476-1484.
Gass, S., (2005). Model world: The great debate – MAUT versus
AHP. Interfaces, 35(4), 308–312.
Gray, P., Johansen, R., Nunamaker, J., Rodman, J. & Wagner,
G. (2011). GDSS past, present, and future. In D. Schuff, D.
Paradice, F. Burstein, D. Power, and R. Sharda (Eds), Decision
support: An examination of the DSS discipline, Springer, 1-24.
Gray, P., & Mandviwalla, M. (1999). New directions for GDSS.
Group Decision and Negotiation, 8(1), 77–83.
He, S., Song, R. & Zhao, Q. (2003) Design of uncertain group
decision support system and its application in intelligent
transportation management. Proceedings of IEEE Conference
on Intelligent Transportation Systems, pp. 1724-1729.
Hevner, A., March, S., Park, J. & Ram, S. (2004). Design science in
information systems research. MIS Quarterly, 28(1), 75-105.
Huber, G.P. (1981). The nature of organizational decision making
Downloaded by [Deakin University Library] at 12:42 12 August 2015
and the design of decision support systems. MIS Quarterly,
5(2), 1-10.
Huber, G.P. (1984a). The nature and design of post-industrial
organizations. Management Science, 30(8), 928-951.
Huber G.P. (1984b). Issues in the design of group decision support
systems. MIS Quarterly, 8(3), 195-204.
IEEE Editorial. (August 1990). Research and education
characteristics of the engineering management discipline.
IEEE Transactions on Engineering Management, 37:3, 172-176.
Ishizaka, A., & Nemery, P. (2012). A multi-criteria group decision
framework for partner grouping when sharing facilities.
Group Decision & Negotiation, 22(4), 1-27.
Jelassi, M.T., & Beauclair, R.A. (1987). An integrated framework
for group decision support systems design. Information &
Management, 13(3), 143-153.
Kotnour, Timothy, and Farr, John V. (2005) Engineering
management: Past, present, and future. Engineering
Management Journal, 17(1), 15-26.
Kraemer, K., & King, J. (1986). Computer-based systems for
cooperative work and group decision making: Status of use
and problems in development. Proceedings of Conference
Computer-Supported Cooperative Work, pp. 353-375.
Limayem, M., Banerjee, P., & Ma, L. (2006). Impact of GDSS:
Opening the black box. Decision Support Systems, 42(2),
945-957.
Lu, J., Ma, J., Zhang, G., Zhu, Y., Zeng, X., & Koehl, L. (2011).
Theme-based comprehensive evaluation in new product
development using fuzzy hierarchical criteria group decision-
making method. IEEE Transactions on Industrial Electronics,
58(6), 2236-2246.
March, S., & Smith, G. (1995). Design and natural science research
on information technology. Decision Support Systems, 15(4),
251-266.
McGrath, J.E. (1984). Groups: Interaction and performance. New
York, NY: Prentice-Hall.
McGrath, J.E. & Kravitz, D. (1982). Group research. Annual
Review of Psychology, 33(1), 95-230.
Mora, M., Forgionne, G., Cervantes, F., Garrido, L., Gupta, J.,
& Gelman, O. (2005). Toward a comprehensive framework
for the design and evaluation of intelligent decision-making
support systems (i-DMSS). Journal of Decision Systems,
14(3), 321-344.
Engineering Management Journal Vol. 26 No. 2
Ngai, E., & Chan, E., (2005). Evaluation of knowledge management
tools using AHP. Expert Systems with Applications, 29(4),
889-899.
Nunamaker, J.F., Briggs, R., Mittleman, D., Vogel, D., &
Baltahzard, P. (1996-1997). Lessons from a dozen years of
group support systems research: A discussion of lab and field
findings. Journal of Management Information Systems, 13(3),
163-207.
Pervan, G. (1998). A review of research in group support systems:
Leaders, approaches and directions. Decision Support
Systems, 23(2), 149-159.
Phillips-Wren, G., Hahn, E., & Forgionne, G. (2004). A multiple
criteria framework for the evaluation of decision support
systems. Omega, 32(4), 323-332.
Phillips-Wren, G., Mora, M., Forgionne, G., & Gupta, J. (2009).
An integrative evaluation framework for intelligent decision
support systems. European Journal of Operational Research,
195(3), 642-652.
Rosanty, E., Dahlan, H. & Hussin, R. (2012). Multi-criteria
decision making for group decision support system. IEEE
e-Library, 105-109.
Rouse, W.B., & Compton, W.D. (2009). Systems engineering and
management. Information Knowledge Systems Management,
8, 231-240.
Saaty, T. (1990). How to make a decision: The analytic hierarchy
process. European Journal of Operations Research, 48(1),
9-26.
Saaty, T. (1994). How to make a decision: The analytic hierarchy
process. Interfaces, 24(6), 19-43.
Sage, A.P. (1981). Behavioral and organizational considerations
in the design of information systems and processes for
planning and decision support. IEEE Trans. in Systems, Man
and Cybernetics, 11(9), 640-678.
Sage, A.P. (1991). An overview of group and organizational decision
support systems. IEEE Control Systems, 11(5), 29-33.
Sánchez, M., Prats, F., Agell, N., & Ormazabal, G. (2005).
Multiple-criteria evaluation for value management in civil
engineering. Journal of Management in Engineering, 21(3),
131-137.
Shih, H., Huang, L., & Shyur, H. (2005). Recruitment and
selection processes through an effective GDSS. Computer
and Mathematics with Applications, 50(10), 1543-1558.
Simon, H.A. (1997). Administrative behavior: A study of decision-
making process in administrative organizations. New York,
NY: Free Press (1st ed. 1945).
Sprague, R.H. (1980). A framework for the development of
decision support systems. MIS Quarterly, 4(4), 1-26.
Stevens, C., & Finlay, P. (1996). A research framework for group
support systems. Group Decision and Negotiation, 5(4-6),
521-543.
Straus, S.G., & McGrath, J.E. (1994). Does the medium matter: The
interaction of task and technology on group performance and
member reactions. Journal of Applied Psychology, 79(1), 87-97.
Straus, S.G. (1999). Testing a typology of tasks: An empirical
validation of McGrath’s (1984) Group Task Circumplex.
Small Group Research, 30(2), 166-187.
Sun, Y., Ma, J., Fan, Z., & Wang, J. (2008). A group decision support
approach to evaluate experts for R&D project selection. IEEE
Transactions on Engineering Management, 55(1), 158-170.
Turoff, M. & Hiltz, S. (1982). Computer support for group versus
individual decisions. IEEE Transactions Communications,
30(1), 82-90.
June 2014 37
 Unal, R., Keating, C., B., Chytka, T.M., & Conway, B.A. (2005).
Calibration of expert judgments applied to uncertainty
assessment. Engineering Management Journal, 17(2), 34-43.
Wu, T., Zhou, X., & Yu, S. (2008). The research of group decision
methods for product design evaluation. IEEE e-library, 326-
329.
Zandi, F., & Tavana, M. (2010). A multi-attribute group decision
support system for information technology project selection.
International Journal of Business Information Systems, 6(2),
179-199.
About the Authors
Dr. Manuel Mora is a professor in the Information Systems
Department at the Autonomous University of Aguascalientes
(UAA), Mexico. Dr. Mora holds a BS in computer engineering
(1984) and an MSc in computer science (artificial intelligence,
1989) from Monterrey Tech, and an EngD in systems engineering
(2003) from the National Autonomous University of Mexico
(UNAM). He has published about 80+ research papers.
Downloaded by [Deakin University Library] at 12:42 12 August 2015
38
Dr. Gloria Phillips-Wren is Professor and Chair of
Information Systems and Operations Management at Loyola
University Maryland. She is co-editor-in-chief of Intelligent
Decision Technologies International Journal (IDT), immediate
Past Chair of SIGDSS, and Secretary of IFIP WG8.3 DSS. She
received her PhD from the University of Maryland Baltimore
County.
Dr. Fen Wang is an associate professor in the Information
Technology & Administrative Management Department at
Central Washington University. She holds a BS in MIS, as
well as both MS and PhD in information systems from the
University of Maryland Baltimore County. She has over ten
years of professional experience in IT management. She has
consulted for a variety of public and private organizations.
Contact: Manuel Mora, Autonomous University of
Aguascalientes, Information Systems, Ave. Universidad 940,
Aguascalientes, AGS 20131, Mexico; phone: 524499108417;
mmora@securenym.net
Engineering Management Journal Vol. 26 No. 2 June 2014