See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/242348201

Integrated performance measurement systems: An audit and development

guide

Article  in  The TQM Magazine · February 1997

DOI: 10.1108/09544789710159443

CITATIONS READS

65 1,010

3 authors, including:

Umit S. Bititci
Heriot-Watt University
149 PUBLICATIONS   4,759 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Impact of National Culture on the Design and Use of performance Measurement and Management Systems View project

Impact of National Culture on Design and Use of performance Measurement and Management View project

All content following this page was uploaded by Umit S. Bititci on 06 June 2014.

The user has requested enhancement of the downloaded file.


Techniques
Integrated
performance
measurement systems:
an audit and
development guide
Umit S. Bititci
Allan S. Carrie and
Liam McDevitt
The authors
Umit S. Bititci is the Director of the Centre for Strategic
Manufacturing at the University of Strathclyde, Glasgow,
UK. Allan S. Carrie is a Professor and Liam McDevitt is a
Research Assistant at the University of Strathclyde.
Abstract
The performance management process is seen as a closed
loop control system which deploys policy and strategy, and
obtains feedback from various levels in order to manage
the performance of the business. The performance
measurement system is the information system which is at
the heart of the performance management process and it
is of critical importance to the effective and efficient
functioning of the performance management system.
Research identifies two critical elements with respect to
the content and structure of the performance measure-
ment system: integrity and deployment. The viable
systems model (VSM) provides a framework for assessing
the integrity of the performance measurement system,
while the reference model developed for integrated
performance measurement systems provides a framework
against which performance measurement systems can be
designed and audited.
The TQM Magazine
Volume 9 · Number 1 · 1997 · pp. 46–53
© MCB University Press · ISSN 0954-478X
46
The need for integrated performance
measurement
Business performance improvements arising
from increased manufacturing integration
continue to be one of the primary competitive
issues of the 1990s. Recent research into
manufacturing systems integration (Carrie
and MacIntosh, 1992) has identified the need
for effective deployment of business objectives
down through the organization and the sub-
sequent measurement of performance in
critical areas as key elements of sustainable
competitive advantage. Other researchers
have also noted the links between perfor-
mance measures and strategic plans and/or
critical success factors of the business. The
research by Grady (1991) and Eccles and
Pyburn (1992) supports the same conclusions
drawn in the earlier research programme.
There is already considerable work being
carried out by the accounting profession on
performance measurement. Indeed, most
manufacturing organizations have extensive
performance measurement systems based on
cost and financial accounting practices.
Recent innovations, such as activity-based
costing, overcome some of the difficulties
associated with traditional methods but still
do not promote continuous improvement and
strategic orientation. Notable work has been
carried out by Kaplan (1983; 1990) and
Johnson and Kaplan (1987) in recognition of
these weaknesses. Neely (1993) summarizes
the shortcomings of the current accounting
practices with respect to performance mea-
surement in manufacturing enterprises.
Because financial measures that are cur-
rently in place are not supporting the change
process, there is a case for new styles of mea-
surement systems that are appropriate to the
needs of the modern business. Green (1991),
McNair and Mosconi (1987), Drucker
(1990) and Russell (1992) show that there is a
need for alignment of financial and non-
financial measures that fit within a strategic
framework.
Furthermore, based on research carried
out by Bititci (1993), Bititci and Swenson
(1993), Blenkinsop and Burns (1991) and
Gelders et al. (1993) there is evidence that,
even in companies where quality-oriented
performance measures are employed, these
are still being used in a manner which does
not promote integration. There have been
several cases cited where the company’s
 Integrated performance measurement systems The TQM Magazine
Umit S. Bititci, Allan S. Carrie and Liam McDevitt Volume 9 · Number 1 · 1997 · 46–53

strategy, improvement projects and perfor-
mance measures were in conflict.
In summary, the need for an integrated set
of performance measures which support
rather than contradict business objectives is
now clearly established (Bititci, 1994).
The performance management process
A paper which was presented by the authors
at the International Federation for Informa-
tion Processing – Working Group 5.7 – Work-
ing Conference in Seattle (Bititci, 1995)
asserted that performance management
should be viewed as a key business process
which is central to the future wellbeing and
prosperity of any manufacturing enterprise.
The performance management process is
the process by which the company manages
its performance in line with its corporate and
functional strategies and objectives. The
objective of this process is to provide a proac-
tive closed loop control system, where the
corporate and functional strategies are
deployed to all business processes, activities,
tasks and personnel, and feedback is obtained
through the performance measurement
system to enable appropriate management
decisions (Figure 1).
In essence, the performance management
process defines how an organization uses
various systems to manage its performance.
At the heart of the performance management
process, there is an information system which
enables the closed loop deployment and
feedback system. This information system is
the performance measurement system which
should integrate all relevant information
from the relevant systems. In this context
integration means that the performance
measurement system should enable the cor-
rect deployment of the strategic and tactical
objectives of the business as well as providing
a structured framework to allow the relevant
information to feed back to the appropriate
points to facilitate the decision and control
processes.
This information system, i.e. the perfor-
mance measurement system, to be effective in
achieving its objectives as described above,
should take account of the strategic and
environmental factors relating to the business
as well as considering the structure of the
organization, its processes, functions and
their relationships. Similarly, the effectiveness
of the performance management process
which makes use of this information system
depends on how this information is used to
manage the performance of the business.
Figure 2 summarizes this view of the perfor-
mance management process.
The structure of the performance
measurement system
The research to date revealed two critical
considerations with respect to the structure of
performance measurement systems: integrity;
and deployment
Integrity
Integrity refers to the ability of the perfor-
mance measurement system to promote
integration between various areas of the busi-
ness. Previous research on manufacturing
systems integration undertaken by the same

Figure 1 The closed loop deployment and feedback Figure 2 The performance management process and the position of the
system performance measurement system

The performance
Vision management process
F Behavioural Information
Business E issues technology
D E The performance
objectives measurement system
E D What is
P B Cultural Strategy measured?
Strategic issues Environment The
L goals A
O C Structure information
system Who uses
Y K Processes the measures?
Critical Attitudes Relationships
M success factors
E
N Reporting Responsibilities
Critical tasks
T Action plan structure How systems are
used to manage
Performance measures performance

47
 Integrated performance measurement systems
Umit S. Bititci, Allan S. Carrie and Liam McDevitt
consortium identified the value of the viable
Systems Model (VSM) (Beer, 1979; Blenkin-
sop, 1993) as a framework for assessing a
systems integrity. The theory behind the VSM
is that for any system to be viable it must have
five components. The VSM identifies these
five components as five systems. These sys-
tems are discussed in the following
paragraphs with respect to the integrity of the
performance measurement system.
System 1
This is the operational unit which produces
the goods or services. In other words it repre-
sents the productive function of the organiza-
tion. It could be interpreted that system 1
consists of business activities such as order
processing, credit control, production plan-
ning, manufacturing, shipping, invoicing, etc.
From a performance measurement systems
perspective system 1 will consist of perfor-
mance measures which objectively measure
the performance of an individual business
activity. For example:
• order processing – order processing lead-
time, order processing accuracy;
• credit control – credit control leadtime,
average time taken to register new
customers;
• production planning – plan stability;
• manufacturing – manufacturing leadtimes,
set-up times, yield, utilization.
System 2
This is the local management system which
co-ordinates the activities of operational units.
This system represents the business process
which contains the business activities of
system 1. For example, an order fulfilment
process may contain all of the business activi-
ties listed above. From a performance
measurement systems perspective, system 2
will consist of performance measures which
objectively measure the performance of an
individual business process. In other words,
the performance measure applied to the
process is a function of the collective perfor-
mance of its constituent activities. For
example, for the order fulfilment process
shown in Figure 3, performance against
measures such as order fulfilment leadtimes,
delivery performance and process cost are all
a function of the performance of the activities
contained within that process.
48
The TQM Magazine
Volume 9 · Number 1 · 1997 · 46–53
System 3
This system represents the tactical manage-
ment system which manages the operations of
systems 1 and 2 by setting targets and priori-
ties. This is the management system which is
responsible for the performance of business
processes and activities in line with the
requirements of the higher level systems.
From a performance measurement systems
perspective this is the system which deploys
the strategic policies and priorities to opera-
tional systems (system 1) as well as monitor-
ing the performance of these operational
systems.
System 4
This is the developmental system which
concerns itself with the external environment
and therefore the future. Its focus is on
improvement. This is the system which identi-
fies the changes necessary to the lower level
systems. This system, by focusing externally,
identifies the improvement gaps and sets
strategies to fulfil corporate objectives. The
performance measures used in this system
tend to be externally focused and compara-
tive, e.g. delivery performance with respect to
the competitors or market requirements.
It would be appropriate to combine the
external focus of this system with the results
category of the European Foundation for
Quality Management (EFQM) model for
total quality management (TQM) where the
focus is on customer satisfaction, people
satisfaction, business results and society
satisfaction. From a reference model point
of view it may be appropriate to expect
externally-oriented measures corresponding
to each one of the four areas.
System 5
This is the boss: this system sets the direction,
the corporate policy and the objectives the
organization would be adopting in the future.
With respect to the performance measure-
ment system, this system sets the corporate
priorities and targets.
The combination of systems 3, 4 and 5 is
known as the meta system, which is responsi-
ble for identifying and managing change.
Figure 3 attempts to illustrate the applica-
tion of the VSM to an organization as
described in the preceding paragraphs.
The VSM places great importance on
communications between its individual
components. It stresses that the higher level
systems amplify (i.e. deploy) higher level
 Integrated performance measurement systems The TQM Magazine
Umit S. Bititci, Allan S. Carrie and Liam McDevitt Volume 9 · Number 1 · 1997 · 46–53

Figure 3 The VSM configuration and its interpretation to the structure of a performance measurement system

The meta
system
System 5
Direction, policy and
strategy setting

System 4
Monitors the external environment
and internal performance
The external environment

Sets improvement targets

System 3
Monitors and manages
the performance of systems 1 and 2

System 2
The process

System 1 System 1
System 1 Planning Shipping
Order
processing
System 1
Credit System 1
Manufac- System 1
control Invoicing
turing

objectives in to local objectives and that The VSM requires its five systems to interact
during this deployment process the objectives and co-ordinate in a controlled fashion. The
are transduced (i.e. translated into locally explanation of the five systems and the inter-
relevant terms). The research presented in pretation of the five systems with respect to
this paper has adopted the term deployment performance measurement systems, provided
as being a function of amplification and in the previous section, suggest that the busi-
transduction. Further: ness directions and policies are set by system
5 using external information provided by
Deployment system 4, and that these policies are deployed
Deployment refers to the deployment of through systems 4, 3 and 2 to the operational
business objectives and policies throughout systems (i.e. system 1s).
the hierarchical structure of the organization However, VSM itself does not clearly
as illustrated in Figure 2. The objective of establish the exact requirements for correct
deployment, in this context, is to ensure deployment but, when it is combined with the
that: three basic objectives specified with respect to
• performance measures used at various deployment a reference model is developed. A
levels of the organization reflect the busi- pictorial view of this reference model is pro-
ness objectives and policies; vided in Figure 4.
• deployment is consistent through the
hierarchy of the organization;
The audit method
• deployment is relevant and correct with
respect to the impact and influence of Having identified and defined the key charac-
individual business areas (i.e. processes, teristics of an integrated performance mea-
functions and activities). surement system (i.e. integrity and deploy-
ment), the researchers at the University of
Strathclyde have developed and tested a
A reference model
method for auditing the integrity and deploy-
The reference model developed by the ment of the performance measurement
research team at the University of Strathclyde system as defined in the reference model. The
is based on the two facets of the performance audit process consists of three phases. These
measurement system already identified as are: data collection; integrity audit; and
integrity and deployment. deployment audit.
49
 Integrated performance measurement systems The TQM Magazine
Umit S. Bititci, Allan S. Carrie and Liam McDevitt Volume 9 · Number 1 · 1997 · 46–53

Figure 4 A reference model for integrated performance measurement systems

Integrity Deployment

System 5: Sets direction, strategy, policy Environmental requirements and

and objectives based on current performance competitive position
and the external indicators obtained from Business results, customer satisfaction,
system 4 people satisfaction, society satisfaction

System 4: Provides external intelligence Business objectives

with respect to the strategic objectives of Do the business objectives reflect the
the business. Expresses strategic objectives environmental requirements and gaps?
in measurable terms and sets targets
The
environment
Strategic measures
System 3: Deploys strategic objectives and Do the performance measured and priorities
Market, priorities to co-ordinating (system 2) and
customers, used at a strategic level reflect the business
operational (system 1) measures. Monitors objectives, environmental requirements and
people, society, and manages the performance of the process
shareholders, associated priorities?
etc.

System 2: Employs process-oriented Co-ordinating/process measures

performance measures to ensure that Do the measures and priorities at this level
individual business activities and tasks are reflect the environmental requirements,
co-ordinated effectively and efficiently business objectives, the strategic measures
and priorities?

System 1: Employs performance measures Operational measures

to measure the performance of each business Do the measures and priorities employed at
activity or task business activity level reflect the higher level
objectives, measures and priorities?

Data collection phase • The market requirements in terms of

The data collection phase, using a workbook
developed by the research team, collects all
information relevant to the performance
management process of the business. During
the early stages of the research programme it
became apparent that most businesses, small
or large, consist of discrete business units
which in turn may be treated as stand-alone
businesses. The common terminology
applied to these business units are strategic
business units (SBUs). The configuration of
SBUs within an organization could vary
dramatically.
In the first instance the data collection
phase requires the identification of the indi-
vidual SBUs and their strategic requirements
in terms of “the qualifiers” and “the differen-
tiators”. The performance measurement
system should ensure that the priorities
between the above two criteria are balanced
and appropriately managed in a dynamic
environment.
The information collected includes:
• Identification of strategic business units
(SBUs) within the business.
qualifiers and differentiators for each
SBU.
• The development plans or objectives for
the business or each of its SBUs.
• The performance measures used and
reviewed by the executive management
team within the business, i.e. the strategic
performance measures.
• The performance measures used and
reviewed within each function of the busi-
ness, i.e. the functional or operational
performance measures.
• Personal objectives and any associated
incentive schemes for the executives, man-
agers, supervisors and operational person-
nel.
• Review, reporting and performance
responsibilities associated with measures
used at all levels.
Once the data are collected they are analysed
to conduct the integrity and deployment
audits respectively. An example of the audit
process and output is provided in the follow-
ing section.

50
 Integrated performance measurement systems The TQM Magazine
Umit S. Bititci, Allan S. Carrie and Liam McDevitt Volume 9 · Number 1 · 1997 · 46–53

Performance measurement systems manner, the research team has compiled a

audit for a small engineering
manufacturing business
The purpose of this section is to illustrate the
application of the audit techniques developed
in a real life case study. The company
concerned is a traditional engineering
company manufacturing blowing equipment.
The company, although autonomous, is
part of a larger group. It employs some 100
people from a single site based in Glasgow,
UK.
checklist to search for objective evidence
supporting the existence of all five systems
within the VSM. Following the analysis of
these data against this checklist various gaps
were identified with respect to the complete-
ness and integrity of the performance mea-
surement system. An extract from the results
of the integrity audit carried out on the collab-
orating organization illustrates the nature of
the conclusions obtained from such an audit
(Table I).

Data collection The deployment audit

Data were collected using standard data
collection forms developed by the research
team. During data collection three SBUs were
identified within the business, although prior
to the audit the company did not formally
recognize these SBUs. These were:
(1) Non-competitive markets, where the
differentiator is production flexibility and
the qualifier is product quality.
(2) Competitive markets, where the differen-
tiator is price and the qualifier is once
again product quality.
(3) Spares and service business, where the
differentiator is delivery speed and the
qualifiers are price and quality.
Integrity audit
In order to conduct the integrity audit in a
comprehensive, rigorous and repeatable
In order to audit the deployment of the per-
formance measurement system a three-stage
audit method has been developed:
• Stage one focuses on the requirements of
each SBU environment and through a
series of matrices assesses the deployment
of environmental requirements through
business objectives to strategic and func-
tional performance measures.
• Stage two focuses on the business objectives
and through a series of similar sets of
matrices assesses their deployment through
the strategic performance measures to
functional levels.
• Stage three focuses on the deployment of
strategic performance measures to the
functional levels, again through the use of
similar matrices.

Table I Extracts from an integrity audit

System Description
System 5 A policy-making system is in place where the business objectives and strategy are stated in the
form of a business plan
There is no evidence of the business plan and objectives being based on external competitive
position and actual performance of the business
System 4 There is no evidence of a formal or informal mechanism for identifying the company’s current
performance with respect to the external environment
The strategic objectives are not expressed in measurable terms with associated targets – with
the exception of the objective relating to reduction of material costs
System 3 There is no evidence of a system which prioritizes and deploys the business objectives to system
2 and 1. Any deployment seems to be random (see deployment audit results)
System 2 There is no evidence of system 2/co-ordination type measures
There is no evidence of the recognition of the key business processes
System 1 There are numerous measures at this level; however, none of these are linked in any way to the
business objectives
Measures at this level do not have targets associated with them
Measures at this level are not prioritized
51
 Integrated performance measurement systems
Umit S. Bititci, Allan S. Carrie and Liam McDevitt
The audit process is designed to be rigorous
in order to ensure repeatability. A scoring
system has been devised to measure the
strength of deployment which results in a
rigorous approach and repeatable conclu-
sions. An extract from the results of the
deployment audit carried out in the collabo-
rating organization illustrates the nature of the
conclusions obtained from such an audit.
Extracts from the deployment audit:
(1) Stage 1: SBU 1 – market requirements:
• Differentiator production flexibility is:
– not deployed as a business objective;
– not deployed as a strategic perform-
ance measure;
– marginally deployed within the
manufacturing function with no
direct measure of performance and
associated target.
• Qualifier product functional perfor-
mance is:
– partially deployed as a business
objective;
– not deployed as a strategic perfor-
mance measure;
– marginally deployed at the manufac-
turing function and is not deployed
in any other function. Expected
deployment path for this measure
would have included manufacturing
and technical functions.
(2) Stage 2: SBU 1 – business objectives:
• Business objective reduce production
and inventory costs by range rational-
ization is:
– not deployed as a strategic measure;
– partially deployed in the manufac-
turing function. Expected deploy-
ment path for this measure would
have been the technical function.
(3) Stage 3: SBU 1 – strategic measures:
• Strategic measure cost is strongly
deployed across all functions. Expected
deployment path for this measure
would have included all functions.
The terminology can be defined as follows:
• strongly deployed – deployment is measur-
able with an associated target;
• partially deployed – deployment is measur-
able with no target or priority;
• marginally deployed – deployment is
indirect;
52
The TQM Magazine
Volume 9 · Number 1 · 1997 · 46–53
• expected deployment path – identifies the
functions where the reference model would
expect to see a strong deployment.
Summary and conclusions
The research and development effort is aimed
at development of practical tools and tech-
niques to facilitate more scientific and precise
management of business performance. To
this end, the first stage of the work produced
an audit method designed to assess the
integrity and relevance of the performance
measurement system used in an organization.
Figure 5 summarizes the output from the first
stage of the research.
Figure 5 Stage 1 research output from the University of Strathclyde
Audit method Reference model
Integrity audit VSM
integrity
framework
Deployment audit
Deployment
Output framework
Gaps in the
design of the Potential
performance deployment
measurement Deployment paths
system gaps
PMs by
function and by
criteria
Recommendations
More specifically the research and develop-
ment work presented in this paper may be
summarized as follows:
• The research project carried out at the
University of Strathclyde focusing on
performance measurement systems identi-
fied the need to study performance man-
agement as a process.
• The performance management process is
seen as a closed loop control system which
deploys policy and strategy, and obtains
feedback from various levels in order to
manage the performance of the business.
• The performance measurement system is
the information system which is at the
heart of the performance management
process and it is of critical importance to
the effective and efficient functioning of the
performance management system.
• Research identified two critical elements
with respect to the content and structure of
 Integrated performance measurement systems
Umit S. Bititci, Allan S. Carrie and Liam McDevitt
the performance measurement system:
integrity and deployment.
• The viable systems model (VSM) provides
a framework for assessing the integrity of
the performance measurement system.
• Within the VSM framework, deployment is
an integral element which largely corre-
sponds to the amplification and transduc-
tion elements of the VSM.
• The reference model developed for inte-
grated performance measurement systems
provides a framework against which perfor-
mance measurement systems can be
designed and audited.
• The audit methods developed to assess the
integrity and deployment of the perfor-
mance measurement system provide
simple but rigorous and repeatable tools
which may be used to improve the effec-
tiveness and efficiency of the performance
management process.
• The complicated theoretical basis of audit
methods is made transparent to the user
through the use of clearly designed work-
books which take the user step-by-step
through the process.
References
Beer, S. (1979), The Heart of Enterprise, John Wiley & Sons,
Chichester.
Bititci, U.S. (1993), Integrated Performance Measures: The
Key to Business Integration and Improvement, 9th
National Conference on Manufacturing Research,
Bath, September.
Bititci, U.S. (1994), “Measuring your way to profit”,
Management Decision, Vol. 32 No. 6, pp. 16-24.
Bititci, U.S. (1995), Performance Measurement for Perfor-
mance Management, IFIP WG5.7, Working Confer-
ence, Seattle, WA, August.
Bititci, U.S. and Swenson, H. (1993), “Use of performance
measures at strategic and operational levels”,
unpublished research report, University of Strath-
clyde, Glasgow.
Commentary
There is an old saying that “if you don’t measure it you won’t manage it”. And we know the simpler
something is to measure the easier it is to manage.
53
View publication stats
The TQM Magazine
Volume 9 · Number 1 · 1997 · 46–53
Blenkinsop, S.A. (1993), Organizational Aspects of Infor-
mation Processing Systems, PhD Thesis, University
of Loughborough, Loughborough.
Blenkinsop, S. and Burns, N.D. (1991), Performance
Measurement as an Integrating Factor in Manufac-
turing Enterprises, 7th National Conference on
Manufacturing Research, September, pp. 231-6.
Carrie, A.S. and MacIntosh (1992), “UK research in
manufacturing systems integration”, in Pels, H.J.
and Worthman, J.C. (Eds), Integration in Production
Management Systems, North Holland, Amsterdam,
pp. 323-36.
Drucker, P.E. (1990), “The emerging theory of manufactur-
ing”, Harvard Business Review, May/June,
pp. 94-102.
Eccles, R.G. and Pyburn, P.J. (1992), “Creating a compre-
hensive system to measure performance”, Manage-
ment Accounting, October, pp. 41-4.
Gelders, L., Mannaerts, P. and Maes, J. (1993), Manufactur-
ing Strategy and Performance Indicators, Proceed-
ings of Industrial Engineering and Production
Management ’93, Mons, Belgium, June.
Grady, M.W. (1991), “Performance measurement, imple-
menting strategy”, Management Accounting, June,
pp. 49-53.
Johnson, H.T. and Kaplan, R.S. (1987), Relevance Lost – the
Rise and Fall of Management Accounting, Harvard
Business School Press, Boston, MA.
Kaplan, R.S. (1983), “Measuring performance – a new
challenge for managerial accounting research”, The
Accounting Review, Vol. 58 No. 4, pp. 686-705.
Kaplan, R.S. (1990), Measures for Manufacturing Excel-
lence, Harvard Business School Press, Boston, MA.
McNair, C.J. and Mosconi, W. (1987), “Measuring perfor-
mance in advanced manufacturing environment”,
Management Accounting, July, pp. 28-31.
Neely, A.D. (1993), Performance Measurement System
Design, Theory and Practice, Manufacturing Engin-
eering Group, University of Cambridge, Cambridge,
April.
Russell, R. (1992), The Role of Performance Measurement
in Manufacturing Excellence, Conference, 27th
Annual British Production and Inventory Control
Society, Birmingham, November.