Computers in Industry 60 (2009) 104113

Contents lists available at ScienceDirect

Computers in Industry
journal homepage: www.elsevier.com/locate/compind

Quantitative relationships between key performance indicators for supporting

decision-making processes
Raul Rodriguez Rodriguez *, Juan Jose Alfaro Saiz, Angel Ortiz Bas
Research Centre on Production Management and Engineering, Polytechnic University of Valencia, Camino de Vera S/N, Edicio 8G, Acceso D Planta 1 Valencia, 46022 Spain

A R T I C L E I N F O

A B S T R A C T

Article history:
Received 20 December 2007
Received in revised form 26 June 2008
Accepted 7 September 2008
Available online 28 November 2008

Performance measurement systems (PMS) are tools widely used by enterprises for managing and making
strategy-based decisions. A PMS denes a group of strategic objectives and associated performance
indicators (KPIs) that provide information as to whether the upstream objectives are being reached or
not, but with no further information about the causes. Up to now, if an objective is not being reached
managers do not have further information regarding the causes; in terms of accurate information they are
limited to the associated KPI. However, regarding the decisions to be made: What would they be based
on? How and where to dig to nd cause-effect relationships? And, even more difcult: How to make it
objective? This study presents a unique proposal able to objectively not based neither on experience nor
subjective judgments identify and quantify relationships between performance elements dened
within a PMS, offering additional information to managers to make cross-enterprise decisions. Finally,
the paper presents the main results obtained from applying the proposal to a real world enterprise and
future research lines.
2008 Elsevier B.V. All rights reserved.

Keywords:
Decision making
Performance measurement
Relationships
Information analysis

1. Introduction
For some time ago, enterprises have been using performance
measurement systems (PMS) to try to measure their performance
and as a management tool. Using the methodology inherent to their
own PMS, analysts dene performance indicators that the enterprise
think are the most appropriate for reecting performance. It is
intended that the process of obtaining the results from these
performance indicators will be adequate in terms of costs (resources
and time). However, at this point in time, all the effort invested is
neither totally repaid nor well-used by the management. This is due
to the fact that the performance indicators are used, almost
exclusively, for measuring (with higher or lower success) the
degree of accomplishment of the dened objectives based on the
expected results. Performance indicators may additionally provide
very important information regarding the existing relationships
between them that allow the re-planning of the objectives they are
associated to, as well as improvement of the decision-making
process. Some of the most evident relationships are handled almost
unconsciously by enterprises. Furthermore, some PMS, such as the
Balanced Scorecard [1], establish some relationships when dening

* Corresponding author. Tel.: +34 645832160.

E-mail addresses: raurodro@upvnet.upv.es (R.R. Rodriguez),
jalfaro@omp.upv.es (J.J.A. Saiz), aortiz@omp.upv.es (A.O. Bas).
0166-3615/$ see front matter 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.compind.2008.09.002

an approach under four pre-established perspectives. However, lots

of inter- and intra-perspective relationships, initially hidden, stay
latent and have the potential to provide very relevant information
for any given enterprises management.
In general terms, a PMS denes some objectives derived from
strategy and one associated set of key performance indicators (KPIs),
following a top-down process. Then, once the PMS is implanted and
working, the different measures gathered by the KPIs inform
whether the upstream objectives they are associated to are being
reached or not, following a bottom-up process. Thus, when a KPI has
a value out of its allocated range, it is said that the upstream
objective to which it is associated is not being reached, without any
further information or analysis. In this sense, both researchers and
managers [26] have recently pointed out the KPI level as a source of
additional meaningful information for organisations. They argue
that analysis at this level will lead to a better understanding of
business relatedness by identifying relationships that take place
between the different KPIs. The immediate effect of discovering
these relationships would be to clearly establish possible KPI
redundancy. However, the fact of discovering relationships between
KPIs is potentially much more protable for an organisation if, once
identied, analysts projected them upstream towards the strategic
objectives and discovered the latent relationships that occur
between objectives of the PMS. Then, cause-effect relationships
between objectives could be explicated and managers would have
additional decision-making information.

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

Thus, the purpose of this paper is to present a proposal able to

quantify relationships in the performance measurement system
context (QRPMS), which objectively identies and quanties
relationships between KPIs and then projects them upstream in
the PMS, establishing meaningful cause and effect relationships at
the objectives levels.
2. Research methodology
The research followed a constructivist approach, based on the
following activities: recompilation, analysis and study of scientic
knowledge, acquisition of main postulates and construction of
initial framework [7].
The initial elements considered in the present constructivist
approach were the following:
 The PMS existing within this ambit.
 The existing techniques used within the performance measurement context that have been used or could be used for the
present research.
 The requirements that should be covered by frameworks/
techniques to address this research.
The aim to accomplish all these requirements has been the base
for creating the QRPMS framework.
This paper is structured with an initial literature review of
performance measurement, reviewing what has been already done
in this ambit for the identication and projection of relationships
between KPIs. As a result of this analysis, the research space for the
present paper was established. Next, the quantitative relationships
performance measurement system (QRPMS) proposal is described.
Then, and to best clarify all the concepts associated with QRPMS, a
practical application based on real data from a manufacturer
organisation is presented.
Finally, the most important conclusions reached, coming from
both the theoretical proposal and from the practical application,
are established.
3. Literature review
Fig. 1 illustrates the idea of introducing an activity of identifying
relationships between KPIs of a PMS and their posterior upstream
projection within a PMS.

105

Looking at Fig. 1, it seems clear is that a previous condition for

applying QRPMS is that the PMS framework dened must support
and show a clear traceability between the performance measurement elements. Researchers have developed different PMS for
managing organisations such as [811], etc. All of them coincide in
dening objectives and some sort of indicators to monitor the
accomplishment of such objectives.
However, none of the above proposals incorporate any
mechanism for identifying relationships between KPIs and posterior upstream projection. Thus, the search should focus on
techniques/frameworks that have been or could be applied at
the PMS context to objectively nd relationships between KPIs.
At the performance management context, [12] afrms that the
multi-criteria decision aid methods (MCDA) are the most
commonly used, classifying them as follows:





Objective programming.
Scoring models.
Hierarchical techniques.
Deployment techniques.

In general, MCDA establish a ranking of the different

competitive priorities for maximizing performance according to
some given criteria [12]. In general, all these techniques carry out
subjective decisions at any stage or are simply not adequate to be
applied in the context of the present work. However, there is one
framework inside the hierarchical techniques called quantitative
model for performance measurement systems (QMPMS) developed by Ref. [13], that needs to be further commented upon. This
framework provides a method for quantifying relationships
between factors that affect performance, not between KPIs.
Furthermore, it applies the Analytic Hierarchical Process (AHP)
[14] technique to quantify the established relationships, which is
another hierarchical technique and subjective. For these reasons it
is discarded for the present research.
At the performance measurement context, there are some
frameworks that aim to identify any sort of relationship between
performance elements. Thus, [15] apply a correlation analysis to
quantify relationships between pairs of KPIs dened within the
design of a PMS. This work uses the quantied relationships for
assessing one model designed to react to external changes that
might affect one organization instead of projecting them upstream
in the PMS. Additionally, the usage of correlation techniques do

Fig. 1. Identication and projection of KPIs relationships.

106

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

limit the analysis, as this technique carries out pair-wise

comparisons between pairs of KPIs instead of analyzing all the
KPIs at the same, losing the possibility of identifying crossed
relationships.
Ref. [16] focuses on establishing the relationships between KPIs
from two perspectives (internal and learning and growth) dened
inside a balanced scorecard designed according to Ref. [1]. It
establishes the relationships between these indicators by applying
the MIC-MAC method [17], which is a subjective process and
therefore will not be used in this research.
Some other works [1820] have in the past carried out different
correlation analysis in order to reduce the spectrum of KPIs to be
controlled and monitored by organisations. Again, the usage of
correlation techniques makes the approach inadequate for this
work.
Looking at the statistical literature, the multivariate analysis
has got the potential to identify relationships between variables
over time [21,22]. From these, the main a priori techniques to be
applied are: factorial analysis, principal component analysis,
structural equation models and the analysis of variance.
From all of these, the factorial analysis should be discarded
since it is necessary to specify the model, in other words, to
establish what are the subjacent variables, the observed
variables, the error terms as well as possible relationships
[21]. Additionally, the analysis of variance needs special
requirements such as independence, approximation to normal
distribution and homoscedasticity, which are, a priori, very
difcult to accomplish regarding the type of variables in this
work and associated data.
However, when assessing the other techniques, the decisional
factor has to do with the proportion of the number of variables
for the study (number of columns in the initial data matrix: the
KPIs) and the number of observations available for each of these
variables (number of rows in the initial data matrix). Thus, the
structural equation model should be only applied when
the proportion of variables/observations is, at least, 3:1 [22].
On the other hand, the principal component analysis gives good
results when this proportion does not take place and even when
there are missing values in the initial data matrix [23,24],
especially when applying a further model that uses partial least
squares (PLS) in the calculation of principal component analysis
[25,26]. Additionally, within this context [27] offers some light,
as they carried out an analysis between possible techniques to be
applied for identifying relationships between variables, concluding that the principal component analysis was the most
appropriate.
But once relationships between KPIs have been stated, it is
necessary to quantify, in magnitude and sense, such relationships
in order to assess their importance. This is one of the core
objectives of the present methodology, as the quantication of
relationships between the KPIs will allow determining their
importance depending on whether the relationship is weak (low
value) or strong (high value). This will lead, as it will be shown
later, to discriminate for the study those relationships found to be
weak. To do this, different models can be applied, though the
regression ones have probed to be appropriated, and more
concretely the partial least squares models as they overcome
the main problem of the classical regression models, the colinearity [2125]. This combination of principal component
analysis/structural equation model plus PLS model has been
already applied in several elds, even in the performance
management one, when [28] used them to identify and quantify
relationships between KPIs dened within a PMS and then using
these relationships to draw scenarios regarding an organizations
future performance.

From the above literature review it is possible to conclude that:

 There are a multitude of methods and frameworks applied to
both performance measurement and management, but almost
all make subjective considerations at any step.
In practice, there are few frameworks that consider the idea
underlying this research. The few ones whose research aim is
close to the present work stop when they nd relationship
between KPIs, not projecting them upstream in the pertinent
PMS. Besides, the method used for nding these relationships is
either inappropriate (subjective) or has got inherent limitations
(pair-wise comparisons) that makes impossible the nding of all
the relationships.
 Though not applied to this ambit extensively, the multivariate
techniques are the most appropriate. Then, down to data
characteristics, either the principal component analysis or the
structural equation model for identifying relationships between
KPIs, and the partial least square model for quantifying them are
the most appropriate methods.
 To resume, there is not any PMS framework that explicatively
incorporates any mechanism for identifying, quantifying and
projecting upstream the existing relationships between KPIs.
Thus, there is a gap within these frameworks regarding efcient
and effective guidelines and tools for analytically and graphically
managing and monitoring the identication, quantication and
upstream projection of the relationships between KPIs in the
context of the PMS. Hence, a proposal for identifying quantitative
relationships at the performance management context (QRPMS)
has been developed. This proposal aims to overcome the
detected weaknesses in the above-mentioned frameworks. The
main particularities of this proposal are described more deeply in
the next point.
4. QRPMS proposal
In the light of the previous points, this point presents the
methodology called quantitative relationship at the performance
measurement system that is the main contribution of this paper.
The main goals of QRPMS are the following:
 To be a standard methodology to be applied to any PMS that
clearly establishes traceability between their objectives and
associated KPIs.
 To identify and quantify relationships between KPIs objectively.
This is very important, as the intention is to manipulate real data
gathered by the KPIs and applying objective mathematical
techniques to thus obtain objective relationships, where no
subjective opinions can bias the achieved results.
 To construct KPI cause-effect maps based on the identied and
quantied relationships. This will directly lead to the discovery
of possible KPI redundancy when the relationship between the
KPIs is very high and then to nd out about non-evident, a priori,
relationships between the KPIs. To do this, visual support
graphics will be provided.
 To provide the necessary mechanisms for projecting these KPI
cause-effect relationships upstream in the PMS towards the
objectives ambit.
 To construct objective cause-effects maps based on such a
projection. Visual supporting graphics will be also developed in
this phase. This will provide with additional information to
decision makers in the most strategic context, as relationships
between objectives will become established and demonstrated
based on the analysis of real data.
 To identify these main KPIs whose variation could potentially
provoke the non-achievement of objectives different to the ones

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

with which they are associated. Then, an organisation could

focus its efforts on closely controlling and monitoring these
causal KPIs improving therefore its understanding about how its
PMS evolves and having a better management tool for supporting
decision-making processes.
QRPMS has got the following four phases (Fig. 2):
1.
2.
3.
4.

Design and analysis of the PMS.

Initial data treatment.
Identication and projection of KPI relationships.
Presentation and analysis of results.

Each phase has different main activities to be carried out and

they are all described and justied next.
4.1. Phase 1. Design and analysis of the PMS
In the rst step the enterprise designs and implements a PMS.
As commented above, there are different frameworks for designing
a PMS. In order to apply this proposal the only condition to be met
by the PMS used is that it provides a clear traceability between its
performance elements; in other words, it must clearly establish the
links between the objectives and the associated performance
indicators.
Once the PMS has been designed, it has to be implemented and
analysed, then decision makers should check out the consistency of
the different objectives for each perspective and associated KPIs in
order to detect possible failures and correct them.
4.2. Phase 2. Initial data treatment
In this phase the data collected by the KPIs is brought together
in order to begin its analysis. Enterprises keeping electronic
registers will much more easily complete these operations,
whereas those enterprises with low IT levels will have to invest
more resources. Once the data is available three operations will be
applied: ltering, homogenisation and centring.
The application of ltering operations will lead to the
identication any anomalous behaviour of the data that could
potentially bias the analysis. Then different statistics such as:
mean, standard deviation, mode, range, maximum and minimum
values, percentiles, histogram and normal graphic representation,
etc., could be applied to each KPI [21]. Should this analysis be
conducted by external analysts, the participation of internal
experienced personnel is necessary. This is due to the fact that
what an external analyst could consider as abnormal behaviour it
might represent a normal situation of the organisation and

107

therefore should be considered for the study instead of being

rejected.
Since the data will have been collected over different time
intervals (yearly, monthly, daily, etc.), it is necessary to homogenise it to a common frequency in order to have it all in the same
temporal frequency. At this stage, the decisional time horizon of
the analysis should be taken into account. For instance, for an
annual analysis, all the data coming from the KPIs, from all the
different time-periods, will be homogenised to a yearly frequency.
Different homogenisation operations might be applied to each KPI
depending on the sort of distribution its data presents. Then, a
simple mean will be applied for normally distributed data and
either the median or the geometric media could be applied to very
asymmetric or with high kurtosis coefcient distributions.
The last task of this phase implies the application of centring
operations, which will allocate the same degree of importance to
each variable (KPI). Since the KPIs are very heterogeneous with
regard to their measurement units (s, time, %, etc.) the centring
technique to be applied in this proposal is that of auto-scaled, as
this technique combines centring and standardisation techniques
[21].
As a result of applying this phase, the data coming from the KPIs
will form a matrix of data ltered, homogenised in frequency and
standardised, and ready for application of the proper mathematical
techniques for identifying relationships between the KPIs. This
task is carried out in the next phase.
4.3. Phase 3. Identication and projection of KPIs relationships
This phase identies and quanties relationships between KPIs
by applying adequate techniques to the initial data matrix reached
in the previous phase. The question now is: What mathematical
technique/s is/are suitable to be applied to identify relationships
between KPIs? This question was solved above, in the literature
review, which concluded that the principal components analysis
(PCA) could be, a priori, applied to any initial data matrix
regardless of the proportion between the number of variables
and the number of observations. Besides, the structural equation
model (SEM) could be applied every time that the proportion above
pointed out occurs. Consequently, and with regard to the initial
data matrix, the analyst will apply one or another of these two
techniques.
Then, an initial exploratory analysis takes place when applying
either the PCA or the SEM to all the KPIs of the study. This
exploratory analysis will identify relationships between KPIs. In a
second step, a conrmatory analysis is conducted applying again
either the PCA with least squares or the SEM, but with the
difference that PCA or SEM is applied only to these KPIs that the

Fig. 2. QRPMS methodology: phases.

108

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

exploratory analysis pointed out that kept any sort of cause-effect

relationship. This is done to conrm the results of the exploratory
analysis and it only takes into account related KPIs, with the aim to
avoid the possible noise that other non-related KPIs could
introduce into the system. Finally, those KPIs maintaining
cause-effect relationships are identied and they are named
Business Drivers Key Performance Indicators (BDKPI) due to the
importance they have in the management and evolution of the
enterprise.
Since it will be possible to apply the PCA with least squares
technique in most of the cases, regardless of the characteristics of
the data, as was stated in the literature review, the process of
identifying the BDKPIs by applying the PCA is highlighted next.
The results achieved from applying the PCA are graphically
interpreted. First, the analyst needs to decide how many principal
components to retain for the study. This is done based on the
percentage of variability that the different principal components
explain, applying the called Kaisers criterion [29]. Analysis of the
principal components retained for the study is made pair-wise, and
there the BDKPIs will be the ones that best explain the total
variance, all these contained within the two concentric ellipses in a
correlation loadings graphic.
Once all the BDKPIs have been identied it is time to quantify, in
magnitude and sense, all the identied relationships. To this end,
partial least squares models are applied, as justied above. This
model comes represented by a typical regression equation, which
shows how good the model is in predicting either one effect
variable from one or more cause variables (called PLS1 models) or
more than one effect variable from more than one cause variable
(called PLS2 models). At this stage, analysis has to decide which PLS
models they want to construct, as they need to specify what the
cause variables are and what the effect ones are. However, analysts
have an important source of information in the correlation
loadings matrixes, as these variables, symmetric by one axis or
in diagonal through the centre (intersection point of both principal
components), are willing to maintain the highest correlation [30].
Additionally, though not least important, this design of PLS models
should be conducted with the enterprises managers, as they, in the
light of the resulted BDKPIs, will be more interested in quantifying
specic relationships between BDKPIs than in other. Consequently,
the design of PLS models is an important task to be developed
conjointly by external analysts (if any) and enterprise managers.
Then, with the completion of this phase, all important causeeffect relationships between KPIs have been identied and
quantied. Then, the Graphic of the BDKPI relationships can be
drawn. Such a graphic is included in the next and last phase of the
proposal.
4.4. Phase 4. Presentation and analysis of results
This phase builds on the work carried out in the previous ones.
Then, it rst shows a Graphic of BDKPI relationships, which
represents all the cause-effect relationships between BDKPIs,
indicating the sense and the intensity of the relationship. This
intensity will be classied as either very strong or strong-average
and represented by a continuous or discontinuous arrow
respectively. Additionally, all the BDKPIs are presented within
the perspective they belong to, which makes easier their posterior
projection upstream and also gives a more complete global picture
of the different relationships. Looking at this graphic, analysts can
study the different cause-effect relationships and conrm
expected relationships, which have been demonstrated with this
study, and also about non-expected relationships that may arise. At
this stage, redundancy between BDKPIs might be identied as well
as to clearly identify what the main BDKPIs causes are.

Once the Graphic of the BDKPI relationships has been

constructed, it is time to project the identied cause-effect
relationships upstream in the PMS. Then, a new graphic, called
Graphic of projections deployment, is built from the Graphic of
BDKPIs relationships. Since the traceability between the objectives
and their associated BDKPIs is clear and unequivocal, the modus
operandi focuses on substituting, in the Graphic of BDKPI
relationships, each BDKPI for its associated objective, maintaining
the relationships. Consequently, the Graphic of projections
deployment will show the cause-effect relationships between
objectives, and therefore analysts will either conrm suspected
relationships they had conducted based on experience, or to
become aware of new relationships that take place within the
organisation.
Thus, supported by the quantied cause-effect relationships
between objectives, analysts have available new additional
information about the behaviour of the organisation, and how
important relationships take place between their main strategic
objectives. Without doubt, this new source of information, created
following objective analysis of real data of the organisation will
open new alternatives to managers when making decisions. Then,
for instance, when one objective is not being reached, looking at
the Graphic of projections deployment, managers will be able to
quickly nd out whether this specic objective (effect objective)
has being inuenced by another (cause objective) or not. If so, it
will be possible to investigate whether the action of this cause
objective is the one responsible for this non-accomplishment of the
objective effect, by analysing in the Graphic of BDKPIs the
relationships which are the corresponding BDKPI effect and BDKPI
cause. From these two, the BDKPI cause will be called Causal
business driver key performance indicator (CBDKPI), and the
organisation will have to put special emphasis on monitoring and
controlling all these CBDKPIs, as changes of the values of them may
lead to variations of other BDKPIs with the consequent domino
effect upstream in the PMS.
As a resume, QRPMS is a generic methodology, as it can be used
by any organization whose PMS has traceability between the
objectives and associated KPIs. Additionally, QRPMS is able rstly
to objectively identify cause-effect relationships between the KPIs
and secondly to project such relationships towards the objectives
level. Moreover, and due to the operations carried out in phase 2 of
the methodology, it is possible to afrm that the QRPMS can
identify and quantify objectively relationships between variables,
KPIs, independently of the nature of the data gathered by these
KPIs (percentage, money, numbers, etc.). On the other hand, the
data gathered by the KPIs will come dened in the PMS, according
to their associated objectives, and it will be related to any process
that the enterprise is interesting in monitoring, controlling and
measuring, as derived from its strategy.
Then, QRPMS helps to identify objectively (based on the
evolution of real data gathered by the KPIs) important relationships between all the elements of a PMS that before to the
application of QRPMS were unknown (latent) or objectively
indemonstrable for the organization, as managers perceived that
there were relationships between the different performance
elements, but they could only based on experience and subjective
judgments to establish them.
5. Application
QRPMS has been applied to one enterprise with 230 people that
manufactures baby clothing, in the top 5 of the Spanish market.
This enterprise had already implemented a PMS in previous years,
and in this case the application of QRPMS took around 3 months,
obtaining very interesting results, which are presented below.

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

5.1. Phase 1. Design and analysis of the PMS

At the moment of implanting QRPMS, the enterprise had
already one PMS implanted and fully working. This PMS was
constructed based on the PMS IE-GIP proposal [11].
PMS IE-GIP clearly provides traceability between all the
performance elements, drawing some supporting graphics for
representing traceability called partial deployment graphics. The
usage of this type of graphics, or similar, is highly encouraged as
they facilitate the posterior upstream projection of the identied
relationships between KPIs. Additionally, PMS IE-GIP also distinguishes between the next performance elements:
 Objectives: What to measure and control within the time horizon
for which the PMS has been designed. This is equivalent to the
classic strategic objectives of Ref. [1].
 Strategies: How to reach the dened objectives. One objective
will have one or more strategies.
 Critical success factors: This element will be constituted by the
objective it is associated to and by the strategy/ies from the ones
above dened that the enterprise wants to use at the moment of
dening the PMS. This is due to the fact that an enterprise might

109

dene, for instance, four strategies to reach one associated

objective but, at the time of dening the PMS, it might not have
the necessary resources to implement all four, having to
implement for example only two of them.
 Key performance indicators: These indicators might be dened
for measuring either one objective or one strategy.
5.2. Phase 2. Initial data treatment
In this phase the data collected by the KPIs is brought together
in order to start its analysis. Table 1 shows the set of KPIs dened
for this PMS, having a total of 42 KPIs and data corresponding to 11
years, from 1995 to 2006. Operations of ltering, homogenisation
and centring were applied to the data coming from these KPIs.
Thus, all the 42 KPIs overcame the ltering operations, as no
abnormal behaviour was detected when applying the above
pointed out statistics.
The frequency of homogenisation chosen was annual, as the
managers had dened the PMS for a 1 year term and they decided
that the results of this analysis would be more protable shown on
an annual basis. Then, for instance, the data coming from the
customer KPI CP9 (number of orders per week) was aggregated by

Table 1
KPIs of the PMS.
Financial perspective
PF1
Net benet (sales)
PF2
Turnover increment
PF3
Costs of structure
PF4
Financial indebtedness
PF5
Final stock of cloths per campaign
Customer perspective
PC1
Customer average order
PC2
Number of customers that ask for repetition/number of total customers
PC3
Number of repeating requests per model and customer/number of customers that ask for repetition
PC4
Percentage of annulled models
PC5
Efcacy of the model
PC6
Global efcacy of each collection per intermediary
PC7
Average order per intermediary
PC8
Number of new customers captured
PC9
Number of orders per week (sales speed)
PC10
Increment of customers
PC11
Number of captured customers that are already customers of other enterprise of the organisation/number of new customers
PC12
Number of non-accomplished orders/total number of orders
PC13
Number of customers that make a claim/total number of customers
PC14
Number of claims per model and customer/total number of customers that make a claim
PC15
Increment of incidences occurred between the last two years
PC16
Average reaction time to one customers claim
Internal perspective
PP1
Stock variability
PP2
Percentage of return item per model
PP3
Percentage of customer acceptation
PP4
Total number of exited models/total number of produced models
PP5
Number of clothes with deviations respect to the standard scaled
PP6
Customer accomplishment degree
PP7
Deviation between the order date and the reception date
PP8
Total number of innovative clothes/total number of produced clothes
PP9
Number of innovative successful clothes/total number of innovative clothes
PP10
Number of re-processed clothes/total number of clothes
PP11
Number of customers buying a determined style/total number of customers
PP12
Number of annulled catalogues/total number of catalogues
Learning and growth perspective
PA1
Number of critical processes totally documented/total number of critical processes
PA2
Number of critical processes controlled by the follow-up tool/total number of critical processes
PA3
Number of yearly training hours/number of yearly work hours
PA4
Hours needed of real training (detected by the enterprise)/hours needed of theoretical training (that employees think they should have)
PA5
Number of established inter-areas work/communication protocols/number of essential processes
PA6
Establishment of inter-areas work/communication protocols (Yes/No)
PA7
Number of inter-areas personnel interchanges/number of areas susceptible of interchange within the enterprise
PA8
Development of an inceptive plan (Yes/No)
PA9
Number of employees that know exactly the objective to be reached/total number of employees

110

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

Fig. 3. Correlation loadings for principal components 1 and 2.

making a simple mean and transformed to annual frequency, which

was done for all the 11 years. This operation was extended to all the
KPIs that came under other frequencies different to the annual.
Finally, centring operations were applied to the data of the KPIs
already ltered and homogenised in frequency.
As a result of this phase, an initial data matrix with 42 KPIs (in
columns) and 11 observations/years (in rows) was formed.
5.3. Phase 3. Identication and projection of KPI relationships
This phase applies a suitable technique to the initial data matrix
to identify relationships between the KPIs. In this particular case,
the proportion between the number of observations and the
number of variables is 11/42 and thus, as justied above, the
feasible technique to be applied is the principal component
analysis with partial squares.

This technique is rst applied to the initial data matrix in an

exploratory analysis, keeping then only the three rst for this
analysis, as they together explained 97.4% of the total variability of
the initial data matrix.
Then, it was possible to determine the BDKPIs, which are those
situated between the two concentric ellipses in Fig. 3 (for the pair
of principal components 1 and 2) and in Fig. 4 (for the pair of
principal components 1 and 3).
The BDKPIs obtained with the analysis are shown in Table 2. It is
important to point out that there are BDKPIs that appear in both
Figs. 3 and 4, this is because the analysis is made under a pair-wise
base, as stated above in the paper, and then the principal
component 1 is present in both gures.
Then, principal component analysis is again applied but only to
the BDKPIs in the conrmatory analysis in order to avoid the
possible noise that the other KPIs could introduce to the analysis.

Fig. 4. Correlation loadings for principal components 1 and 3.

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

111

Table 2
BDKPIs of the study.
Perspectives

BDKPIs
Principal component 1

Principal component 2

Principal component 3

Financial
Customer
Internal
Learning & growth

PF1, PF2
PC1, PC8, PC9, PC10
PP1

PF3
PC15
PP3, PP8, PP9
PA2, PA3, PA8

PF5
PC12, PC13, PC16
PP8, PP9, PP10
PL1, PL3, PL4, PL6, PL7

The results obtained concluded that the same BDKPIs were placed
again in the area of maximum variance explained, and therefore
the nal BDKPIs of the study are those presented in Table 2.
An important fact is that, from a total of 42 KPIs at the beginning
of the study, the analysis has identied relationships between 24
KPIs (the BDKPIs showed in Table 2). This is a rst important result
for managers, as the analysis allows them to establish new
relationships between indicators as well as to either demonstrate
or reject other suspected, based on experience, relationships.
At this point, PLS models are applied to quantify the relationships between the identied BDKPIs. In this regression models, it is
necessary to set what is/are the cause-effect variable/s and what is/
are the effect variable/s. Enterprise managers and analysts agreed
on the next general models:
General model 1:
Explicative (cause) variables: PF2, PP8, PP9, PP1, PC8, PC9, PC10,
PA3.
Effect variables: PF1, PF5, PC15, PP3.
General model 2:
Explicative (cause) variables: PA1, PA3, PA4, PA6, PA7.
Effect variables: PF3, PF5, PC12, PP9.
General model 3:
Explicative (cause) variables: PC3, PC7, PC9.
Effect variables: PF1, PP1.
PLS1 models and PLS2 models were conducted for all these
three general models, carrying out 13 totals PLS models, and
obtaining important results that were analysed both individually
and conjointly to have a global picture of the differing quantication of these cause-effect variables. Thus, all the relevant
relationships, those with a high regression coefcient, were taken
into account for drawing the Graphic of BDKPIs relationships,
which is shown in the next phase.
5.4. Phase 4. Presentation and analysis of results
With all the analysis carried out in the previous phases, it is
possible to build the correspondent Graphic of BDKPIs relationships (Fig. 5).
From this graphic of BDKPIs it is possible to set cause-effect
relationships between different BDKPIs, which are also quantied
in both sense and magnitude. From the results obtained in the last
phase, only those strong relationships between BDKPIs are
represented in the graph. However, inside this category of strong
relationship, two subcategories are represented in the graph with
very strong (continuous line) and strong (discontinuous line).
At this point, the projection of this Graphic of BDKPIs
relationships is carried out, by substituting each BDKPI by its
correspondent objective/strategy in the PMS (Fig. 6).
This Graphic of projections deployment shows the projection
upstream of the previous graphic of BDKPIs relationships. Analysts
can now establish cause-effect relationships between the main
strategic performance elements of the PMS: the objectives and the
strategies.

Fig. 5. Graphic of BDKPIs relationship.

For instance, and focusing on the nancial objective 1 (to

increment the net prot by 10%), FO1, this is the most important
objective for the enterprise, and this study has pointed out several
objectives and strategies that cause strong effect both directly and
indirectly, as well as positively and negatively to this FO1. All these
relationships are shown in Table 3.
Then, analysts have additional information about what
strategic performance elements are affecting FO1; these are the
following:
 PO1 ( ): This process objective of Reduction of the excess
between the produced and the sold to not exceed the 15% of the
real sales, is directly, strongly and negatively affecting FO1. This
means that, negative variations of PO1 (achievement of this
objective) will cause positive variations on the effect objective
(FO1). Looking at the BDKPIs level, this means that positive
variations of PP1, BDKPI associated to the cause objective PO1,

Fig. 6. Graphic of projection deployment.

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

112
Table 3
Relationships for the effect objective FO1.
Objective
effect

Objectives with direct

and strong relationship

FO1

PO1 ( )
LO3 (+)

Strategies with direct and

strong relationship

Objectives/strategies with indirect

and strong relationship

LO1 ( )
CS1.3 (+)
CS1.4 (+)
PS3.2 (+)
LS3.1 ( )

will directly and negatively impact PF1, BDKPI associated to the

effect objective FO1. Therefore, PP1 becomes a CBDKPI.
LO3 (+): This learning and growth objective of Development of a
customised training plan is directly, strongly and positively
affecting FO1. Positive variations of LO3 will produce positive
variations of FO1. Therefore, at the BDKPIs level, positive
variations of PL3 will lead to positive variations of PF1.
Consequently, PL3 becomes a CBDKPI.
CE1.3 (+): This customer strategy of To improve the relationship
quality-price is directly, strongly and positively affecting FO1.
Then, positive variations of CSE1.3 will lead to positive variations
of FO1, and at the BDKPIs level, positive variations of PC3 will
produce positive variations of PF1. PC3 becomes then a CBDKPI.
CS1.4 (+): This customer strategy of To set clear objectives to
intermediaries regarding the date/volume/sale affects directly,
positively and strongly FO1. Positive variations of CS1.4 exercise
positive variations of FO1. Thus, the associated BDKPI of CS1.4,
PC7, becomes a CBDKPI.
PS3.2 (+): This internal process strategy of To nd internal and
external innovation sources affects directly, strongly and
positively to FO1. This means that positive variations of PS3.2
will produce positive variations of FO1. Thus, positive variations
of PP9 will lead to positive variations of PF1, with PP9 becoming a
CBDKPI.
LO1 ( ): This learning & growth objective of To document the
critical process of the enterprise and develop specic procedures
for determined processes/work places affects indirectly,
strongly and negatively FO1. Then, positive variations of LO1
will produce negative variations of the FO1. At the BDKPI level,
positive variations of PL1 will lead to negative variations of PF1,
with PL1 becoming a CBDKPI.
LS3.1 ( ): This learning & growth strategy of To detect training
needs affects strongly, indirectly and negatively FO1. Therefore,
positive variations of LS3.1 will cause negative variations of FO1.
Positive variations of PL4 will lead to negative variations of PF1,
with PL4 becoming a CBDKPI.

To sum up, managers analysing this effect objective FO1 (to

increment the net prot by 10%) and its different cause objectives
and strategies could come to several conclusions. Thus, in order to
reach the objective FO1, the enterprise should mainly focus its
resources on reducing its manufacturing variability (PO1); fostering
customised training plans (LO3); improving their qualityprice
relationship (CS1.3), cutting costs and improving quality level;
improving relationships with its intermediaries by setting better and
clearer objectives regarding volume, date and sale data (CS 1.4);
nding internal and external innovation sources (PS3.2), as this strategy, in principle not very important, with this analysis has revealed
to be of key importance to the economic results of the enterprise.
Additionally, managers should also take into account the
negative impact on FO1 that the investment on documenting the
critical process of the enterprise and developing specic procedures
for determined processes/work places (LO1) as well as on detecting

CBDKPI

PP1
PL3
PL1
PC3
PC7
PP9
PL4

training needs (LS3.1). This negative impact on the enterprises prot

may be due to the investment needed to be made in order to put
them into practice within the enterprise. However, the impact is
important but relative as they maintained an indirect relationship
with the FO1. It would be acceptable to think that, in the future, this
negative incidence could either turn into a positive one or even
disappear once the actions they propose have been carried out.
At the indicators level, and for the FO1, mangers should mainly
focus on ve CBDKPIs: PP1, PL3, PP9, PC3 and PC7; taking also into
account the two CBDKPIs of PL1 and PL4. If the PMS reveals that
FO1 is not being reached this might be because any of these
CBDKPIs is affecting PF1 and forcing it out of its range. Managers
should then look at the evolution and the accomplishment of these
seven CBDKPIs and associated objectives/strategies.
Conducting similar analysis for all the other effect objectives/
strategies (FO3, FS1.4 and CO3) shown in the Graphic of projections
deployment (Fig. 6) the study has detected other CBDKPIs, that
added to the above presented, provide the next nal list of
CBDKPIs: PF2, PC3, PC7, PP1, PP8, PP9, PL1, PL3, PL4, PL6 and PL7.
Hence, from 42 KPIs the study has revealed that there are eleven
especially important KPIs (the CBDKPIs) to be controlled and monitored over time, as their variation will lead to variations on important
effect performance elements of the PMS, as well as to provide managers with important additional information for making decisions.
Deriving from the application of QRPMS to this enterprise, and
what it has offered to managers, it is possible to establish the
following practical comments:
 Mangers found that the application of QRPMS was, in practice, a
non-difcult task. This was due, mainly, to the action of an
external analyst conducting the analysis and to the fact that their
presence and interaction on the analysis was nonnegotiable,
becoming an active part of such an application.
 Managers of the enterprise showed their enthusiasm and
support to the results achieved by the application of QRPMS,
as they at least could have a solid framework that served to
demonstrate real interactions between KPIs and also to project
them upstream the PMS, with consequent additional information about both the PMS elements in particular and of the
enterprise in general.
 In practice, the main problem this enterprise has claimed to have
had to overcome when applying QRPMS is the fact of having all
the data coming from the indicators available.
6. Conclusions
This paper has described a new methodology called QRPMS,
which is able to objectively nd and quantify relationships
between KPIs dened within a PMS and then project them
upstream towards the strategic level of the PMS, then becoming a
management support tool providing additional information to
decision makers. After analyzing the most applied techniques and
frameworks in the performance measurement and management

R.R. Rodriguez et al. / Computers in Industry 60 (2009) 104113

area that have been or could be applied in this ambit, and having
probed the research opportunity, QRPMS was dened to offer a
solid, complementary and efcient framework to organizations to
improve their management. QRPMS can be applied to any
organization that has a PMS that assures traceability between
their performance elements (objectives, strategies and KPIs),
applying different techniques for identifying (principal component
analysis or structural equation model) and quantifying (partial
least squares) the exiting relationships between KPIs.
Among the main characteristics of QRPMS are the following:
 Conrmation of a priori, and based on experience, some intuited
relationships, quantifying them in sense and magnitude.
 Discard of a priori, and based on experience, some intuited
relationships, quantifying them in sense and magnitude.
 Objective demonstration of the existence of other important
relationships between performance elements, quantifying them
in sense and magnitude.
 Identication of the CBDKPIs, whose monitoring and control will
help to manage the enterprise more efciently, as they will offer
new additional information when associated objectives/strategies
upstream are not being reached. Additionally, managers might cut
costs, as they can focus on closely monitoring only these CBDKPIs,
reducing the number of indicators to be controlled.
 Better knowledge about their enterprise, which will directly
affect re-denition activities of the PMS as well as management
in general.
Future research should focus on enriching the proposal by
applying QRPMS to others organizations, as well as to adapt the
framework to inter-enterprises contexts, where the application of
QRPMS can help managers to make better conjointly taken decisions.
References
[1] R.S. Kaplan, D.P. Norton, The balanced scorecard. Measures that drive performance, Harvard Business Review (January/February) (1992) 7179.
[2] D. Itner, D. Larker, Coming up short on nonnancial performance measurement,
Harvard Business Review (November) (2003) 91105.
[3] J.J. Alfaro, PMS IE-GIP, Doctoral Thesis, Universidad Politecnica de Valencia, 2003.
[4] K.A. Merchant, Measuring general managers performances market, accounting
and combination-of-measures systems, Accounting, Auditing and Accountability
19 (6) (2006) 893917.
[5] C. Wolf, P. Harmon, The State of Business Process Management, Business Process
Trends, 2006 June, Accessed 10th March 2007, Available at www.bptrends.com.
[6] H.S. Jagdev, A. Brennan, J. Browne, Strategic Decision Making in Modern
Manufacturing, Springer, New York, 2003.
[7] P. Coughlan, D. Coughlan, Action research: action research for operations management, International Journal of Operation and Productions Management 22 (2)
(2002) 220240.
[8] K.F. Cross, R.L. Lynch, The SMART way to sustain and dene success, National
Productivity Review 8 (1) (1988) 2333.
[9] A.A. Atkinson, J.H. Watherhouse, R.B. Wells, A stakeholder approach to strategic
performance measurement, Sloan Management Review 38 (3) (1997) 2537.
[10] A.D. Neely, C. Adams, P. Crowe, The performance prism in practice, Measuring
Business Excellence 5 (2) (2001) 612.
[11] J.J. Alfaro, A. Ortiz, R.R. Rodriguez, Performance measurement system for enterprise networks, International Journal of Productivity and Performance Management 56 (2007) 305334.
[12] G.J.C. Da Silveira, Improving trade-offs in manufacturing: method and illustration,
International Journal Production Economics 95 (1) (2005) 2738.
[13] P. Suwignjo, U.S. Bititci, A.S. Carrie, Quantitative models for performance measurement system, International Journal of Production Economics 64 (2000) 231241.
[14] T.L. Saaty, The Analytic Hierarchy Process, McGraw-Hill, New York, 1980.
[15] A.D. Youngblood, T.R. Collins, Addressing balanced scorecard trade-off issues
between performance metrics using multi-attribute utility theory, Engineering
Management Journal 15 (1) (2003) 1118.
[16] D. Cardona Siado, A. Garcia, Modelo de Indicadores para el despliegue de la
Estrategia de Calidad, 2005, Accessed 14th June 2007, Available at http://hdl.handle.net/1992/300.
[17] M. Godet, From Anticipation to ActionA Handbook of Strategic Prospective,
UNESCO, Paris, 1993.
[18] K. Bauer, KPI: reduction the correlation way, DM Review (February issue, 2005) 13.
[19] J. Fraser, Beyond KPIs: impacting supply chain performance, Manufacturing
Business Technology (May 2005) 3435.

113

[20] I. Lange, O. Schneider, M. Schnetzler, L. Jones, in: J. Olhager, F. Persson (Eds.), IFIP
International Federation for Information Processing, Volume 246, Advances in
Production Managment Systems, Springer, Boston, 2007, pp. 379386.
[21] J.F. Hair, R.E. Anderson, R.L. Tatham, W.C. Black, Multivariate Data Analysis,
Prentice Hall International Inc., New York, 1995.
[22] J.E. Jackson, A Users Guide to Principal Components, Hoboken, New Jersey, 2003.
[23] P.P.C. Nelson, P.A. Taylor, J.F. MacGregor, Missing data methods in PCA and PLS:
score calculations with incomplete observations, Chemometrics and Intelligent
Laboratory Systems 35 (1993) 4565.
[24] S. Wold, K. Esbensen, P. Geladi, Principal component analysis, Chemometrics and
Intelligent Laboratory Systems 4 (1987) 3752.
[25] P. Geladi, B. Kowalski, Partial least-squares regression: a tutorial, Analytica
Chimica Acta 185 (1986) 132.
[26] H. Martens, M. Martens, Multivariate Analysis of Quality, John Wiley & Sons Ltd.,
New York, 2001.
[27] R. Rodriguez, J.J. Alfaro, A. Ortiz, Relationships among key performance indicators
within the performance measurement system context: literature review, in: PMA
Conference, London, (2006), pp. 689701.
[28] B. Patel, T.J. Chaussalet, P.H. Millard, Balancing the NHS balanced scorecard,
European Journal of Operational Research 185 (2008) 905914.
[29] H.F. Kaiser, The application of electronic computers to factor analysis, Educational
and Psychological Measurement 20 (1960) 141151.
[30] M. Aluja, D. Marineau, Practical Application of the Principal Component Analysis,
Palais, Paris, 1999.
Dr. Raul Rodriguez Rodriguez (Industrial Management
Engineer, MBA, PhD) is a lecturer in Operations
Management and Operations Research at the Polytechnic University of Valencia (UPVL). He has worked in
several projects about performance management,
operations management, supply chain management
and information systems in different activity sectors
such as automotive, retail, consumer goods, ceramic or
textile. Additionally, he is a full member of the CIGIP
(Research Centre on Production Management and
Engineering) at the UPVL, having participated on
several research projects at European level (ECOSELL,
GPM-SME) as well as on national research projects (INPREX, SP7). He lectures in
both graduate and undergraduate courses and his research preferences are on
performance measurement/ management, operations management and decisionmaking sciences. He is a Certied Quality Engineer and a full member of the
American Society for Quality. He has published several papers in books,
international journals and delivered some conferences in these elds.
Dr. Juan Jose Alfaro Saiz is an assistant professor in
Operations Management and Operations Research at the
Polytechnic of Valencia. He is an Engineer in Business
Organization and he received his doctoral degree in
Business Organization at the Polytechnic University of
Valencia in 2003. He is a member of the CIGIP (Research
Centre on Production Management and Engineering). He
has worked as a Researcher in several Spanish Government Projects (CICYT), one ESPRIT Project, V-CHAIN
(Virtual Enterprise for Supply Chain Management),
ECOSELL (Extended Collaborative Selling chain), and
others. He is a teacher in the Polytechnic University of
Valencia. His research interests include performance
measurement systems, supply chain management, integration enterprise and
modelling process business. He has published several papers in books, journals
and conferences in these elds.
Dr. Angel Ortiz Bas is an assistant professor in
Operations Management and Operations Research at
Polytechnic University of Valencia. He is an industrial
engineer and he received his doctoral degree in
Industrial Engineering from the Polytechnic University
of Valencia in 1998. He works as a consultant in several
projects about production management, supply chain
management, information systems and enterprise
modelling and integration in metal mechanic, ceramic
and automotive enterprises. He is a member of the CIGIP
(Research Centre on Production Management and
Engineering). He works as a Researcher in ve Spanish
Government Projects (CICYT) and one ESPRIT Project
and is co-leader of the GROWTH Project, V-CHAIN (Virtual Enterprise for Supply Chain
Management). He is a teacher at the Polytechnic University of Valencia and in the
FORD Spain Industrial Engineer School, and also teaches several Masters courses
(MBA). He is member of the IFAC/IFIP Task Force on Enterprise Integration. His major
research interests are production planning and control, supply chain management,
enterprise integration, information management, business process modelling. He has
published several papers in books, journals and conferences in these elds.