Journal of Sport Management, 2013, 27, 217-229

© 2013 Human Kinetics, Inc. Official Journal of NASSM

www.JSM-Journal.com
ARTICLE

Data Envelopment Analysis and Cross-Efficiency

Evaluation in the Management of Sports Teams:
The Assessment of Game Performance of Players
in the Spanish Handball League
Óscar Gutiérrez and José L. Ruiz
Universidad Miguel Hernández

Data Envelopment Analysis (DEA) and cross-efficiency evaluation are shown as support tools for sports team
management in the context of a study of assessment of the individual game performance of handball players
of the Spanish premier league. A sample of 66 players that play as backs in their teams is evaluated from the
perspective of their offensive game. DEA yields a measure of the overall performance of the game of the
players, and allows to identifying relative strengths and weaknesses by means of a benchmarking analysis.
The cross-efficiency evaluation has provided a peer-appraisal of the players with the different patterns of game
that the 10 players rated as efficient have used in the DEA assessments, and has made it possible to derive a
full ranking of players.

Having available tools that provide coaches and
managers of sports teams with a thorough knowledge
of the players may help them carry out their functions
in their organizations. These functions include building
the teams, organizing training programs, planning match
strategies or hiring players, and to perform them, the
information regarding the relative strengths and weak-
nesses of the own players, the opponents, and others
that can be recruited, and that concerned with their
performance against different models of game may be
deemed crucial. Data Envelopment Analysis (Charnes,
Cooper, & Rhodes, 1978) and cross-efficiency evalua-
tion (Doyle, & Green, 1994; Sexton, Silkman, & Hogan,
1986) are proposed here as support tools that may shed
light on these issues related to the game performance of
players, thus helping coaches and managers improve the
performance of their teams. We illustrate the use of these
methodologies in a study of assessment of the individual
game performance of handball players in the context of
the Spanish premier league.
Handball, as a team sport, is characterized by the
execution of the open skills in the many different situ-
ations that may arise in the context of the game. For
each of these skills, there are an infinite number of pos-
sible solutions that can be taken, which depend on the
individual player characteristics, previous experiences
Gutiérrez is with Centro de Investigación del Deporte, Uni-
versidad Miguel Hernández, Elche, Alicante, Spain. Ruiz is
with Centro de Investigación Operativa, Universidad Miguel
Hernández, Elche, Alicante, Spain.
of game, etc. Most of these actions have a different
character and this means that there are many paths to
success. The design of sport teams as an example of
dynamic systems may serve to explain the mechanisms
of self-regulation of players, which vary and adjust their
behavior according to the changes as they occur in the
development of the game. For this reason, we can find
different models of players who can achieve optimal
performance at different levels and all of them deserve
being considered insofar as they depend on the charac-
teristics of each player.
Similarly, collective play will be adjusted according
to the characteristics of both teams and players, as well as
the quality of the opponent team players (Trninić, Trninić,
& Papić, 2009). To optimize the performance of a team
it is necessary to consider the degree of dominance of
the gaming systems at the collective level, the level of
motor skills and motor performance of each player, the
relationships with colleagues and opponents, etc. That
is, we have a complex system of variables that is impos-
sible to be completely controlled from outside, so own
self-regulatory mechanisms of dynamic systems may be
suitable for better understanding. In the process of train-
ing a team, the trainer must facilitate the conditions for the
development of the player and the game’s tactical model
that optimizes the tactical performance and the technical
efficiency in game (Trninić, et al., 2009).
Therefore, it is very important to know the appropri-
ate performance levels of each player according to their
own characteristics, their level of skills and the position
they hold in the field to guide their training so that they
can exploit their strengths and improve their weaknesses.

217
218    Gutiérrez and Ruiz
Moreover, this improvement should be done selectively
depending on the player’s individual potential and the role
they play in the game system, generating some specific
fixes (Trninić, et al., 2009).
DEA is a methodology that uses Linear Program-
ming for the analysis of relative efficiency of “decision
making units” (DMUs) involved in a production process.
For each DMU, it provides an efficiency score that allows
to assessing the relative efficiency of its performance in
the use of several inputs to produce several outputs. The
DEA features we describe next suggest that this method-
ology can be very useful to address the issues concerned
with game performance discussed above.
In DEA there is no need to a priori know the weights
that are to be attached to the inputs and outputs for their
aggregation in the assessment of the DMUs. In the con-
text of the assessment of players in sports, this means
that there is no need to prespecify the importance to
be attached to the different aspects of the game when
this information is to be aggregated. The DEA models
determine such aggregation weights by maximizing
the rating of the player under evaluation, provided that
those weights are admissible for the other players. It is
worth emphasizing the fact that the DEA weights are
player-specific and that, as a consequence, each player
can exploit his strengths in the assessments. Eventu-
ally, we have a self-evaluation in which the players are
assessed with the patterns of game that show them in
their best possible light. This is in contrast to the tra-
ditional approach based on a common set of weights
for all the players, which implicitly assumes a unique
model of player or pattern of game in the assessments.
The specification of a common set of weights usually
raises serious difficulties, and it may also be questioned
the arbitrariness of the choice that is made. There are,
however, numerous studies that have attempted to
specify, in response to indications of expert coaches,
the appropriate weight for each aspect of the game. See,
e.g., the cases of football (Tenga, Holme, Ronglan, &
Bahr, 2010a, b), basketball (Swalgin, 1998; Trninić,
Perica, & Dizdar, 1999; Trninić, & Dizdar, 2000; Trninić,
Dizdar, & Dezmar, 2002) or waterpolo (Hraste, Dizdar,
& Trninić, 2008).
DEA assesses relative efficiency, which means that
the game performance of the different players is assessed
as the result of their comparison with the others. In fact,
this methodology provides a benchmarking analysis:
The players are classified into efficient and inefficient,
so the latter are assessed with respect to the former. DEA
allows us to identify the weaknesses in the game of the
inefficient players and to set efficient targets, which rep-
resent levels of performance in each aspect of the game
that would make each of them perform efficiently. Thus,
these targets can be used to suggest potential directions of
improvement for the game of each player. Like the DEA
weights, the targets are player-specific too. These result
from the selection of a benchmark that is made taking
into consideration the type of game of the player under
assessment. The key issue is that each player may have
a different way to achieve the efficiency of performance,
which will obviously depend on the own characteristics
of his game.
However, in the DEA literature it has been widely
questioned the use of the DEA efficiency scores for
ranking purposes, in particular of players, because, as
said before, the score of each player is obtained from
weights that are usually different from those of the
others. For this reason, we use here the cross-efficiency
evaluation, which is an extension of DEA that is based
on the idea of assessing each player not only with his
own weights but also with those that the others have
used in their self-evaluation. Thus, the cross-efficiency
evaluation provides a peer-evaluation of the players in
which we can evaluate the game performance of each of
them with reference to different patterns of game and,
in addition, it can be used to derive a full ranking of
players based on the resulting cross-efficiency scores.
It has also been claimed that cross-efficiency evalu-
ation may help prevent the effects of the unrealistic
weighting schemes we often find in DEA analyses as
the result of the total weight flexibility. We point out in
particular the problems with the zero weights, which
would lead to assessments of players in which many
of the game factors considered are ignored (see, for
example, Anderson, Hollingsworth, & Inman, 2002,
for discussions).
Literature Review
The DEA methodology has been successfully used in
many real world applications both in public and private
sectors, and it has also been used in the context of sports.
In particular, regarding the assessment of game perfor-
mance of players, Cooper, Ruiz, and Sirvent (2009) assess
basketball players by using the statistics of the Spanish
premier basketball league. Cooper, Ramón, Ruiz, and
Sirvent (2011) provide a ranking of basketball players
with a cross-efficiency evaluation. In Ruiz, Pastor, and
Pastor (2011), DEA and cross-efficiency evaluation are
combined for the assessment and ranking of professional
tennis players with the ATP statistics. Ramón, Ruiz, and
Sirvent (2012) also rank tennis players with a common
set of weights obtained from DEA weights. See also the
use of DEA for the evaluation of baseball players (Ander-
son, & Sharp, 1997; Chen, & Johnson, 2010; Sexton,
& Lewis, 2003; Sueyoshi, Ohnishi, & Kinase, 1999),
golf players (Fried, Lambrinos, & Tyner, 2004; Fried,
& Tauer, 2011; Ueda, & Amatatsu, 2009) and football
players (Alp, 2006).
In football, this methodology has been applied from
the point of view of the soccer teams (Boscá, Liern, Mar-
tínez, & Sala, 2009; Espitia-Escuer, & García-Cebrián,
2004; García-Sánchez, 2007; González-Gómez, &
Picazo-Tadeo, 2010; Haas, 2003, Haas, Kocher, & Sutter,
2004), the coaches (Dawson, Dobson, & Gerrard, 2000)
and that of the clubs (Barros, & Leach, 2006, Barros,
Assaf, & Sá-Earp, 2010 and Barros, & García del Barrio,
2011).
 Assessment of Players in Spanish Handball League   219
Relative efficiency in sports at the level of countries
has also been measured with DEA models, in particu-
lar for measuring the performance of the participating
nations at the Summer Olympics Games (Lozano, Villa,
Guerrero, & Cortés, 2002; Soares de Mello, Angulo-
Meza, & Branco da Silva, 2009; Wu, Zhou, & Liang,
2010; Zhang, Li, Meng, & Liu, 2009).
Finally, we can find applications of DEA analyz-
ing the efficiency in sports from other perspectives.
Fizel, and D’Itri (1996, 1997, 1999) deal with organi-
zational performance and use DEA to create measures
of managerial efficiency. Barros, and Santos (2003)
analyze the productivity of organizational sports train-
ing courses implemented by Portuguese federations.
Volz (2009) provides efficiency scores not only of
team performance but also of player salaries in Major
League Baseball and Einolf (2004) measures franchise
payroll efficiency in National Football League and
Major League Baseball.
Data and Contextual Setting
The Spanish Premier Handball League, the ASOBAL
league, is considered as one of the bests in the world,
with the best players, with teams that can sign the
best players since they have big budgets, and where
media coverage is very wide. However, in handball in
general (and in other team sports) there is a need of
more developments in the area of the assessment of
performance, along the line of those that have been
made, for example, in basketball. The information that
the ASOBAL league provides is confined to simple
rankings based on the statistics regarding the differ-
ent aspects of the game separately (shooting from a
specific distance, ball handling, defense. . . ). Thus,
this league reports on the best players scoring from
the line of 9 m or those that more goal assists give, but
we do not have available any overall assessment of the
game performance of the players when several game
factors are wanted to be considered simultaneously. To
address this issue, we propose the use of DEA here.
In addition, DEA provides a benchmarking analysis
of the players, and the cross-efficiency evaluation
makes it possible a ranking of players based on these
overall assessments.
The study in the present paper evaluates the play-
ers that have played in the back position in the different
teams of the ASOBAL league during the 2008–09 season.
It is important to note that DEA makes a homogeneity
assumption in the sense that the units under assessment
(the players in our case) should be comparable, so that
a common set of outputs can be defined (see Dyson,
Allen, Camanho, Podinovski, Sarrico, & Shale, 2001, for
discussions). For this reason, the assessment of players
with DEA in some sports is made for those playing in a
given position (see, for example, the case of basketball
in Cooper, et al., 2009, and Cooper, et al., 2011). Here,
we make an evaluation of the backs from the perspective
of their offensive game. The offensive game of the backs
in handball is mainly developed in the vicinity of the
9-m line, looking for a throw at a distance to provoke the
output of the defenders and creating space near the area of
6 m. The 9-m throw is therefore an important resource in
the game of the backs, but they may also end the attacks
with a throw from the line of 6 m, with a pass of goal (an
assist), trying to break through the defense and create a
scoring chance or with a movement that is awarded with
a 7-m throw. In nonpositional situations the backs play
also an important role, because they usually complete the
fastbreaks after either an intercepted pass or a steal occur
and the team can switch fast from defense to offense. The
following 7 variables, which provide information regard-
ing different aspects of the game like shooting, both in a
positional attack and in transition, or ball handling, were
considered for the analysis:
- “G9m”, which is denoted by y1 in the formulations,
is the number of goals per game scored from the
line of 9 m by the percentage of success, which the
ratio between the number of 9m. goals during the
season and the number of attempts. It is therefore an
indicator of shooting from the 9 m. line adjusted for
opportunities. We could have considered separately
the number of 9 m. goals per game and the percent-
age of success, but we preferred to aggregate both
variables into a single indicator to avoid mixing a
percentage with a volume measure (see again see
Dyson, Allen, Camanho, Podinovski, Sarrico, &
Shale, 2001, for a discussion of the pitfalls that can
be encountered in DEA applications).
- “G6m” (y2) and “GFB” (y3) are defined in a similar
manner as G9m, in each case regarding the goals
scored from the line of 6m. and the fastbreak goals.
- “Assis” (y4) is the number of assists per game.
- “Fouls” (y5) is the negative exponential of the
number of fouls per game made. Note that this trans-
formation of the number of fouls made per game,
which is a negative aspect of the game, allows us
to treat this factor as an “output” in the sense that
it is a “the more the better” variable. We have used
here the negative exponential instead of the inverse
(as has been done in other papers) because it is a
transformation whose effects in the actual range of
data are weaker. Note also that some players have
made no foul during the season.
- “7Mc” (y6) is the number of seven meters caused
per game.
- “TO” (y7) is the negative exponential of the number
of turnovers per game (the comment above regarding
Fouls also apply with this factor).
As can be seen, these variables measure the perfor-
mance per game of the players in different aspects of the
game. Such variables could perhaps reflect the player
performance more appropriately if they were defined
instead in terms of the number of minutes played per
game, but that information was not available.
220    Gutiérrez and Ruiz
The sample consisted originally of 95 players. Nev-
ertheless, for our analysis, we have only considered the
66 players that have participated in more than 40% of the
total matches of the season. With this number of matches
we seek not only the reliability of the data but also to
have a sample size large enough so as to avoid problems
with the dimensionality of the models used, as we have
many variables (7). We note that this exclusion of players
should not affect the conclusions we have drawn in this
paper, since the players that do not participate regularly
in the matches during the season are not expected to be
efficient, so they would not play a role in the assessments.
The data corresponding to the 7 variables above for the
66 players considered have been taken from the official
statistics of the Spanish Association of Handball Clubs
(ASOBAL) (http://www.asobal.es/). These 66 players
can thus be described by means of the output vectors Pj
= (y1,j,...,y7,j)' , j = 1,. . . ,66. Table 1 records a descriptive
summary of the data
Methodology
We use the so-called CCR DEA model (Charnes, et al.,
1978) for the analysis of efficiency. In its formulation, the
66 players in the sample play the role of the DMUs, the
7 variables previously listed are incorporated as outputs
and we do not incorporate any explicit inputs, since in
our analysis we leave out of consideration such things as
player salaries, etc. (as a consequence, although we use
the term efficiency throughout the paper, we are actually
concerned with “effectiveness”; see Prieto, & Zofio,
2001, for discussions). We only include in the model a
single constant input equals 1, which means that every
player is doing the best for playing his game, i.e., each
player is performing as good as he can. It should be noted
that, in the case of having one constant input, the CCR
model coincide with the BCC model (Banker, Charnes,
& Cooper, 1984), i.e., in these special circumstances the
specification of returns to scale is not particularly relevant
(see Lovell, & Pastor, 1999, for details and discussions
on this particular type of DEA models).
For a given player, say player 0, the following for-
mulation of the DEA model provides the weights that
Table 1  Descriptive Summary of the Data
Mean St. Deviation Min Max
G9m 0.6053 0.4072 0 1.5990
G6m 0.5711 0.3837 0.0179 1.6344
GFB 0.1354 0.1580 0 0.7241
Assis 0.4020 0.3510 0 1.4783
Fouls 0.7882 0.1366 0.4925 1 λ1 × P1 + ... + λ66 × P66 ≥ φ0 × P0
7Mc 0.2198 0.1090 0.0417 0.5294
TO 0.7570 0.1520 0.3247 1 λ1 ,..., λ66 ≥ 0
allows us to aggregate the information regarding the 7
outputs above into a single value
Maximize θ0 = ω1 × y1,0 + ... + ω 7 × y7,0
subject to :
ω1 × y1,1 + ... + ω 7 × y7,1 ≤ 1
.............................
ω1 × y1,66 + ... + ω 7 × y7,66 ≤ 1
ω1 ,..., ω 7 ≥ 0 (1)
We can see that the objective in (1) is to find the
weights w’s that maximize the corresponding weighted
sum of outputs for player 0, subject to the condition that
this weighted sum, calculated with these weights for the
rest of players, is in all cases lower than or equal to a
given value, which is usually set at 1. Thus, player 0 is
said to be efficient if Θ0 = 1. Otherwise, he is inefficient,
and the lower Θ0 the lesser his efficiency. Looking at
model (1) we also realize that in DEA there is no need
to a priori know the weights that represent the impor-
tance to be attached the different aspects of the game.
When solving (1) each player has total freedom in the
choice of such weights, which are determined trying to
show the player under assessment in his best possible
light. Nevertheless, weight restrictions can be added
to (1) to incorporate into the analysis value judgments
from experts regarding the relative importance of the
variables, so the resulting weights reflect the expert
opinions and their accepted views (see chapter 4 in
Cooper, Ruiz, & Sirvent, 2011, for a recent survey on
choices and uses of DEA weights). In the particular
case of our assessment of the backs players from the
perspective of their offensive game, we have taken into
consideration the opinion of coaches of the Spanish
national handball league with more than 10 years of
experience who believe that the weights attached to
fouls and turnovers cannot be greater than those of the
rest of variables, since goals and assists are considered
as having more importance when we are concerned with
the offensive game of these players. To be specific, the
following restrictions have been added to (1)
ω1 ≥ ω 5 ω1 ≥ ω 7
ω2 ≥ ω5 ω2 ≥ ω7
ω3 ≥ ω5 ω3 ≥ ω7
ω4 ≥ ω5 ω4 ≥ ω7
ω6 ≥ ω5 ω 6 ≥ ω 7 (2)
By virtue of the dualty theory in linear programming,
DEA also provides a benchmarking analysis by solving
the following linear programming problem
Maximize φ0
subject to :
λ1 + ... + λ66 = 1
(3)
 Assessment of Players in Spanish Handball League   221
The optimal value of (3), ϕ0, is actually the inverse of
Θ0 in (1). Therefore, player 0 is efficient if ϕ0 = 1, while
he is rated as inefficient if ϕ0 > 1. Figure 1 illustrates
graphically the idea behind model (3). Suppose that we
have a simple situation in which 3 handball players are
to be assessed regarding two game factors, say, G6m and
G9m (with the notation used before in the description of
the variables). Their records during the season in these
two variables are P1(2,7) for player 1, P2(10,3) for player
2 and P3(4,3) for player 3, i.e., player 1, for example,
scored 2 6m. goals per game and 7 9m. goals per game,
and so on. The gray area is the so-called production
possibility set (PPS), and includes the players (real or
virtual) that are assumed to be potential benchmarks in
the assessments. Roughly speaking, in the PPS we have
combinations of real players, and others that represent
worse performances. The points on the frontier of the PPS
(the bold line) represent obviously “best practice” per-
formances. Players 1 and 2 are rated as efficient because
we cannot find in the PPS other players that score more
6m. goals and more 9m. goals than them. In that case,
ϕ1 and ϕ2 cannot be greater than 1. However, player 3 is
inefficient because other players in the PPS outperform
him regarding these two game factors. In particular, the
point (6.4,4.8) shows that player 3 should score 6.4 6m.
goals and 4.8 9m. goals to perform at the levels of the
efficient players. In other words, he should improve by
60% in these two game factors. The point (6.4,4.8) is a
benchmark for player 3 that results of a combination of
player 1 and player 2 in which the participation of the
former is 45% and that of latter is 55%, i.e., λ1 = 0.45
Figure 1 — Graphical explanation of model (3).
and λ2 = 0.55 in model (3) (obviously, λ3 = 0) so that
( 4.8 )
6.4 = 0.45 × P + 0.55 × P
1 2. Summing up, ϕ = 1.60
3
would be the efficiency score of player 3, and 6.4 in G6m
and 4.8 in G9m are efficient targets provided by the DEA
model (3) for this player (by using player 1 and player 2
as referents). These latter represent levels in each of the
two game factors considered that would make player 3
perform efficiently.
Thus, we can see that model (3) provides for
player 0 both an efficiency score and efficient targets.
The efficiency score ϕ0 provides us with an insight
into how far player 0 is from the efficiency. Besides,
ϕ0 can also be interpreted in terms of the percentage
of improvement by which player 0 should expand his
outputs to perform efficiently. The targets, which are
the coordinates of the benchmark, i.e., λ1 × yr,1 + ... +
λ66 × yr,66, r = 1,. . . ,7, where the l’s are the optimal
solutions of (3), play an important role since they may
indicate keys for the inefficient players to improve their
game (we note that the λ’s of the results in the present
paper are actually the optimal solutions of the model
dual to (1) after this latter problem has incorporated
the weight restrictions (2)).
Finally, we use the cross-efficiency evaluation for
the ranking of players. The cross-efficiencies of player
0 are the assessments of this player with the weights
of the others. That is, if (ω 1d ,..., ω 7d ) are the weights of
player d, obtained by solving (1) for that player, then the
cross-efficiency
1
Ed ,0 = (4)
ω 1d × y1,0 + ... + ω 7d × y7,0
222    Gutiérrez and Ruiz

is an evaluation of player 0 with the weights of player
d. The cross-efficiency score of player 0 is the average
of such cross-efficiencies, i.e.,
1 identify possible weaknesses in the game of the inefficient
E0 = ( E1,0 + ... + E66,0 ) (5)
66
The full ranking of players is determined according
to their cross-efficiency scores, which provides a peer-
appraisal in which each of them is assessed with reference
to different patterns of game that the different players
have used in their DEA assessments.
For those readers interested in details on the DEA
models, their formulations and properties, see the text-
book by Cooper, Seiford, and Tone (2007).
Results
The DEA model revealed that 10, out of the 66 play-
ers, are efficient. For each of them, Table 2 records the
contributions to the efficiency of each of the factors of
the game. These contributions, which are usually called
virtual weights, are the product of the weights obtained
and the corresponding actual data. For a given efficient
player 0 these would be ωr × yr0, r = 1,..,7, where the
ω’s are the weights provided by (1) when solved for
that player. They are dimensionless and can be seen
as percentages of contribution of each factor to the
total efficiency, so that they provide us with an insight
into the role that each aspect of the game played in the
assessment of each player. This table also reports the
number of times each of the efficient players has acted
as referent in the assessment of the inefficient players.
This is determined as the number of times the corre-
sponding λj in model (3) is nonzero in the assessment
of the different players.
The results of the benchmarking analysis provided
by DEA are reported in Table 3. This table records for
the inefficient players of the top 6 team in the 2008–2009
season (we only report these results just for reasons of
space) their actual data (in the first row of each player) and
the corresponding efficient targets (in the second row).
Comparisons between these two values can be made to
players and to suggest directions of improvement. To be
precise, if the actual data for a given variable is lower
than the corresponding target, this will be showing that
the actual level in that aspect of the game is worse than
that of his benchmark, and this can be seen as a weak-
ness in his game. For each inefficient player in the table,
in the last column we also report which efficient players
compose the corresponding benchmark, together with
their weights as efficient referents in such benchmark
(i.e., the λj’s provided by model (3) for the player under
assessment). Table 3 also records the DEA efficiency
scores, ϕ0, which show how far the inefficient players
are from the efficiency.
Table 4 records the cross-efficiencies (4) and the
cross-efficiency scores (5) of the players that eventually
rank in the top 20 (again, just for reasons of space). We
note that in our analysis here we have used a variant of
the standard cross-efficiency evaluation that assesses
the players by only using the weights of the players that
have been rated as efficient in the self-evaluation (see
Ramón, Ruiz, & Sirvent, 2011). Thus, the columns of
this table correspond to each of these 20 players, and
in each of them we have the assessments of their game
(the cross-efficiencies) with the weights of each of the
10 efficient players (in the corresponding row). The
cross-efficiency scores of these players are recorded
in the last row of this table under the corresponding
column. These cross-efficiency scores, therefore, are
the result of an evaluation of each player with the dif-
ferent patterns of game that the efficient players have
used in their self-evaluation. They also determine
the ranking of players. We can see, for instance, that
Nagy ranks 1st, followed by Vugrinec and Rutenka in
this order.
Table 5 records the full ranking of the players.

Table 2  Efficient Players: Contributions of Variables to the Efficiency and Frequency

as Reference Player
#
Player G9m G6m GFB Assis Fouls 7Mc TO TOTAL referent
Stefansson (CRL) 10.53% 12.52% 35.03% 12.65% 12.53% 3.30% 13.43% 100% 5
Rutenka (CRL) 9.70% 41.64% 12.74% 5.68% 6.97% 10.92% 12.34% 100% 34
Entrerrios (CRL) 7.43% 18.84% 1.78% 14.39% 10.65% 32.59% 14.30% 100% 21
Nagy (BAR) 22.78% 26.96% 5.26% 20.18% 13.27% 3.78% 7.77% 100% 44
Víctor Hugo (VLL) 67.01% 4.46% 0.80% 5.47% 3.83% 13.76% 4.67% 100% 15
Buntic (ADE) 14.90% 14.78% 2.28% 43.37% 12.36% 1.92% 10.39% 100% 2
Belaustegui (LOG) 38.30% 0.54% 0.34% 17.61% 2.24% 39.07% 1.90% 100% 0
Guardiola (LOG) 11.03% 8.53% 4.30% 47.58% 11.05% 10.81% 6.70% 100% 4
Gurbindo (TRR) 2.96% 39.52% 35.84% 14.56% 2.82% 1.50% 2.82% 100% 0
Nilsson (CQN) 76.15% 8.39% 0.84% 2.43% 5.83% 0.91% 5.44% 100% 8
 Assessment of Players in Spanish Handball League   223

Table 3  Inefficient Players: Benchmarking Analysis (Actual Data and Efficient Targets)
DEA
Player score G9m G6m GFB Assis Fouls 7Mc TO Benchmarks
Fernández (CRL) 1.2163 0.49 1.32 0.18 0.27 0.87 0.32 0.87 Rutenka, CRL (0,81) Nagy, BAR (0,19)
0.84 1.61 0.46 0.57 0.57 0.39 0.84
Pajovic (CRL) 1.4415 0.28 0.83 0.15 0.43 0.79 0.33 0.62 Rutenka, CRL (0,23) Entrerríos CRL (0,69)
1.57 2.49 0.44 1.56 1.22 0.64 1.22 Nagy, BAR (0,08)
Metlicic (CRL) 1.5454 0.46 0.44 0.22 0.25 0.84 0.25 0.96 Rutenka, CRL (0,80) Nagy, BAR (0,20)
0.84 1.61 0.46 0.57 0.57 0.39 0.83
Morros (CRL) 1.0575 0.89 0.79 0.51 0.50 0.68 0.15 0.86 Stefansson CRL (0,69) Nagy, BAR (0,01)
0.94 0.84 0.53 0.65 0.75 0.18 0.77 Nilsson, CQN (0,29)
Lozano (BAR) 1.6592 0.69 0.45 0.07 0.52 0.93 0.11 0.64 Nagy, BAR (1)
1.29 1.53 0.30 1.14 0.75 0.21 0.44
Hansen (BAR) 1.3671 0.97 0.62 0.02 0.59 0.93 0.17 0.60 Rutenka, CRL (0,75) Víctor Hugo VLL (0,25)
1.33 1.30 0.25 1.04 0.69 0.24 0.49
Krivocapic (VLL) 2.0542 0.39 0.33 0.00 0.10 0.82 0.20 0.67 Rutenka, CRL (0,89) Nagy, BAR (0,05)
0.80 1.57 0.47 0.48 0.54 0.41 0.89 Víctor Hugo VLL (0,06)
Ávila (VLL) 1.3235 0.03 0.08 0.03 0.03 0.82 0.40 1.00 Entrerríos CRL (1,00)
0.49 1.24 0.12 0.76 0.70 0.53 0.94
Bilbija (VLL) 1.5828 0.49 0.83 0.08 0.35 0.96 0.08 0.79 Nagy, BAR (1)
1.29 1.53 0.30 1.14 0.75 0.21 0.44
Vugrinec (SAN) 1.0537 1.21 1.06 0.25 1.00 0.74 0.23 0.68 Rutenka, CRL (0,03) Entrerríos CRL (0,04)
1.27 1.39 0.27 1.05 0.71 0.24 0.50 Nagy, BAR (0,80) Víctor Hugo VLL (0,14)
Kjelling (SAN) 1.3599 0.98 0.49 0.15 0.60 0.82 0.10 0.90 Nagy, BAR (0,87) Nilsson, CQN (0,13)
1.33 1.45 0.27 1.03 0.74 0.20 0.46
Malmagro (SAN) 1.1672 1.26 0.20 0.00 0.33 0.94 0.20 0.94 Nagy, BAR (0,14) Víctor Hugo VLL (0,58)
1.48 0.83 0.13 0.66 0.59 0.23 0.60 Nilsson, CQN (0,28)
Jorgensen (SAN) 1.8225 0.21 0.64 0.00 0.12 0.89 0.24 0.73 Rutenka, CRL (0,91) Entrerríos CRL (0,09)
0.71 1.60 0.47 0.46 0.54 0.44 0.93
Aguirre, M. (ADE) 1.2081 0.91 0.71 0.30 0.81 0.77 0.19 0.89 Stefansson CRL (0,12) Rutenka, CRL (0,05)
1.19 1.44 0.36 1.07 0.74 0.22 0.51 Nagy, BAR (0,83)
Kos (ADE) 1.3958 0.85 0.27 0.08 0.69 0.79 0.23 0.50 Entrerríos, CRL (0,13) Nagy, BAR (0,17)
1.18 0.85 0.17 0.97 0.66 0.32 0.61 Víctor Hugo VLL (0,49), Guardiola, LOG (0,22)
Bicanic (ADE) 1.0452 1.24 0.39 0.17 1.00 0.58 0.25 0.78 Nagy, BAR (0,51) Víctor Hugo VLL (0,35)
1.29 1.08 0.22 1.05 0.69 0.26 0.52 Guardiola, LOG (0,14)
Ferrer (GRA) 1.5964 0.36 0.33 0.10 0.89 0.68 0.14 0.78 Nagy, BAR (0,14) Buntic, ADE (0,43)
0.99 0.94 0.21 1.43 0.84 0.23 0.59 Guardiola, LOG (0,42)
Robledo (GRA) 1.3588 0.11 0.12 0.30 0.03 0.94 0.33 0.94 Rutenka, CRL (0,76) Entrerríos, CRL (0,24)
0.67 1.54 0.41 0.51 0.57 0.45 0.93
Campos (GRA) 1.5959 0.82 0.35 0.10 0.20 0.79 0.17 0.77 Rutenka, CRL (0,08) Nagy, BAR (0,52)
1.31 1.17 0.24 0.92 0.64 0.27 0.55 Víctor Hugo VLL (0,40)
Cutura (GRA) 1.4682 0.92 0.65 0.00 0.38 0.54 0.17 0.81 Rutenka, CRL (0,01) Nagy, BAR (0,57)
1.36 1.14 0.22 0.96 0.65 0.25 0.52 Víctor Hugo VLL (0,42)
Puig (GRA) 1.6733 0.39 0.90 0.15 0.22 0.57 0.22 0.72 Rutenka, CRL (0,74) Nagy, BAR (0,26)
0.88 1.61 0.45 0.62 0.59 0.37 0.80

Discussion in all of the aspects of the game. The different patterns

The DEA models have their own way to handle the
information provided by the statistics. In particular,
model (1) provides the aggregation weights for each
player that allow us to make an overall assessment of his
game that accounts simultaneously for the performance
of virtual weights in Table 2 reveal that each of the
10 efficient players has achieved the efficiency with
different models of game. This shows how DEA takes
into consideration the own characteristics of the game
of the different players in the assessment of their game
performance.
224
Table 4  Cross-Efficiency Evaluation
Player being evaluated
Weights V.
of.. Nagy Vugri. Ruten. Buntic Guardio. Entrerrios Stefanss. Bicanic Gurbin. Hugo Aguirre Belaust. Morros Nilsson Dasilva Fdez. Jurkiew. Hansen Stank Prce
Stefansson 1.00 1.11 1.01 1.11 1.14 1.25 1.00 1.32 1.11 1.38 1.21 1.38 1.16 1.39 1.42 1.33 1.27 1.49 1.52 1.51
Rutenka 1.00 1.15 1.00 1.16 1.23 1.15 1.18 1.46 1.13 1.43 1.33 1.55 1.32 1.41 1.41 1.25 1.37 1.42 1.50 1.56
Entrerríos 1.00 1.08 1.00 1.08 1.04 1.00 1.17 1.22 1.19 1.21 1.23 1.19 1.33 1.41 1.30 1.23 1.46 1.34 1.37 1.44
Nagy 1.00 1.10 1.09 1.06 1.13 1.18 1.18 1.28 1.24 1.30 1.24 1.32 1.29 1.32 1.33 1.31 1.35 1.44 1.43 1.48
V. Hugo 1.00 1.07 1.20 1.20 1.20 1.34 1.45 1.11 1.53 1.00 1.31 1.10 1.37 1.03 1.14 1.57 1.46 1.66 1.35 1.23
Buntic 1.00 1.10 1.21 1.00 1.05 1.23 1.22 1.26 1.31 1.34 1.26 1.30 1.39 1.49 1.48 1.48 1.51 1.40 1.50 1.62
Belaustegui 1.05 1.10 1.09 1.22 1.00 1.00 1.46 1.09 1.42 1.00 1.37 1.00 1.58 1.34 1.27 1.49 1.94 1.43 1.36 1.33
Guardiola 1.00 1.10 1.18 1.00 1.00 1.15 1.19 1.23 1.27 1.32 1.26 1.24 1.41 1.60 1.53 1.48 1.59 1.36 1.51 1.64
Gurbindo 1.00 1.22 1.00 1.19 1.24 1.39 1.00 1.68 1.00 1.82 1.40 1.97 1.26 1.86 1.96 1.50 1.42 1.49 1.87 1.92
Nilsson 1.00 1.08 1.35 1.16 1.28 1.63 1.47 1.15 1.63 1.05 1.32 1.18 1.36 1.00 1.18 1.74 1.40 1.78 1.42 1.27
Cross-eff. 1.00 1.11 1.11 1.12 1.13 1.23 1.23 1.28 1.28 1.29 1.29 1.32 1.35 1.38 1.40 1.44 1.48 1.48 1.48 1.50
 Assessment of Players in Spanish Handball League   225

Table 5  Full Ranking of Players

Player Cross-eff. Scores Player Cross-eff. Scores
Nagy (BAR) 1.0049 Rudovic (ARR) 1.9122
Vugrinec (SAN) 1.1106 Ferrer (GRA) 1.9238
Rutenka (CRL) 1.1141 Bertos (ALM) 1.9414
Buntic (ADE) 1.1175 Campos (GRA) 1.9497
Guardiola, I. (LOG) 1.1319 Puig (GRA) 1.9559
Entrerríos, A. (CRL) 1.2318 Cid (ANT) 1.9890
Stefansson (CRL) 1.2321 Petric (CQN) 2.0434
Bicanic (ADE) 1.2804 Da Cruz (ALM) 2.0570
Gurbindo (TRR) 1.2823 Martinovic (OCT) 2.0881
Víctor Hugo (VAL) 1.2860 Stefanovic (ARR) 2.1906
Aguirre, M. (ADE) 1.2928 Infestas (OCT) 2.2003
Belaustegui (LOG) 1.3226 Kobin (ARR) 2.2270
Morros (CRL) 1.3465 Madsen (TRR) 2.2918
Nilsson (CQN) 1.3843 Sorrentino (ZAR) 2.3094
Dasilva (TEU) 1.4021 García, A. (GRA) 2.3456
Fernández (CRL) 1.4380 Jorgensen (SAN) 2.3984
Jurkiewicz (ARR) 1.4757 Pavlovic (LOG) 2.4750
Hansen (TRR) 1.4797 Ortega (ZAR) 2.4830
Stankovic (ZAR) 1.4822 Gauchi (LOG) 2.4900
Prce (OCT) 1.4997 Curkovic (ADE) 2.4990
Kjelling (SAN) 1.5268 Grosas (CQN) 2.5042
Hansen (BAR) 1.5626 Romero (BAR) 2.5155
Obradovic (ANT) 1.5627 Krivocapic (VLL) 2.6609
Malmagro (SAN) 1.6568 Jimenez (TRR) 2.7027
Kos (ADE) 1.7117 Polakovic (OCT) 2.7079
Vatne (ZAR) 1.7344 Savkovic (TEU) 2.7195
Cutura (GRA) 1.7594 Jimenez (ANT) 2.7744
Pajovic (CRL) 1.7814 Muiña (ZAR) 2.8692
Maqueda (ALC) 1.8094 Hiliuk (CQN) 2.8711
Metlicic (CRL) 1.8335 López (TRR) 3.0941
Lozano (BAR) 1.8472 Kedzo (TEU) 3.3844
Bilbija (VLL) 1.8973 Guirado (ALM) 3.8470
Jahns (ANT) 1.9054 Vargas (ARR) 4.0295
(*) In bold DEA efficient player

On one hand, we have players, like Stefansson,
Rutenka, Entrerríos and Nagy, with a good performance
in different aspects of the game, having thus a more bal-
anced profile as players. To be specific, Nagy is the player
with the most balanced profile of weights, while Rutenka,
Stefansson and Entrerríos have exploited to some extent
some of their strengths to achieve the efficiency. For
example, the relative large contribution of G6m in
Rutenka (41.64%) shows that he is a player who exploits
in the game his ability in the penetration into the goal,
while the value 35.03% in GFB for Stefansson reveals the
rapid transition of the defense phase to the attack phase
as one of the main features of his game. In the case of
Entrerríos, the 32.59% in 7Mc shows that he provokes
many chances of 7m. On the other, we can see that some
players have achieved the efficiency putting the weight
mainly on one aspect of the game, and this may be show-
ing that they are more specialized players. For example,
the contribution to the efficiency of G9m in Nilsson and
Víctor Hugo, 76.15% and 67.01% respectively, shows
the typical profile of throw at a distance, with a game
mainly based on a very high percentage of goals from
226    Gutiérrez and Ruiz
9 m. Finally, we should also mention the relative large
contributions of G6m and GFB in Gurbindo, and those of
G9m and 7Mc in Belaustegui, while Buntic and Guardiola
are players who create chances for his teammates rather
than being terminators, as revealed by their contributions
in Assis (43.37% and 47.58%, respectively).
Table 2 shows in addition that Nagy, Rutenka and
Entrerríos (together with Víctor Hugo) have played an
important role as benchmarks for the remaining play-
ers. To be specific, they have acted as referents in the
assessments of 44, 34, 21 and 15, respectively, of the
inefficient players.
The analysis of benchmarking provided by DEA
is another key feature of this methodology. This has
made it possible to identify and quantify the sources of
inefficiency in the game of the inefficient players and,
therefore, suggest potential directions of improvement.
To be precise, we can do it by comparing the actual data
with the corresponding efficient targets for each variable
in Table 3. We note again that the targets provided by this
methodology are player-specific, since the DEA model
(3) selects a benchmark for each player in accordance
with the characteristics of his game.
We have players like Bicanic, Vugrinec and Morros,
whose DEA efficiency scores, which are respectively
1.0452, 1.0537 and 1.0575, reveal that they are rated
near the maximum efficiency, whereas other like Bilbija
(1.5828) or Metlicic (1.5454) are quite inefficient. The
analysis of benchmarking allows us to go one step further
and identify the sources of the detected inefficiency. We
can see that the actual data of Bicanic, Vugrinec and
Morros are very close to their targets, which show a good
performance of their game. However, in all the three
cases we can identify some weakness as the result of their
comparison with other players: the actual data in Assis
for Morros (0.50) is 76.92% of the corresponding target
(0.65) in that variable, as the result of the comparison of
this player with a virtual player that is a combination of
Stefansson (70%) and Nilsson (30%) approximately, and
this indicates that Morros should improve in generating
scoring situations for their peers to achieve the efficiency.
The same can be stated for Bicanic and Vugrinec, in
that case regarding the variable G6m. It is particularly
noticeable the case of Bicanic, whose data in that variable
(0.39) is only 36.1% of the corresponding target (1.08)
(as the result of his comparison with a virtual player that
is a combination of Nagy (51%), Víctor Hugo (35%) and
Guardiola (14%)), and this reveals an important weakness
of the game of this player in the penetration into the line
of 6 m. Note, however, that the actual value in G9m for
Bicanic is very close to his efficient targets, which means
that this is a good player in the throw at a distance.
A different pattern of performance can be found in
players like Malmagro (whose efficiency score equals
1.1672), who has a good performance in G9m but has an
important weakness in G6m (the actual data 0.20 is 24.1%
of the corresponding target 0.83). This type of profile
suggests a specialist player in the launch throw, which
is characteristic of the backs players. Malmagro should
also improve in Assis and, like others, in GFB, which is
one of the more typical game features of the wings, where
the actual data equals 0. The handball regulation allows
changes of players from defense to attack, i.e., when
the defense phase is completed the team can change a
player by other. Therefore, the players who only attack,
as Malmagro, have zero in this variable. Nevertheless,
the target provided by this analysis of relative efficiency
for this player (0.13) may be showing that this level of
performance in this variable can be easily achieved.
In the cases of Metlicic and Bilbija, for example, the
large differences between the actual data and the targets
are showing the poor performance of these players in most
of the aspects of the game. The actual data of Metlicic,
who is compared with a virtual player resulting of a
combination of Rutenka (80%) and Nagy (20%), are far
away from the targets, especially in G6m, GFB and Assis.
It should be noted, however, that Metlicic outperforms
his referent in the variables Fouls and TO, but these are
the aspects of the game considered as having less impor-
tance. In the case of Bilbija, who is compared directly
with Nagy, very important weaknesses are detected in
almost all the variables, especially in G9m (when the
backs are assumed to have a good throw for a distance),
GFB, Assis and 7Mc.
If we collect this information for groups of players
in the same team, we would be analyzing the results from
the perspective of a team coach. For example, if we have
a look at the backs of Ademar (Bicanic, Aguirre and Kos),
we could conclude that this team appears to have a good
performance in shooting from the line of 9m, while its
offensive game has a serious weakness in the line of 6m.
The DEA efficiency analysis has been complemented
with a cross-efficiency evaluation, which provides a peer-
evaluation of the players that makes it possible to rank
them. As said before, Nagy is the player that ranks 1st but,
in addition, we note that in the column in Table 4 under
this player almost all the cross-efficiencies equal 1. This
means that the cross-efficiency evaluation shows Nagy
as an all-round performer, because he is rated with the
maximum efficiency with the models of game that have
used in their self-evaluation practically all the efficient
players. Only the cross-efficiency of Nagy provided by
Belaustegui is greater than 1 (1.05). Perhaps, this is due to
the fact that Belaustegui has achieved the efficiency with
a pattern of game that puts very much weight on 7Mc, and
Nagy is not especially good in that aspect of the game.
The cross-efficiency score of Nagy, 1 (approximately), is
quite larger than those of the remaining players. Vugrinec,
Rutenka, Buntic and Guardiola follow Nagy with cross-
efficiency scores that are greater than 1.10. These four
players have, in general, good cross-efficiencies when
evaluated with the profiles of weights of the other players.
Note, however, that in the case of, for example, Rutenka,
he has been poorly rated with the weights of Nilsson
(1.35), and also with those of V. Hugo and Buntic (these
cross-efficiencies are, respectively, 1.20 and 1.21). The
reason behind this may be that Nilsson, and also V. Hugo,
are evaluated with a pattern of game that puts the weight
 Assessment of Players in Spanish Handball League   227
mainly on G9m, while Buntic puts much weight on Assis,
and Rutenka does not have his strengths in these two game
factors (see Table 2). Similar analyses can be made for
Buntic and Guardiola.
The case of Vugrinec is especially noticeable because
he is a player rated as inefficient in the DEA self-evalua-
tion that eventually ranks 2nd; therefore, before most of
the efficient players (all except Nagy). As a result of the
relative nature of the DEA analysis, Vugrinec cannot be
rated as efficient, since Nagy is a player with a similar
pattern of game that outperforms him (remember that
Nagy contributes by 80% in the benchmark used in the
assessment of Vugrinec). Nevertheless, Vugrinec is very
close to his benchmark, and this is why this player, like
Nagy, is given good ratings when evaluated with the
weights of the others.
Finally, we would like to stress that the players
assessed as efficient in the DEA self-evaluation rank high
since these are in the top 15 (see Table 5). These all are
international players in their countries, where they are
also players of reference. In fact, the top 10 players in
our analysis have an average of 55 international matches
with their national teams.
Conclusions
DEA and cross-efficiency evaluation have been shown as
support tools for the management of sports teams in the
context of a study of assessment of the individual game
performance of handball players of the premier Spanish
handball league. These methodologies provide coaches
and managers with a thorough knowledge of the players
participating in a competition, which may help them
improve the performance of their teams.
DEA provides an index of the overall performance of
players that aggregates into a single value their behavior
in all of the aspects of the game. This aggregation does
not require to a priori specifying the importance to be
attached to the different game factors. DEA assesses rela-
tive efficiency, so that players are assessed in relation to
the others. As an important feature of this methodology
we would like to point out that DEA takes into consider-
ation the own characteristics of the game of the different
players both in the choice of the aggregation weights
and in the setting of the targets. In both cases, these are
player-specific, which means, on one hand, that each
player can exploit his strengths in his evaluation and, on
the other, the areas of potential improvement for his game
are identified by considering individual characteristics.
The key issue is that different players may have differ-
ent ways to achieve the efficiency of performance. The
cross-efficiency evaluation allows to ranking the play-
ers by means of a peer-appraisal in which each player
is assessed with respect to the patterns of game that the
others have used in their DEA self-evaluation.
We have addressed here the assessment of game
performance in the case of players. Nevertheless, the use
of these methodologies can be straightforwardly extended
for an assessment at the level of teams. Likewise, although
our concern here has been effectiveness, since we have
left out of consideration any information regarding player
salaries, etc., DEA and cross-efficiency evaluation can
incorporate such type of inputs into the models, if avail-
able, and develop measures of efficiency. Moreover,
we might also evaluate organizational performance and
develop measures of, for instance, managerial efficiency,
like others have already done. The results obtained in
this paper indicate promise in these potential uses of
these methodologies for the assessment of performance
in sports.
Acknowledgments
We are very grateful to two anonymous reviewers for their
helpful comments and suggestions. We would like also to thank
Ministerio de Ciencia e Innovación (MTM2009-10479) and
Generalitat Valenciana (ACOMP/2012/144) for their financial
support.
References
Alp, I. (2006). Performance Evaluation of Goalkeepers of the
World Cup. G.U. Journal of Science, 19(2), 119–125.
Anderson, T.R., Hollingsworth, K., & Inman, L.B. (2002).
The fixed weighting nature of a cross-evaluation model.
Journal of Productivity Analysis, 17(3), 249–255.
doi:10.1023/A:1015012121760
Anderson, T.R., & Sharp, G.P. (1997). A new measure of base-
ball batters using DEA. Annals of Operations Research,
73, 141–155. doi:10.1023/A:1018921026476
Banker, R.D., Charnes, A., & Cooper, W.W. (1984). Some
Models for Estimating Technical and Scale Inefficiencies
in Data Envelopment Analysis. Management Science, 30,
1078–1092. doi:10.1287/mnsc.30.9.1078
Barros, C.P., & Santos, A. (2003). Productivity in sports
organisational training activities: A DEA study. Euro-
pean Sport Management Quarterly, 3(1), 46–65.
doi:10.1080/16184740308721939
Barros, C.P., & Leach, S. (2006). Performance evaluation of
the English Premier Football League with data envelop-
ment analysis. Applied Economics, 38(12), 1449–1458.
doi:10.1080/00036840500396574
Barros, C.P., Assaf, A., & Sá-Earp, F. (2010). Brazilian Foot-
ball League Technical Efficiency: A Simar and Wilson
Approach. Journal of Sports Economics, 11(6), 641–651.
doi:10.1177/1527002509357530
Barros, C.P., & García del Barrio, P. (2011). Productivity drivers
and market dynamics in the Spanish first division football
league. Journal of Productivity Analysis, 35(1), 5–13.
doi:10.1007/s11123-010-0196-9
Boscá, J.E., Liern, V., Martínez, A., & Sala, R. (2009). Increas-
ing offensive or defensive efficiency? An analysis of Italian
and Spanish football. Omega, 37, 63–78. doi:10.1016/j.
omega.2006.08.002
Charnes, A., Cooper, W.W., & Rhodes, E. (1978). Measuring
the efficiency of decision making units. European Journal
of Operational Research, 2, 429–444. doi:10.1016/0377-
2217(78)90138-8
Chen, W.C., & Johnson, A.L. (2010). The dynamics of perfor-
mance space of Major League Baseball pitchers 1871-
2006. Annals of Operations Research, 181(1), 287–302.
doi:10.1007/s10479-010-0743-9
228    Gutiérrez and Ruiz
Cooper, W.W., Ramón, N., Ruiz, J.L., & Sirvent, I. (2011).
Avoiding large differences in weights in cross-efficiency
evaluations: application to the ranking of basketball play-
ers. Journal of Centrum Cathedra, 4(2), 197–215.
Cooper, W.W., Ruiz, J.L., & Sirvent, I. (2009). Selecting non-
zero weights to evaluate effectiveness of basketball players
with DEA. European Journal of Operational Research,
195, 563–574. doi:10.1016/j.ejor.2008.02.012
Cooper, W.W., Ruiz, J.L., & Sirvent, I. (2011). Choices and
uses of DEA Weights. In Handbook On Data Envelopment
Analysis (William W. Cooper, Lawrence W. Seiford and
Joe Zhu, Eds.) Springer, 93-126.
Cooper, W.W., Seiford, L.M., & Tone, K. (2007). Data Envel-
opment Analysis: A Comprehensive Text with Models,
Applications, References and DEA-Solver Software (2nd
ed.). New York: Springer Science & Business Publishers.
Dawson, P., Dobson, S., & Gerrard, B. (2000). Estimating
coaching efficiency in professional team sports: Evidence
from English association football. Scottish Journal of
Political Economy, 47(4), 399–421. doi:10.1111/1467-
9485.00170
Doyle, J.R., & Green, R.H. (1994). Strategic choice and data
envelopment analysis: Comparing computers across many
attributes. Journal of Information Technology, 9, 61–69.
doi:10.1057/jit.1994.7
Dyson, R.G., Allen, R., Camanho, A.S., Podinovski, V.V., Sar-
rico, C.S., & Shale, E.A. (2001). Pitfalls and protocols in
DEA. European Journal of Operational Research, 132,
245–259. doi:10.1016/S0377-2217(00)00149-1
Einolf, K.W. (2004). Is Winning Everything?: A Data Envelop-
ment Analysis of Major League Baseball and the National
Football League. Journal of Sports Economics, 5(2),
127–151. doi:10.1177/1527002503254047
Espitia-Escuer, M., & García-Cebrián, L.I. (2004). Measur-
ing the Efficiency of Spanish First-Division Soccer
Teams. Journal of Sports Economics, 5(4), 329–346.
doi:10.1177/1527002503258047
Fizel, J.L., & D’Itri, M.P. (1996). Estimating managerial effi-
ciency: The case of College basketball coaches. Journal
of Sport Management, 10, 435–445.
Fizel, J.L., & D’Itri, M.P. (1997). Managerial Efficiency,
managerial succession and organizational performance.
Managerial and Decision Economics, 18, 295–308.
doi:10.1002/(SICI)1099-1468(199706)18:4<295::AID-
MDE828>3.0.CO;2-W
Fizel, J.L., & D’Itri, M.P. (1999). Firing and hiring of manag-
ers: Does efficiency matter? Journal of Management, 25,
567–585. doi:10.1177/014920639902500405
Fried, H.O., Lambrinos, J., & Tyner, J. (2004). Evaluating
the performance of professional golfers on the PGA,
LPGA and SPGA tours. European Journal of Opera-
tional Research, 154, 548–561. doi:10.1016/S0377-
2217(03)00188-7
Fried, H.O., & Tauer, L.W. (2011). The impact of age on the
ability to perform under pressure: golfers on the PGA
tour. Journal of Productivity Analysis, 35(1), 75–84.
doi:10.1007/s11123-009-0151-9
García-Sánchez, I.M. (2007). Efficiency and effectiveness of
Spanish football teams: a three-stage-DEA approach.
Central European Journal of Operations Research, 15(1),
21–45. doi:10.1007/s10100-006-0017-4
González-Gómez, F., & Picazo-Tadeo, A.J. (2010). Can We Be
Satisfied With Our Football Team? Evidence from Spanish
Professional Football. Journal of Sports Economics, 11(4),
418–442. doi:10.1177/1527002509341020
Haas, D.J. (2003). Technical Efficiency in the Major League
Soccer. Journal of Sports Economics, 4(3), 203–215.
doi:10.1177/1527002503252144
Haas, D.J., Kocher, M.G., & Sutter, M. (2004). Measuring
efficiency of German football teams by data envelop-
ment analysis. Central European Journal of Operations
Research, 12, 251–268.
Hraste, M., Dizdar, D., & Trninić, V. (2008). Experts Opinion
about System of the Performance Evaluation Criteria
Weighted per Positons in the Water Polo Game. Collegium
Antropologicum, 32, 851–861.
Lovell, C.A.K., & Pastor, J.T. (1999). Radial DEA models
without inputs or without outputs. European Journal of
Operational Research, 118(1), 46–51. doi:10.1016/S0377-
2217(98)00338-5
Lozano, S., Villa, G., Guerrero, F., & Cortés, P. (2002). Measur-
ing the performance of nations at the Summer Olympics
using data envelopment analysis. The Journal of the
Operational Research Society, 53, 501–511. doi:10.1057/
palgrave.jors.2601327
Prieto, A.M., & Zofio, J.L. (2001). Evaluating effectiveness
in public provision of infrastructure and equipment: The
case of Spanish municipalities. Journal of Productivity
Analysis, 15(1), 41–58. doi:10.1023/A:1026595807015
Ramón, N., Ruiz, J.L., & Sirvent, I. (2011). Reducing differ-
ences between profiles of weights: A “peer-restricted”
cross-efficiency evaluation. Omega, 39(6), 634–641.
doi:10.1016/j.omega.2011.01.004
Ramón, N., Ruiz, J.L., & Sirvent, I. (2012). Common sets of
weights as summaries of DEA profiles of weights: with
an application to the ranking of professional tennis play-
ers. Expert Systems with Applications, 39, 4882–4889.
doi:10.1016/j.eswa.2011.10.004
Ruiz, J.L., Pastor, D., & Pastor, J.T. (to appear).Assessing pro-
fessional tennis players using data envelopment analysis
(DEA). Journal of Sports Economics.
Sexton, T.R., & Lewis, H.F. (2003). Two-stage DEA: An appli-
cation to major league baseball. Journal of Productivity
Analysis, 19, 227–249. doi:10.1023/A:1022861618317
Sexton, T.R., Silkman, R.H., & Hogan, A.J. (1986). Data
envelopment analysis: Critique and extensions. In R.H.
Silkman (Ed.), Measuring efficiency: An assessment of
data envelopment analysis (pp. 73–105). San Francisco:
Jossey-Bass.
Soares de Mello, J.C., Angulo-Meza, L., & Branco da Silva,
B.P. (2009). A ranking for the Olympic Games with
unitary input DEA models. IMA Journal of Management
Mathematics, 20, 201–211. doi:10.1093/imaman/dpn025
Sueyoshi, T., Ohnishi, K., & Kinase, Y. (1999). A Benchmark
Approach for Baseball Evaluation. European Journal
of Operational Research, 115, 429–448. doi:10.1016/
S0377-2217(98)00126-X
Swalgin, K. (1998). The basketball evaluation system: a com-
puterized factor weighted model with measures of validity.
The International Scientific Journal of Kinesiology and
Sport, 30, 31–37.
Tenga, A., Holme, I., Ronglan, L.T., & Bahr, R. (2010a). Effect
of playing tactics on achieving score-box possessions in
a random series of team possessions from Norwegian
professional soccer matches. Journal of Sports Sciences,
28, 245–255. doi:10.1080/02640410903502766
Tenga, A., Holme, I., Ronglan, L.T., & Bahr, R. (2010b). Effect
of playing tactics on goal scoring in Norwegian profes-
sional soccer. Journal of Sports Sciences, 28, 237–244.
doi:10.1080/02640410903502774
 Assessment of Players in Spanish Handball League   229
Trninić, S., & Dizdar, D. (2000). System of the Performance
Evaluation Criteria Weighted per Positions in the Bas-
ketball Game. Collegium Antropologicum, 24, 217–234.
Trninic´, S., Dizdar, D., & Dezman, B. (2002). Pragmatic Validity
of the Combined Model of Expert System for Assessment
and Analysis of the Actual Quality Overall Structure of Bas-
ketball Players. Collegium Antropologicum, 26, 199–210.
Trninić, S., Perica, A., & Dizdar, D. (1999). Set of Criteria for
the Actual Quality Evaluation of the Elite basketball Play-
ers. Collegium Antropologicum, 23, 707–721.
Trninić, M., Trninić, S., & Papić, V. (2009). Development Man-
agement Model of Elite Athletes in Team Sports Games.
Collegium Antropologicum, 33(2), 363–372.
Ueda, T., & Amatatsu, H. (2009). Determination of bounds
in DEA assurance region method - its application to
evaluation of baseball players and chemical companies
-. Journal of the Operations Research Society of Japan,
52(4), 453–467.
Volz, B. (2009). Minority Status and Managerial Survival in
Major League Baseball. Journal of Sports Economics,
10(5), 522–542. doi:10.1177/1527002508330860
Wu, J., Zhou, Z., & Liang, L. (2010). Measuring the Perfor-
mance of Nations at Beijing Summer Olympics Using
Integer-Valued DEA Model. Journal of Sports Economics,
11(5), 549–566. doi:10.1177/1527002509352619
Zhang, D., Li, X., Meng, W., & Liu, W. (2009). Measuring the
performance of nations at the Olympc Games using DEA
models with different preferences. The Journal of the
Operational Research Society, 60, 983–990. doi:10.1057/
palgrave.jors.2602638