Incorporating weighting into

risk assessment: can this make

an overall risk rating more
meaningful?
Lihong Zhu1, John Holt2 & Rob Black3

1. Ministry of Agriculture and Forestry, New Zealand

Lihong.Zhu@maf.govt.nz
2. Natural Resources Institute, University of Greenwich, UK
3. Department of Law, University of Greenwich, UK
 Outlines
Introduction
Why weighting?
How weighting?
What can weighting do for us?
Discussions and Conclusions

Incorporating weighting into PRA

 Pest risk assessment
Essential stage in PRA: risk assessment
Assessing the likelihood and consequences
of pest introduction
Evaluating whether the risk is significant
and therefore should be regulated
Criteria including entry, establishment,
spread and consequences
Incorporating weighting into PRA
 Pest risk assessment
Risk factors identified by subdividing each criterion into
a series of tangible risk factors,
A risk score attributed to each risk factor
Overall assessment based on a synthesis of the scores:
Simple everage
Weighted average
High & biased weighted everage
(Zhu etc 2000, Holt 2005)

Incorporating weighting into PRA

 Weighting the risk element
But how to achieve a more logical & biologically meaningful
overall risk rating?
incorporating weighting i.e. a weighting is given to each
risk factor/element
a weighting is a value given to a risk factor according to
how important it is perceived to be, or how significant it
contribute to the overall risk rating: the larger the value,
the more important the factor
Why give weighting?
What weighting systems are adopted?
How to derive weighting?

Incorporating weighting into PRA

 Table 1. Disadvantage of simple average:
A hypothetical example illustrates the impact of
simple average of risk factor scores.
Risk factor Factor 1 Factor 2

Risk score 1 9

Weighting 0.2 0.8

Simple average: 5

Weighted average: 7.4

Incorporating weighting into PRA

 Why give weighting?
The risk factors are not all equally important
Weighting should be given to each risk factor
to reflect its perceived importance
Those more important should properly
contribute more to the final result (i.e.
estimated overall risk) than those less
important

Incorporating weighting into PRA

 Weighting systems
How many risk elements being taken into
account
0-2 or 0-3 weighting systems
0-1 weighting system
Associated combining formulae needed

Incorporating weighting into PRA

 How to derive weightings
0-1 weighting system:
Weighting is set at 0-1 scale, the higher the
weighting, the more important the risk element
Summation of weightings for all risk elements
equals 1
Therefore the overall risk score is normalised and is
independent from the number of risk elements

Incorporating weighting into PRA

 How to derive weightings
Ranking the risk elements
Converting ranks to numerical
weightings between 0 and 1
Effectively, a linear rescaling of the
rankings is calculated such that the
sum = 1
Incorporating weighting into PRA
 How to derive weightings
Derived from expert opinion: Delphi
technique is a tool to achieve a consented
set of rankings from a pool of experts
Derived from multivariate data analysis:
principle components analysis (PCA)

Incorporating weighting into PRA

 Weighting derived from
principal components analysis (PCA)
PCA - a data transformation technique, which can reduce
the number of variables whilst accounts for the most of total
variance
PCA applied to PRA:
High variability in a risk factor means that it has the
potential to discriminate the level of risk between cases.
Variance is a measure of dispersion of risk scores
around the mean of each risk factor (variable)
Some risk factors are correlated

Incorporating weighting into PRA

 PCA
PCA was performed for data of 264 species and subset data of
individual pest groups, 7 risk factors used
Latent vector and percentage variation of principal component
(PC) were computed for data of all species (Table 3)
The first 5 PC axes account for almost 90% of the total
variation, which means number of principal risk factor can be
reduced
Economic impact and host range have the heaviest loadings in
PC1 that counts for 38.32% variation, which suggests they are
the two most important factors considered while assessing pest
risks

Incorporating weighting into PRA

 Table 3. Loadings & percentage variation
explained by each PC

Principal
Component (PC) PC1 PC2 PC3 PC4 PC5 PC6 PC7
loadings
Climate -0.25 -0.07 0.47 0.63 -0.54 -0.16 -0.01
Dispersal -0.35 -0.04 0.39 0.37 0.33 0.79 0.04
Economic -0.5 -0.26 0.15 0.2 0.54 -0.56 -0.12
Environment -0.04 0.24 0.85 0.33 -0.34 -0.06 0.03
Host -0.64 0.65 0.12 -0.38 -0.12 -0.02 0.04
Pathway -0.27 -0.49 0.11 -0.26 -0.23 0.03 0.75
Trading -0.28 -0.46 0.14 -0.33 -0.36 0.18 -0.65
Percentage
38.32 16.28 13.08 11.86 9 7.83 3.63
Variation (%) Incorporating weighting into PRA

Total variation (%) 54.6 67.68 79.54 88.54 96.37 100

PCA weightings for various pest categories
Weightings of individual risk factors for various pest
categories (Table 4)
Risk factors were sorted by weighting rank for all
species
Weightings of risk factors were not significantly different
because of the small number of variables
Weightings for some pest categories (nematode,
bacteria, phytoplasma and mite) were considered
unacceptable because of insufficient case of PRA data

Incorporating weighting into PRA

 Table 4. Weightings for various pest taxa

Weighting Weighting Weighting

Risk factor Weighting (Fungi - 78 (Insecta - (Viruses -
(All species) species) 67 species) 60 species)
Host 0.19 0.16 0.21 0.16
Economic 0.17 0.17 0.2 0.12
Dispersal 0.15 0.12 0.06 0.12
Trading 0.14 0.13 0.14 0.17
Climate 0.13 0.12 0.12 0.18
Pathway 0.12weighting into PRA0.15
Incorporating 0.14 0.11
Environment 0.11 0.15 0.13 0.14
 Is it possible to have a
generalised weighting pattern?
Rank correlation of weightings for various pest categories
examined by Spearman test, it revealed that
No any pair of pest categories are significantly correlated
on weighting rank, which means it is difficult to find a
general pattern of weighting that suits all the pest
categories
Insecta and Fungi have the most similar weighting rank;
correlations exists between Insecta and nematodes;
Nematodes and Bacteria have the most contrary weighting
rank

Incorporating weighting into PRA

 What can weighting do?
Summary assessment scheme
Summary assessment a quick qualitative risk
assessment for immediate action or with limited
information: low, medium, or high risk? Scheme comprised
(Figure 1):
Key risk factors identified producing most of the risk
Weightings derived showing the relative importance of
each factor
Correlations analysed showing the relationships of key
factors

Incorporating weighting into PRA

 Figure 1. Correlations & PCA derived weightings in a
summary risk assessment
Entry
Pathway 0.07 0.65
potential
Trading partners 0.05
Climate Establishment
suitability 0.18 potential
0.33
Host range 0.28 Spread
potential Dispersal
Summary risk potential 0.14
assessment 0.34
0.41
Environmental 25/10/02 - v14
Economic
impact 0.02 impact 0.27 0.42

0.4

Incorporating weighting into PRA

 Does summary scheme work?
Summary risk rating derived from weighted
averaging (7 factors involved), in comparison with
averaged overall risk and weighed averaged risk
from detailed risk assessment (45 risk factors
involved) (Figure 2)
Does this summary scheme work (Table 5 & Figure
3)?

Incorporating weighting into PRA

 9

Figure 2. overall risks: Comparison of summary and detailed assessment

8

7 Weighted average
Summary risk
Overall risk

6 Average risk (EPPO)

0
Pest species D.pon D.alb Cydia E.jac B.pas A.gla C.fum
C.ros H.cun D.sibi M.par D.stro X.altai A.sart S.mor
deros odent pomo obso siflor bripe iferan
a ea ricus allela bi cus a awitzi
ae ata nella ni ae nnis a
Weighted average 7.34 7.05 6.48 6.26 6.15 6.1 5.58 5.54 5.51 5.44 5.31 5.2 5.12 5.04 4.82
Summary risk 7.82 Incorporating
7.05 6.66 weighting into PRA
6.96 6.77 6.4 5.57 5.33 5.01 5.6 5.4 5.34 5.06 4.79 5.72
Average risk (EPPO) 6.36 6.43 5.84 5.75 5.8 5.61 5.2 5.45 5.66 4.93 5.36 5.2 5.27 5.5 4.75
 Does summary scheme work?
Table 5 Correlation
Correlations:
Average_risk_EPPO 1
Summary_risk 0.718884 1

Weighted_average 0.898227 0.912542 1

Average-
risk- Summary Weighted
EPPO _risk _average

Incorporating weighting into PRA

 Figure 3. Does summary scheme work?
8

8
7

Average risk (EPPO)

6

7
5
Summary risk

4
4 5 6 7 8
6 Weighted average

5
7

Average risk (EPPO)

6

4
5
4 5 6 7 8
Weighted average 4
4 5 6 7 8
Summary risk

Incorporating weighting into PRA

 Discussions
Incorporating weighting into PRA
Historical data of pest introductions and
invasions
Previous PRA cases
Expert opinion
Previous data do not necessarily apply to new
situations; however, these can provide at least a
starting point for new pests.
Incorporating weighting into PRA
 Conclusions
Weightings can be derived for individual risk elements/factors by
applying statistical techniques or assigning by expert judgement
identify the more important risk elements
filter out the factors that are low contributors to the overall
assessment while retaining the important ones
Without compromising the rigor we striving for in risk assessment
It is difficult to develop a generic weighting pattern for different
pest categories
A quick summary scheme can be developed, it gives a quick and
precautious idea of risk rating

Incorporating weighting into PRA

 Acknowledgement
Ministry of Agriculture and Forestry
Higher Education Funding Council for
England through University of
Greenwich

Incorporating weighting into PRA