BASIC STATISTICS FOR THE

ESTIMATION OF
UNCERTAINTY OF
MEASUREMENT
By Wesley Johnson & Jane Weitzel

QUALITY ANALYSIS
CONSULTANTS

Outline
Normal distribution and measurement
statistics
Significance testing
Regression methods
Analytical Data Quality
Process Capability
QUALITY ANALYSIS
CONSULTANTS

Goals of Presentation
123 ppm
106 to 140 ppm
QUALITY ANALYSIS
CONSULTANTS

Normal Distribution and

Measurement Statistics

QUALITY ANALYSIS
CONSULTANTS

Mean, Standard Deviation and

Variance
Mean x

x /n
i

Standard Deviation s
Variance s
2

2
(
x

x
)
i

2
(
x

x
)
i

(n 1)

(n 1)

QUALITY ANALYSIS
CONSULTANTS

Mean, Standard Deviation and

Variance
xi

(xi- x)

(xi- x)2

15.2

0.143

0.0204

14.7

-0.357

0.1257

15.1

0.043

0.0018

15.0

-0.057

0.0033

15.3

0.243

0.0590

15.2

0.143

0.0204

14.9

-0.157

0.0247

x = 15.057

= 0.2572
QUALITY ANALYSIS
CONSULTANTS

Mean, Standard Deviation and

Variance
xi

(xi- x)

(xi- x)2

15.2

0.143

0.0204

14.7

-0.357

0.1257

0.043

0.0018

-0.057

0.0033

0.243

0.0590

15.2

0.143

0.0204

14.9

-0.157

0.0247

15.1

16.0

15.0
15.3

16.5

x = 15.057
15.3571

= 0.2572 3.1671
QUALITY ANALYSIS
CONSULTANTS

Mean, Standard Deviation and

Variance
Mean = 105.4/7 = 15.057
Standard Deviation = (0.2572/6)1/2 = 0.0207
Variance = 0.2572/6 = 0.04287

QUALITY ANALYSIS
CONSULTANTS

Normally Distributed Data

The most frequently observed value is the mean or
average
The values are distributed symmetrically about the
mean
There are more values closer to the mean than farther
away
If there is no systematic error, the mean approaches
the true value
Random error results can be treated statistically using
the laws of probability
QUALITY ANALYSIS
CONSULTANTS

Simulated Sampling Experiment

INTERVAL
<7.00

TALLY COUNT

FREQUENCY

11

7.00 7.99

11111 1111

8.00 8.99

11111 11111 11111 11111 11111 1

26

9.00 9.99

11111 11111 11111 11111 11111 11111 1

31

10.00 10.99

11111 11111 11111 1111

19

11.00 11.99

11111 11111 1

11

11

> 11.99

QUALITY ANALYSIS
CONSULTANTS

10

HISTOGRAM OF SAMPLING EXPERIMENT

35

30

25

20
Series2
15

10

0
6.5

7.5

8.5

9.5

10.5

11.5

12.5

RANGE

QUALITY ANALYSIS
CONSULTANTS

11

Frequency

Normal Distribution Plot

40
30
20
10
0
2

26

31

19

11

6.5

7.5

8.5

9.5

10.5

11.5

12.5

Interval

QUALITY ANALYSIS
CONSULTANTS

12

Interlaboratory Data
Trial No.

Set 1
ppm Pb

Set 2
ppm Pb

Set 3
ppm Pb

1
2
3
4
5
6
7
8
9
10

10
14
6
11
13
8
9
7
12
10

12
8
4
15
6
12
5
17
6
15

12
14
9
11
10
15
13
8
16
12

mean =
s=

10.0
2.6

10.0
4.76

12.0
2.6

QUALITY ANALYSIS
CONSULTANTS

13

Data Distribution
Frequency

Gaussian curves
Lab 1
Lab 2
Lab 3
0

10

15

20

Concentration

QUALITY ANALYSIS
CONSULTANTS

14

Normal Error Curve

-4

-3

-2

-1

s
QUALITY ANALYSIS
CONSULTANTS

15

QUALITY ANALYSIS
CONSULTANTS

16

Z Factors
Z is in units of standard deviations and is a
measure of how far a value is from a mean
or a limit or some other value.

( x x)
Z
s
QUALITY ANALYSIS
CONSULTANTS

17

P Values
P is the proportion of observations expected
to lie outside a specified range.
P can also refer to the proportion of process
output beyond a specified limit that is Z
standard deviation units away fro the
process average.
QUALITY ANALYSIS
CONSULTANTS

18

Significance Testing

QUALITY ANALYSIS
CONSULTANTS

19

One-tailed and Two-tailed

Distributions
If there is a requirement to express the
probability that a value will lie outside of a
range (either above or below it) two-tailed
probabilities are used.
If there is a requirement to express an upper
or a lower limit below which or above
which specified proportions of data points
will lie, one-tailed probabilities are used.
QUALITY ANALYSIS
CONSULTANTS

20

QUALITY ANALYSIS
CONSULTANTS

21

Confidence Interval for a Mean

This is a significance test for comparing a
mean to a true value.
It tells, with a given level of confidence,
where the population mean lies with respect
to the sample mean.

ts
CI for = x
n
QUALITY ANALYSIS
CONSULTANTS

22

Two-tailed t Table
Factor for Confidence Interval
(n-1)

90%

95%

99%

6.31

12.7

63.7

2.92

4.30

9.92

2.13

2.78

4.60

1.81

2.26

3.25

14

1.76

2.14

2.98

1.64

1.96

2.58

QUALITY ANALYSIS
CONSULTANTS

23

VAMSTAT Review

QUALITY ANALYSIS
CONSULTANTS

24

Confidence Interval for the

Estimate of Standard Deviation

QUALITY ANALYSIS
CONSULTANTS

25

Statistical Tolerance Interval

This describes the limits within which a
specified percentage of the population is
expected to lie with a given probability.
(from J.K. Taylor)
Tolerance interval = x ks where k is taken
from a table of factors (two-tailed)
QUALITY ANALYSIS
CONSULTANTS

26

A Significance Test to see if Two

Means are Different (1)
Calculate spooled and tobs from
s pooled

(n1 1)( s1 ) 2 (n2 1)( s2 ) 2

n1 n2 2

tobs

( x1 x2 ) 1 1

s pooled
n1 n2

QUALITY ANALYSIS
CONSULTANTS

27

A Significance Test to see if Two

Means are Different (2)
Compare tobs with tcrit from a two-tailed table
of t values for the appropriate degrees of
freedom (n1 + n 2 2) and confidence level
If tobs < tcrit you cant say the means are
different at that confidence level
QUALITY ANALYSIS
CONSULTANTS

28

Paired t-test (1)

This is used to test for a bias between two methods
(or analysts, or laboratories) when running
samples of varying composition
Calculate the mean difference ( xdiff) and the
standard deviation of the differences (sdiff) between
the two sets of results for each of n samples and
calculate tobs from

tobs x diff

sdiff

QUALITY ANALYSIS
CONSULTANTS

29

Paired t-test (2)

If tobs < tcrit taken from a two-tailed t table
then there is no evidence that there is a bias
between the two at that confidence level for
n-1 degrees of freedom.

QUALITY ANALYSIS
CONSULTANTS

30

A significance Test for a Comparison

of Variances - the F-test

s1 = standard deviation from n1 measurements

s2 = standard deviation from n2 measurements
s1 > s2, degrees of freedom = n-1
Calculate F = s12/s22
Look up the critical value Fc for the appropriate
degrees of freedom () for each set.
If F > Fc then s1 > s2 is significant at the confidence
level
If F < Fc then s2 is not significantly less than s 1.
QUALITY ANALYSIS
CONSULTANTS

31

Significance Testing for Outliers

There are many tests available:

Dixon Test
Grubbs Test
Huge Error Test
Kolmogorov-Smirnov Test

It is important to test suspect data points in

order to apply the statistics described here
QUALITY ANALYSIS
CONSULTANTS

32

The Huge Error Test

M = |xs-xm|/s where xs is the suspect data
point and xm and s are calculated with xs
excluded.
If M > 4, the data point is rejected. If n 8
then rejection is considered correct with a
probability of 95%
QUALITY ANALYSIS
CONSULTANTS

33

Regression Methods

QUALITY ANALYSIS
CONSULTANTS

34

Regression Methods
Regression methods are commonly
understood as a means of fitting the best
straight line through a calibration data set
There are other non linear relationships
We will deal with least squares technique as it
is most common

y = + x
QUALITY ANALYSIS
CONSULTANTS

35

Residuals
The least squares regression produces a line
describing the relationship between x and y
such that the sum of the squares of the
differences between the actual experimental
values of y (yi) and those that can be
calculated from the fitted line i is a
minimum. The values (yi - i) are called
residuals.
QUALITY ANALYSIS
CONSULTANTS

36

Residuals and Goodness of Fit

The residuals should be randomly
distributed about a mean of 0 with a
standard deviation sy/x
If a plot of sy/x vs x is made, there should be
a random distribution of sy/x about a line at 0
with 2/3 of the points lying within a region
that is 2 sy/x of the mean line at 0
QUALITY ANALYSIS
CONSULTANTS

37

Residuals and Goodness of Fit

A common figure that is generated in
computer regression calculations is the R 2
value the proportion of the variance due
to the regression. The better the data fit a
straight line, the closer R2 will be to 1. This
can be misleading however, and there
should be a visual inspection of the plotted
data to confirm the relationship is linear.
QUALITY ANALYSIS
CONSULTANTS

38

An R2 Value Greater Than 0.99 Can Be

Obtained From a Non-linear Calibration.
6
2

y = -0.0393x + 1.2079x - 0.0929

R = 0.997

Re ading

4
3
2

y = 1.0114x + 0.0381
2

R = 0.9938

1
0
0

-1
Concentration

QUALITY ANALYSIS
CONSULTANTS

39

Calibration as a Source of MU
For regression the uncertainty in xo is given
by the equation:
s xo

sy / x
b

1 1
( yo y ) 2
2
m n b ( xi x ) 2

QUALITY ANALYSIS
CONSULTANTS

40

Calibration as a Source of MU
This estimate of the standard deviation of xo can
be used as a contribution from the calibration step
and the short term instrument variability when
estimating the overall measurement uncertainty of
the measurement process. If it is determined to be
less than 1/3 of the largest source of uncertainty
(such as sampling uncertainty for example) in the
overall measurement procedure then this can be
demonstrated and its contribution ignored.
QUALITY ANALYSIS
CONSULTANTS

41

Analytical Data Quality

QUALITY ANALYSIS
CONSULTANTS

42

Method Validation as a
Source of MU Information
We will discuss the method validation
report as a source of information for
determining MU

QUALITY ANALYSIS
CONSULTANTS

43

Reference
The Fitness for Purpose of Analytical
Methods
A Laboratory Guide to Method Validation
and Related Topics
Eurochem Working Group

QUALITY ANALYSIS
CONSULTANTS

44

Validation Options
Standard methods (AOAC etc.) Validation
ISO 17025 5.4.2

In-house method validation

ISO 17025 5.4.4

QUALITY ANALYSIS
CONSULTANTS

45

Standard Methods
If a method is validated by a standards
organization then the user need only
establish performance data for their own
use
ISO 17025 5.4.2
QUALITY ANALYSIS
CONSULTANTS

46

Standard Methods
Performance of the method in your lab
includes determining
Accuracy
Precision

This data can be used to determine MU

QUALITY ANALYSIS
CONSULTANTS

47

Standard Methods
Interlaboratory precisions can be used for MU
(AOAC)
See the CAEAL policy
In the absence of any other source of repeated data

Will overestimate precision for one lab

That is, MU kxSR
With k = coverage factor (between 2 and 3, depending on the
number of labs)
SR = Reproducibility SD = (Sr2+SL2)
Sr = Repeatability SD
SL = Between-laboratory SD
QUALITY ANALYSIS
CONSULTANTS

48

In House Methods
Accuracy
Uncertainty of CRM
Bias, Recovery
10 repetitions or 3 concn x 3 repetitions

Precision
Intra assay, inter assay, repeatability,
intermediate precision, reproducibility
10 repetitions or 3 concn x 3 repetitions
QUALITY ANALYSIS
CONSULTANTS

49

In House Methods
LOD, LOQ
Determine if precision varies with
concentration

Ruggedness
Good source of precision information

Calibration
The contribution to variation from calibration
may be included in accuracy and precision tests
QUALITY ANALYSIS
CONSULTANTS

50

Internal Quality Control

Measures included in routine analysis to verify
analytical performance
E.g.
Reference material (CRM and In House)
Spikes
Replicate determinations on actual samples
Especially duplicates

Control Charts
Blanks
QUALITY ANALYSIS
CONSULTANTS

51

Fitness for Purpose

The entire purpose of an analysis is to
determine if something is fit for use
Is the water fit for people to drink?
Is the soil fit for agricultural use?
Is the effluent fit for adding to the local sewage
system?

To assess fitness for use the MU must be

known
QUALITY ANALYSIS
CONSULTANTS

52

Variation From Sampling

Sampling occurs when original sample is
taken and when sub-samples are taken in
the lab
Lab is responsible for variation introduced by
their sub-sampling
Use of replicates (duplicates) is an effective
technique to determine variation from subsampling
QUALITY ANALYSIS
CONSULTANTS

53

Variation from Sampling

Fitness for purpose should include any
uncertainty introduced as a result of any
sampling that is a part of the process,
including that prior to the submission of
a portion to the laboratory for analysis.
(x)2 = (sam)2 + (an)2
QUALITY ANALYSIS
CONSULTANTS

54

Fitness for Purpose

The end user of the data can use the MU to
determine if the something is fit for use
They can now know how much variation is
caused by the analytical determination

Compare value to a specification

A commonly used parameter is process
capability
QUALITY ANALYSIS
CONSULTANTS

55

Process Capability
Some formulas
Cpl = ( X LSL)/3
Cul = (USL - X)/3
Cpk = min{Cpl, Cpu}
LSL = Lower Specification Limit
USL = Upper Specification Limit
QUALITY ANALYSIS
CONSULTANTS

56

Conclusion
123 ppm
106 to 140 ppm
QUALITY ANALYSIS
CONSULTANTS

57