Measurement Error

Measurement Error

In order to improve our processes, it is necessary to collect measurement data on the "critical
to" characteristics. When there is variation in this measurement data, it can either be attributed
to the characteristic that is being measured, or to the way that measurements are being taken,
which is known as measurement error. When there is a large measurement error, it affects the
data and may lead to inaccurate decision-making.

Measurement error is defined as the effect of all the sources of measurement variability that
cause an observed value to deviate from the true value of the characteristic that is being
measured. The measurement system is the complete process used to obtain measurements,
such as the procedures, gages and personnel inputs that are employed to obtain
measurements. Each component of this system represents a potential source of error. It is
important to identify these sources of error. This can be done by evaluating the measurement
system with statistical tools.

There are several types of measurement error which affect the location and the spread of the
distribution. Accuracy, linearity and stability affect location. Measurement accuracy describes
the difference between the observed average and reference value of the measurements. A
linearity problem describes a change in accuracy through the expected operating range of the
measuring instrument. A stability problem suggests that there is a lack of consistency in the
measurement over time. Precision is quantified as the standard deviation between these
replicated measurements. For estimating accuracy and precision, multiple measurements of
one single characteristic must be taken.
The primary contributors to measurement system error, however, affect the spread of the
distribution and describe the measuring system's repeatability (precision) and reproducibility.
Repeatability is the variation in measurements obtained by one operator measuring the same
characteristic on the same parts with the same measuring instrument. Reproducibility refers to
the variation in the average of measurements of an identical characteristic taken by different
operators using the same instrument.

Given that reproducibility and repeatability are important types of error, they are the object of a
specific study called a Gage Repeatability & Reproducibility study (Gage R&R). This study can
be performed on both attribute-based or variable-based measurement systems. It enables for
an evaluation of the consistency in measurements among operators after having at least two
operators measure several parts at random on a few trials. If there is no consistency, then the
measurement system must be improved.

A Variable Gage R&R study can be done with Minitab software. There are two methods which
can be used to estimate repeatability and reproducibility. Xbar and R charts are easy to use
and provide a graphical representation. ANOVA, a more powerful tool, is a statistical technique
that is used to estimate and analyze the components of variance, taking into account the
interaction between different parts and different operators. The graphical output of Minitab
software will provide both methods of analysis. With this output, it is possible to determine
whether or not the measurement system is reliable enough .

Key Questions

 What is meant by the phrase "measurement error" and what consequences can it have?

 What is a "measurement system" and what are the typical sources of measurement error?

 What is the difference between measurement "precision" and "accuracy"?

 What is the difference between "reproducibility" and "repeatability"?

 What is the process for estimating measurement precision and accuracy?

 What is the process for estimating measurement system reproducibility and repeatability?
Key Questions

Measurement error is a statistical term which describes the net effect of all the sources of
measurement variability that cause an observed value to deviate from the "true" value of the
characteristic that is being measured. A large measurement error affects the data and may lead
to inaccurate decision-making.

A measurement system is the complete process used to obtain measurements. Each

component of the process represents a potential source of error, such as procedures,
measuring instruments, personnel inputs (operators), environmental factors and materials.

Measurement precision refers to the ability of a measuring instrument to reproduce the same
measurements. Measurement accuracy refers to the ability of a measuring instrument to give
measurements which are close to the "true" value.

Repeatability is the variation in measurements obtained by one operator measuring the same
characteristic on the same parts with the same measuring instrument. Reproducibility refers to
the variation in the average of measurements of an identical characteristic made by different
operators using the same instrument.

For estimating precision and accuracy, multiple measurements of one single characteristic must
be taken. Precision is quantified as the standard deviation between these replicated
measurements. Accuracy is quantified as the difference between the observed average of
measurements and the reference value.

For estimating reproducibility and repeatability, Gage R&R studies should be performed

Measurement System As A Process

"We can visualize the components of this system with the following cause-and-effect diagram.
Each of these components, Material, Method, Machine, Environment and People represents a
potential cause of measurement error. For example, let's consider the Machine component as
the measuring instrument, where the cleanliness, the temperature, the design, the calibration
etc. can have an impact on the measurement error."

so a measurement system can be influenced by many factors that may have a direct effect on
the quality of the measurements that have been taken.so What do we do to prevent this from
happening ,it is important to identify the factors that specifically influence the measurement
system."

Categories of Measurement Error Which Affect the location

,just like every process, the distribution used to describe the measurement system's variation
can be characterized by the location and the width or spread. There are several categories of
measurement which affect the location."

"Let's say you measure one characteristic with a measuring instrument, such as a gage, and
you note that there is a systematic difference between the true or reference value of the product
or service characteristic and the measurement that you have obtained. Well, depending upon
the situation, the types of error that affect location can be referred to as accuracy, linearity and
stability."

"The accuracy of the measuring instrument is reflected in the extent to which the average of the
repeated measurements on the same product agree with the "true" or reference value of the
product. You may also hear someone refer to accuracy as "bias"."

but how do we know what is the true value of the product? the true value serves as an agreed
upon reference for comparison. This can be obtained through research in various organizations
or by simply taking an average of several measurements that have been obtained with
measuring equipment with more accuracy, such as metrology lab equipment."

when we are evaluating our measurement system, if there is a change in accuracy through the
expected operating range of the instrument.This would mean there is a linearity problem, where
we would find that at the limit of the measuring instrument's range, the error is proportionally
larger than at it's middle range."

"For example, comparing a 0 to 6 inch caliper with the appropriate standard shows that the
instrument is relatively accurate between 0" and 4.55", but errors will occur between 4.551" and
6". This change in accuracy throughout the range of the caliper is a linearity problem."

"Stability refers to the consistency of measurements over time. If there is change among the
measurements over time, stability is a problem".

but how do we evaluate accuracy, linearity, and stability?.we usually take multiple, repeated
measurements on master, or standard parts. Accuracy is calculated by taking multiple
measurements on a part and calculating the difference between the observed average and the
reference value. Linearity is calculated in a similar way, however, we compare measurements
to different master parts throughout the operating range of the measurement instrument."

"For the stability, a single characteristic on the same master part(s) should be measured over
an extended time period. We can monitor the results through graphical output such as a control
charts."

Categories of Measurement Which Affect the Spread

the importance of identifying the variation due to the measurement system in order to reduce
process variation. "repeatability and reproducibility are important contributors to measurement
error".but what is the difference between repeatability and reproducibility?, although they both
affect the spread of the distribution, they focus on different factors".

"Repeatability of the measurement process implies that the measurement process variability
itself is consistent. It is the variation in the measurements obtained with one measuring
instrument when one operator uses the same gage for measuring identical characteristics of
the same parts."
precision is actually the same thing as repeatability, but instead of focusing on the
measurement system, it usually refers to evaluating a measuring instrument in a metrology lab
environment".but here we are focusing on the system.

"We also focus on the measurement system for reproducibility. However, reproducibility of the
measurement system implies that variability among the operators is consistent. Another way of
describing this is that operator variability represents the incremental bias that can be attributed
to each operator. In this case, reproducibility is the variation in the average of measurements
made by different operators using the same gage when measuring identical characteristics of
the same parts."

Evaluating Accuracy and Precision of an Instrument

let's take the 3 most commonly produced hole sizes and, on the master ring gages that
represent these holes, take 30 repeated measurements with one operator. We will be able to
determine the accuracy of the instrument by quantifying the difference between the average of
the 30 repeated measurements and the reference value of the master ring gages, as found with
metrology equipment. We will determine the precision of the instrument by quantifying the
sample standard deviation of the 30 measurements."

Exploring Linearity Error

now that we have completed an accuracy and precision study on the measuring system, we
can conclude that the system is quite accurate and reliable, however we don’t know if this gage
has the same accuracy for all of the sizes of objects being measured. this could be linearity
error in this instrument. Actually, there is a gage linearity study in Minitab that could help us
determine this."
We already have data from our first study which consists of 30 repeated measurements on
each of three master ring gages. These don't represent the complete range of the measuring
device, but we want to know if there is a big difference in accuracy between the three hole
sizes. We can visualize the linearity of a gage through Minitab which shows a best-fit line
relating the average deviation to the master measurements. This will tell us if our gage has the
same accuracy for all the sizes of objects being measured. Generally, the closer our slope is to
zero, the better the gage linearity."

"If we look at our data, there does seem to be a linearity effect because the accuracy is linearly
increasing with the size of the holes. However, on the other hand, if we look at the Minitab
output, the variation that is due to linearity and accuracy for this gage is 0.01% and 0.3% for the
overall measurement system variation. So overall, we can conclude that the linearity error
between the three holes for this instrument is not overly important."

Attribute Gage Study

a Critical to Quality characteristic that we are measuring is rivet bucking height. The first thing
that we need to do is to evaluate our measurement system. Although this characteristic is
continuous by nature, we are currently measuring it with a go/no-go gage. So, in order to
evaluate this measurement system, we perform an Attribute Gage R&R."

we select 20 rivets that have been installed on sample parts. Referring to the specifications of
these parts,we know which ones that have been installed slightly below or above limits. we take
note of the rivets that are good and bad and enter them into data spreadsheet in Excel, under
the attribute column. This will enable us to then determine how consistently operators evaluate
a set of samples against a known standard, which we refer to as "attribute". This will provide us
with the data that is necessary to evaluate the measurement system

Attribute Gage Study With Excel

"On the spreadsheet, the appraiser's score shows us how consistently the operators are able to
repeat their own measurements. This score is obtained by comparing the operator's trials for all
samples to determine the proportion of consistent scores. The value referred to as the score
versus attribute shows us how consistent the operators' measurements are with the known
standard, under the attribute column."

"Here, we take the attribute, trial 1 and trial 2, and calculate the proportion of scores that are
consistent across the samples. The results of the first operator reveal a score of 95% (19/20)
between the two trials, and 90% (18/20) for the score versus the attribute value. For operator 2,
the appraiser score is 100% and the %score vs. attribute is 95%."

"We also want to know how consistent our measurement decisions are overall. In this case, we
want to know the effective scores for both operators. And, then, we want to compare the
effective score with the standard score. The target value is 100%, meaning that all
measurement decisions are consistent."

"In our case, both scores are 85%. We obtain these numbers by comparing all of the operators'
scores for both trials in all 20 samples and determining the proportion of consistent samples.
When measurement decisions are not consistent, it means that we should improve our
measurement system by improving the measuring method and/or the gage itself, or by
providing better training to the operators."

Evaluating Repeatability and Reproducibility With ANOVA

it is important to note that the ANOVA method of analysis will provide us with a more accurate
assessment of the measurement system study than the Xbar and R Chart method.

ANOVA stands for Analysis of Variance. This is a standard statistical technique which attempts
to analyze the variation between measurement observations and then identify the important
contributing factors. When we do a Gage R&R, both the ANOVA and the Xbar and R chart
method break down the measurement system variation into Reproducibility and Repeatability.
However, the ANOVA method allows us to go one step further in our analysis by breaking down
reproducibility into Operator and Operator by Part components."

The Operator by Part component explores the interaction between each operator and part. This
allows us to quantify the variation between average part measurements for each operator,
enabling us to account for different situations where, for example, one operator may obtain
more variation when measuring smaller parts rather than larger parts, or vice versa."

Another reason why the ANOVA is more accurate is because the estimators that are calculated
are of a higher quality. With the Xbar and Range Chart, the range is used to estimate the
variation, but with the ANOVA, the variance is used. This is a more precise estimator that we
call variance components."

ANOVA / Variance Component Analysis

While analyzing a Gage R&R study with Minitab,the ANOVA method partitions the total
variance in the measurements into different components. For a traditional Gage R&R study, the
variation is broken down into four categories of components: Operator, Part-to-Part, Operator
by Part and Repeatability."

"By looking at the operator component, it allows us to quantify the variation observed between
different operators who are measuring the same set of parts. As for the part component, it
allows us to quantify the variation observed for a characteristic measured on different parts,
regardless of the operator. And, looking at the Operator by Part component explores the
interaction between the operators and the parts. Finally, the repeatability quantifies the variation
due to the instrument itself and the position of the parts in the instrument. "

And there is an ANOVA table. The information in this table is then used to calculate the
variance components.

Graphical Output of a Gage R&R Study With Minitab

now that we have the graphical output of Gage R&R study, we can interpret it and make some
decisions. the first two graphs represent the Xbar and R charts.
"To begin with, note that all of the points on the Xbar chart represent the averages of the
repeated measurements on the parts for the three operators. The range chart represents the
range, which is the maximum value minus the minimum value of the repeated measurements
on the parts for the three operators."

"On the Xbar charts, since the measurements are outside of the control limits, they are
indicating that variation is mainly due to differences between parts. As for the range chart, it is
okay because none of the range values are too large since they are within the control limits. By
the way, control limits are variation limits for repeatability error. On the Xbar, they are compared
with the part averages, and on the Range chart, they limit the repeatability error for each part.
The pattern we see on these charts indicates that there is no problem."
Xbar and R Chart

on the Xbar chart, we see that the averages of the repeated measurements on each part for the
three operators are quite similar. The pattern is virtually the same. Since the operators have
obtained the same measurements for the same parts, can we conclude that the variability
among them is consistent ?"

By looking at thr range chart for operator #3 we can see that these ranges are lower compared
to the other operators.It makes sense though because operator #3 is actually a Quality
Inspector who has a more consistent measuring method than the others."

Graphical Representation of Operator and Part Components

besides the Xbar and R charts, the graphical Minitab output of a Gage R&R study also provides
us with graphs representing operator and part components."

"The Operator*Part Interaction graph shows us the effect of different parts on each operator's
measurements. By looking at the average measurements of each part for each operator, we
can view whether or not differently configured parts affect an operator's measurements. In this
case, since the lines are almost parallel, we can see that between the operators, there is not
much difference in the measurements for each part."
on the By Operator graph, we see that the line connecting the average measurements between
the operators is horizontal. So, this tells us that the averages between these operators are
virtually the same".

"The next thing we want to know is if there is any variation between the parts. The By Part
graph, representing the variation in average measurements taken on the same characteristic
for each part, shows us that there are in fact large differences among the parts. The non
horizontal line reveals that when we just look at the parts themselves, we see a lot variation
among the measurements of the characteristic."