1

An Overview of
the Methodology & Concepts
Presented by: Larry Scott
Principal: Process Technologies
Welcome to:
Animation by Nark Anderson @ Stat-Ease !nc.
DOE Strategies:
Agenda
Intro to DOE:
Full Resolution, Two-Level Factorials
Example: Perfect Popcorn
Fractional Factorials
Regular Fraction
Irregular Fraction
Minimum Run Resolution IV & V
Overall Strategy of a Factorial DOE
2
Many of the most
useful designs are
extremely simple.
Sir Ronald Fisher
Father of Factorial Design
Process Process
Controllable Factors (X) Controllable Factors (X)
Responses (Y) Responses (Y)
Uncontrollable Variables (Z) Uncontrollable Variables (Z)
What is DOE: What is DOE: 6 6- -sigma enthusiast Y = f (x sigma enthusiast Y = f (x
i i
) )
DOE is:
A series of tests,
in which purposeful changes
are made to input factors,
so that you may identify causes
for significant changes
in the output responses.
3
Presentation Intent
An overview of the strategy of
DOE with a focus on the concept of
resolution.
Resolution is probably the most
under utilized opportunity in DOE,
resulting from lack of awareness.
Road Map
+
History
Fisher & Yates; DOE concepts 1920
Plackett & Burman designs - 1940s
1st Textbook: Box Hunter & Hunter - 1960
Optimal Designs: a- , d- 1960s
Douglas Montgomery resurgence 1976
DOE software commercially available 1980s
Genechi Taguchi 1980s
Six Sigma drives interest in DOE - 1980s
Minimum Run Designs; Whitcomb & Oley - 1999
Experiment Requirements
System Signal estimates
System Noise (variance) estimates
System Power
Resource Availability $$$$$$
# of Model Coefficients
System Resolution
5
Experiment Requirements
System Signals
reflect the magnitude of the output
are a function of the range or spread
of the input variables.
X1: Temperature - Low = 50 deg C
High = 100 deg C
X2: Pressure - Low = 10 psi
High = 15 psi
Experiment Requirements
System Signal estimates
System Noise (variance) estimates
System Power
system signal
variance level
confidence level
Resource Availability $$$$$$
# of Model Coefficients
System Resolution
6
Experiment Requirements
System Signal estimates
System Noise (variance) estimates
System Power
Resource Availability $$$$$$
# of Model Coefficients
Y = b
0
+ b
1
X
1
+ b
2
X
2
+ b
12
X
1
X
2
System Resolution
Full Resolution, TwoLevel Design
Run high/low combos of two or more factors
Use statistics to identify the critical few
Main effects A, B
Interactions (the hidden gold!) AB
What could be simpler?
Y = b
0
+ b
1
A
1
+ b
2
B
2
+ b
12
A
1
B
2
7
Two Level Factorial Design
As Easy As Popping Corn!
Kitchen scientists* conducted a 2
3
factorial experiment
on microwave popcorn. The factors are:
A. Brand of popcorn
B. Time in microwave
C. Power setting
Response 1: Taste (1-10)
Response 2: Weight (un-popped kernels - UPKs).
* For full report, see Mark and Hank Andersons' Applying DOE to Microwave
Popcorn, PI Quality 7/93, p30.
Two Level Factorial Design
Factors in coded values
1 3.1 74 8
7 0.5 42 + + 7
8 0.3 32 + + + 6
6 0.7 77 + + 5
4 1.2 80 + + 4
5 0.7 81 + 3
3 1.6 71 + 2
2 3.5 75 + 1
Order oz. rating percent minutes expense Order
Std UPKs Taste Power Time Brand Run
R
2
R
1
C B A
8
Two Level Factorial Design
Actual factor values
* Average scores multiplied by 10 to make the calculations easier.
1 3.1 74 75 4 Cheap 8
7 0.5 42 100 6 Cheap 7
8 0.3 32 100 6 Costly 6
6 0.7 77 100 4 Costly 5
4 1.2 80 75 6 Costly 4
5 0.7 81 100 4 Cheap 3
3 1.6 71 75 6 Cheap 2
2 3.5 75 75 4 Costly 1
Order oz. rating* percent minutes expense Order
Std UPKs Taste Power Time Brand Run
R
2
R
1
C B A
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
9
R
1
- Popcorn Taste
Average A-Effect
75 74 = + 1
80 71 = + 9
77 81 = 4
32 42 = 10
42 32
77 81
74 75
71 80
P
o
w
e
r
Brand
T
i
m
e
A
1 9 4 10
y 1
4
+
= =
There are four comparisons of factor A (Brand), where
levels of factors B and C (time and power) are the
same:
( )
y y
Effect y
n n
+
+
=

A
75 80 77 32 74 71 81 42
y 1
4 4
+ + + + + +
= =
42 32
77 81
74 75
71 80
P
o
w
e
r
Brand
T
i
m
e
R
1
- Popcorn Taste
Average A-Effect
10
R
1
- Popcorn Taste
Analysis Matrix in Standard Order
I for the intercept, i.e., average response.
A, B and C for main effects (ME's).
These columns define the runs.
Remainder for factor interactions (FI's)
Three 2FI's and One 3FI.
32 + + + + + + + + 8
42 + + + + 7
77 + + + + 6
81 + + + + 5
80 + + + + 4
71 + + + + 3
75 + + + + 2
74 + + + + 1
Taste
rating ABC BC AC AB C B A I
Std.
Order
Popcorn Taste
Compute the effect of C and BC
y y
y
n n
+
+
=

C
BC
81 77 42 32 74 75 71 80
y
4 4
y
4 4
+ + + + + +
= =
+ + + + + +
= =
-3.5 -6 0.5 -20.5 -1 y
32 + + + + + + + 8
42 + + + 7
77 + + + 6
81 + + + 5
80 + + + 4
71 + + + 3
75 + + + 2
74 + + + 1
rating ABC BC AC AB C B A Order
Taste Std.
11
Popcorn Taste
Compute the effect of C and BC
y y
y
n n
+
+
=

C
BC
81 77 42 32 74 75 71 80
y
4 4
y
17
74 75 42 32 71 80 81 7
2
4
7
1.5
4
+ + + + + +
= =
+ + +

+ + +
= =
-3.5 -21.5 -6 0.5 -17 -20.5 -1 y
32 + + + + + + + 8
42 + + + 7
77 + + + 6
81 + + + 5
80 + + + 4
71 + + + 3
75 + + + 2
74 + + + 1
rating ABC BC AC AB C B A Order
Taste Std.
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
12
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
improve employee morale
2
3
4
5
6
1
Sparsity of Effects Principle
Trivial Many: the remainder that result from random variation.
These effects will be centered on zero.
Since they are based on averages,
you can assume normality
by the Central Limit Theorem*.
Two types of effects:
Vital Few: the big ones we want to catch
20 % of ME's and 2FI's will be significant.
13
Estimating Noise
How are the trivial many effects distributed?
Hint: Since the effects are based on averages
you can apply the Central Limit Theorem.
Hint: Since the trivial effects estimate noise
they should be centered on zero.
How are the vital few effects distributed?
No idea! Except that they are too large to be
part of the error distribution.
Half Normal Probability Paper
Sorting the vital few from the trivial many.
7.14
21.43
35.71
50.00
64.29
78.57
92.86
P
i
0
|Effect|
BC
B
C
Significant effects:
The model terms!
Negligible effects: The error estimate!
1+
Half-Normal Probability Paper
Significant effects
(the vital few) fall
abnormally high (to
the right) on the
absolute effect
scale. These are
the keepers.
What do you do with
the little ones?
7.14
21.43
35.71
50.00
64.29
78.57
92.86
P
i
0
|Effect|
A
B
AB
3 6 9
12
15
Analysis of Variance (taste)
Sorting the vital few from the trivial many.

Source
Sum of
Square

df
Mean
Square
F
Value

Prob > F
Model 2343.0 3 781.0 31.5 0.001<p<0.005
Residual 99.0 4 24.8
Cor Total 2442.0 7

1. Add SS for significant effects: B, C & BC.
Call these vital few the Model.
2. Add SS for negligible effects: A, AB, AC & ABC.
Call these trivial many the Residual.
15
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
16
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
Popcorn Analysis
with Software
Using statistical software, lets see how
well the programming staff did on this
exercise and compare results.
17
Software Packages
Design-Expert 7 (Stat-Ease, Inc.)
Minitab
JMP (SAS)
Statgraphics
Statistica
Qualitek 4
Popcorn Analysis Taste
Half Normal Plot of Effects
Design-Expert Software
Taste
Shapiro-Wilk test
W-value = 0.973
p-value = 0.861
A: Brand
B: Time
C: Power
Positive Effects
Negative Effects
Half-Normal Plot
H
a
l
f
-
N
o
r
m
a
l

%

P
r
o
b
a
b
i
l
i
t
y
|Standardi zed Effect|
0.00 5.38 10.75 16.13 21.50
0
10
20
30
50
70
80
90
95
99
B
C
BC
18
Popcorn Analysis Taste
Pareto Chart of t Effects
Pareto Chart
t
-
V
a
l
u
e

o
f

|
E
f
f
e
c
t
|
Rank
0.00
1.53
3.06
4.58
6.11
Bonferroni Limit 5.06751
t-Value Limit 2.77645
1 2 3 4 5 6 7
BC
B
C
( )
0.05
df 4
2
t 2.77645
=
=
=
( )
0.05
2
df 4
k 7
t 5.06751
=

=
=


=
Popcorn Analysis Taste
ANOVA
Analysis of variance table [Partial sum of squares]
Sum of Mean F
Source Squares df Square Value Prob > F
Model 2343.00 3 781.00 31.56 0.0030
B-Time 840.50 1 840.50 33.96 0.0043
C-Power 578.00 1 578.00 23.35 0.0084
BC 924.50 1 924.50 37.35 0.0036
Residual 99.00 4 24.75
Cor Total 2442.00 7
19
Popcorn Analysis Taste
Predictive Equations
For process understanding, use coded values:
1. Regression coefficients tell us how the response changes
relative to the intercept. The intercept in coded values is
in the center of our design.
2. Units of measure are normalized (removed) by coding.
Coefficients measure half the change from 1 to +1 for
all factors.
Actual Factors:
Taste =
-199.00
+65.00*Time
+3.62*Power
-0.86*Time*Power
Coded Factors:
Taste =
+66.50
-10.25*B
-8.50*C
-10.75*B*C
Real-World Example: Breakthrough
BC Interaction!
86
72
58
++
30
Power (-)
Power (+)
- Time +
T
a
s
t
e
20
Popcorn Taste
BC Interaction
37.0 32 42
+ +
79.0 77 81
+
75.5 80 71
+
74.5 75 74
Taste C B
B- 4 min B+ 6 min
80
70
60
50
40
30
Taste
C- 75%
C
+

1
0
0
%
Popcorn Analysis Taste
Interaction Plot (BC)
Design-Expert Software
Taste
Design Points
C- 75.000
C+ 100.000
X1 = B: Time
X2 = C: Power
Actual Factor
A: Brand = Cheap
C: Power
4.00 4.50 5.00 5.50 6.00
Interaction
B: Time
T
a
s
t
e
30.0
44.0
58.0
72.0
86.0
21
Popcorn Analysis Taste
Contour Plot (BC)
Design-Expert Software
Taste
Design Points
81
32
X1 = B: Time
X2 = C: Power
Actual Factor
A: Brand = Cheap
4.00 4.50 5.00 5.50 6.00
75.00
81.25
87.50
93.75
100.00
Taste
B: Ti me
C
:

P
o
w
e
r
40.0
45.0
50.0
55.0
60.0
65.0
70.0
75.0
75.0
Popcorn Analysis Taste
3D Plot (BC)
4.00
4.50
5.00
5.50
6.00
75.00
81.25
87.50
93.75
100.00
37.0
47.8
58.5
69.3
80.0


T
a
s
t
e


B: Time
C: Power
22
Popcorn Analysis Taste
AC Interaction Plot Comparison w/ 3D Plot
4.00
4.50
5.00
5.50
6.00
75.00
81.25
87.50
93.75
100.00
37.0
47.8
58.5
69.3
80.0


T
a
s
t
e


B: Time
C: Power
C: Power
4.00 4.50 5.00 5.50 6.00
Interaction
B: Time
T
a
s
t
e
30.0
44.0
58.0
72.0
86.0
C-
C+
Popcorn Analysis UPKs
Your Turn!
1. Analyze UPKs!
2. Pick the time and power
settings that maximize popcorn
taste while minimizing UPKs.
23
Choose factor levels to try to simultaneously satisfy
all requirements. Balance desired levels of each
response against overall performance.
Popcorn: Optimization of Multiple
Responses!
C: Power
4.00 4.50 5.00 5.50 6.00
Interaction
B: Ti me
T
a
s
t
e
30.0
40.0
50.0
60.0
70.0
80.0
90.0
C-
C+
C: Power
4.00 4.50 5.00 5.50 6.00
Interaction
B: Ti me
U
P
K
s
0.1
1.0
1.9
2.7
3.6
C-
C+
1. At the Numerical Optimization node set
the goal for Taste to maximize with a
lower limit of 60 and an upper limit of
90:
Popcorn Optimization
2+
2. Set the goal for UPKs to minimize with
a lower limit of 0 and an upper limit of
2:
Popcorn Optimization
3. Optimized Solutions :
# Brand* Time Power Taste UPKs Desirability
1 Costly 4.00 100.00 79.0 0.70 0.642 Selected
2 Cheap 4.00 100.00 79.0 0.70 0.642
3 Cheap 6.00 75.00 75.5 1.40 0.394
4 Costly 6.00 75.00 75.5 1.40 0.394
*Has no effect on optimization results.
Popcorn Optimization
25
4. A quick visual for evaluating desirability
with: B:Time vs. C:Power -
Popcorn Optimization
Design-Expert Software
Desirabi lity
Design Points
C- 75.000
C+ 100.000
X1 = B: Ti me
X2 = C: Power
Actual Factor
A: Brand = Costly
C: Power
4.00 4.50 5.00 5.50 6.00
Interaction
B: Ti me
D
e
s
i
r
a
b
i
l
i
t
y
0.000
0.250
0.500
0.750
1.000
Predi cti on 0.64
5. Contour Plot
Popcorn Optimization
Des i g n-E xpe rt S oft ware
Des i ra bi l i t y
De s i gn P oi nts
X 1 = B : T i me
X 2 = C : P o wer
A c t ua l Fac to r
A : B ran d = C he ap
4.0 0 4. 50 5. 00 5. 50 6 .00
7 5. 00
8 1. 25
8 7. 50
9 3. 75
1 00 .0 0
De sir a b ility Co n to u r
B : T i m e
C
:

P
o
w
e
r
0.100
0.100
0.200
0.200
0.300
0.300
0.400
0.500
P re d i ct i 0 . 6 4
26
6. A 3D Surface -
Popcorn Optimization
Des i g n-E xpe rt S oft ware
Des i ra bi l i t y
X 1 = B : T i me
X 2 = C : P o wer
A c t ua l Fac to r
A : B ran d = C he ap
4.0 0
4.50
5.00
5.50
6.00
75.0 0
81.25
87.50
93.75
100.00
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7


D
e
s
i
r
a
b
i
l
i
t
y


B : Ti m e
C: P ower
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
2
3
4
5
6
1
7
27
Predictive Capability Inside Design,
Extrapolation Capability Outside Design!
42 32
77 81
74 75
71 80
P
o
w
e
r
Brand
T
i
m
e
Is that all there is?
Benefits of DOE
quantify multiple variables simultaneously
identify variable interactions
independent variable analysis
identify high impact variables
improve process & product function
predictive capability within design space
extrapolation capability outside design space
improve employee morale improve employee morale !!!!!!!!!!!!!!!!!!!!
2
3
4
5
6
1
7
8
28
Road Map
Full
Resolution
Road Map
29
Not so Breaking News!
Full Factorial Arrays are not Efficient
Do we really need the power of a Full
factorial design?
A 2
5
factorial (32 runs) provides data to
evaluate:
5 ME, 10 2FI, 10 3FI, 5 4FI, 1 5FI
1
6
4
7
3
5
2
8
+
+

+
+

C
+
+

+
+
AB
+

+
AC
+

+
+

+
BC
8 + + + II
5 + II
3 + + II
2 + + II
7 + I
6 + I
4 + + I
1 I
Std ABC B A Block
Fractional
2
3
Factorial in Two Blocks
30
Resolution
The ability to effectively estimate a
coefficient.
Full: A, B, C, AB, AC, BC, ABC
III: ME + 2FI A + BC
IV: ME + 3FI A + BCD
2FI + 2FI AB + CD
V: ME + 4FI A + BCDE
2FI + 3FI AB + CDE
Feature:
Color-Coded Design Template
31
New!
Minimum-Run Resolution IV Designs*
The minimum number of runs for
resolution IV design is only two times the
number of factors (runs = 2k). This can offer
quite a savings when compared to a regular
resolution IV 2
k-p
fraction.
*(Invented by Whitcomb and Oehlert. Details posted to www.StatEase.com
- proceedings of talk at Annual Quality Congress, Toronto, 5/04)
Minimum-Run Resolution IV Designs
50 64 25 30 32 15
48 64 24 28 32 14
46 64 23 26 32 13
44 64 22 24 32 12
42 64 21 22 32 11
40 64 20 20 32 10
38 64 19 18 32 9
36 64 18 16* 16 8
34 64 17 14 16 7
32* 32 16 12 16 6
Min 2
k-p
k Min 2
k-p
k
50 64 25 30 32 15
48 64 24 28 32 14
46 64 23 26 32 13
44 64 22 24 32 12
42 64 21 22 32 11
40 64 20 20 32 10
38 64 19 18 32 9
36 64 18 16* 16 8
34 64 17 14 16 7
32* 32 16 12 16 6
Min 2
k-p
k Min 2
k-p
k
No savings for 8 or 16 factors.
32
Advantages of DOE vs. OFAT
- Efficiency
- Simplicity + Effectiveness
- Confidence (from stats)
- !nteraction Data
- Predictive Capability within Design Space
- Extrapolation Capability outside Design Space
- variation Data
- Cost !nfo
9
8
7
6
5
4
Fraction Irregular Fraction k
k p
V
2

3
replicate 12
4

=


4
replicate 16
4

=


3
replicate 24
4

=


2
replicate 16
4

=


3
replicate 48
4

=


2
replicate 32
4

=


3
replicate 48
8

=


4
replicate 64
8

=


3
replicate 48
16

=


4
replicate 64
16

=


3
replicate 96
16

=


4
replicate 128
16

=


Resolution V Irregular Fractions*
33
Minimum Run Resolution V
(MR5) Designs
Regular fractions (2
k-p
fractional factorials) of 2
k
designs often contain considerably more runs than
necessary to estimate the coefficients in the 2FI
model.
The smallest regular resolution V design for k=7
uses 64 runs (2
7-1
) to estimate 29 coefficients.
Balanced minimum run resolution V (MR5)
design for k=7 has 30 runs, a savings of 34 runs.
Disadvantage partial aliasing. MR5 designs
are irregular fractions.
MR5 Designs
Provide Considerable Savings
466 1024 30 106 256 14
326 1024 25 92 256 13
232 512 21 80 256 12
212 512 20 68 128 11
192 512 19 56 128 10
172 512 18 46 128 9
154 256 17 38 64 8
138 256 16 30 64 7
122 256 15 22 32 6
MR5 2
k-p
k MR5 2
k-p
k
3+
Resolution Summary
22 V MR-5
32 VI 2
6-1
fraction
24 IV Semifold
32 IV Foldover
16 IV 2
6-2
screening
Runs Resolution Design
At least consider a resolution V fraction before deciding to screen
for main effects using a resolution IV design.
Road Map
35
RSM: When to Apply It
Region of Operability
Region of Interest
Use factorial
design to get close
to the peak. Then
RSM to climb it.
RSM vs OFAT OFAT
-2 -1 0 1 2
30
45
60
75
90
Factor A
R
e
s
p
o
n
s
e
-2 -1 0 1 2
60
65
70
75
80
85
90
Factor B
Response
R
e
s
p
o
n
s
e
65
73
80
88
95
Response
-4
-2
0
2
-4
-2
0
2
4
Factor A
Factor B
R
e
s
p
o
n
s
e
36
Excuses
Claim of no interactions
OFAT is the standard
Stats are confusing
Experiments are too large
Data is too variable
Run the experiments, then evaluate the stats
Vary one factor at a time to reduce confusion
Many of the problems existing in industry
today are not caused by people within the
workforce, but result from the systems
within which people must work.
Author Darryl Landvater
Why Experimental Design?
37
Lack of Acceptance ?
Perception as Difficult
DOE Statistics
Perception of High Cost
Efficient
Focus on Short Term Solutions
Failure to solve problems permanently
Lack of Change Management
Failure to focus on Doing Things Differently

If you always do what you always did;
youll always get what you always got.
- wise but unknown philosopher
Old Habits Methods Die Hard