Design of Experiment Using Minitab Book - 1

Section 1
INTRODUCTION
DuPont Quality Management and Technology SOE/MTB - 1.1

© 2000 E. I. du Pont de Nemours and Company Revised 3/10/2000
WHAT IS DESIGN OF EXPERIMENTS (DOE)?
DOE is a strategic process, with supporting

methods and tools, for guiding the
- planning
- execution
- analysis of results
- application of results
of experimental or developmental programs.

BENEFITS OF DOE
 Highest leverage quality tool available --- use to

design quality in up front
 Reduces product/process development cycle time
 Most efficient strategy for gaining process
understanding
 Develops true cause-and-effect relationships
 Provides solution to current problem plus
information for solving future problems
 An objective, fact-based system for decision
making, complete with quantitative measures of
uncertainty

WHERE CAN DOE BE APPLIED?
 DOE is useful in every stage of product life

cycle
– Product and Process Development
– Process Scale-Up, Operations Startup, Customer
Verification
– Process Control, Product and Process
Improvement

APPLICATIONS IN PRODUCT AND
PROCESS DEVELOPMENT
 Define and translate customer needs

 Design robust products
 Design robust processes
 Reduce time to commercialization
 Develop test methods
 Define operating procedures
 Enhance business integration of R&D with
Operations

APPLICATIONS IN SCALE-UP, STARTUP
AND CUSTOMER VERIFICATION
 Reduce time to process qualification

 Identify key process variables
 Determine product specifications
 Design product field tests
 Develop standard operating conditions and
standard operating procedures

APPLICATIONS IN PROCESS CONTROL
 Develop process models

 Calibrate process control knobs
 Adjust to changing customer needs
 Troubleshoot process problems
 Develop sampling protocols

APPLICATIONS IN PRODUCT AND
PROCESS IMPROVEMENT
 Reduce product variability

 Improve first-pass first-quality yield
 Increase capacity
 Reduce transition time
 Make process more robust
 Improve test methods

WHAT YOU SHOULD GET OUT OF SOE
 An appreciation of the basic underlying

concepts of DOE
 A general strategy for approaching
experimentation
 A set of efficient, widely-applicable tools for
designing and analyzing experiments
 Hands-on experience at using these tools
 Some sense of when and where to seek
expert help

Section 2
WORKSHOP 1
DuPont Quality Management and Technology SOE/ECHIP - 2.1

WORKSHOP 1 PROBLEM
A new polymer is being readied for a plant process. One
major problem remains: the COLOR (yellowness) has
often been unacceptable in experimental production to
date. The COLOR value should be made as low as
possible. Prior work has indicated that COLOR may be
affected by the following variables.
EXP. RANGE
ABBREV. VARIABLE NAME LOW HIGH UNITS
C Catalyst Concentration 1.0 1.8 %
T Reactor Temperature 130 190 deg C
A Additive Amount 1 5 kg
Also from prior work, it is known that MODULUS can be
predicted by the following equation over the
experimental range of interest:
MODULUS = -69.5 + 100*C + 0.15*T - 5.0*A

WORKSHOP 1 PROBLEM
Your problem is:

1. To demonstrate an approximate set of conditions
to obtain low COLOR together with low
MODULUS, and
2. To support your conclusion with a description of
the effects of the factors on the response,
COLOR.
Your boss’s best guess of a good place to start is:

CATALYST = 1.25 %
TEMPERATURE = 137 deg C
ADDITIVE = 3 kg
WORKSHOP 1 TEAM REPORTS
Rec. Settings Results Effects on Color
# of
Team Cata Temp Addi Color Mod. Cata Temp Addi Runs Method/Comments
1
2
3
4
5
6
7
8
9
10
WORKSHOP 1 TEAM REPORTS
Rec. Settings Results Effects on Color # of
Team Cata Temp Addi Color Mod. Cata Temp Addi Runs Method/Comments
11
12
13
14
15
16
17
18
19
20
WORKSHOP 1:
Concepts Introduced
 Experimental Variability Is a Fact of Life

 Properties Can Be Represented As
Functions of Control Variables
 Geometry of Experimental Region
 Contour Plots
 Interaction
 Multiple Responses

Section 3
FOUNDATIONS OF
THE STRATEGY

UNDERLYING PRINCIPLES
 World is multivariate
 Experimental error is a fact of life
 Experimentation is a process
 Multi-stage approach
 Statistical strategy
 The 6 B’s of DOE

WORLD IS MULTIVARIATE
 Almost always more than one factor of interest

that can be varied
e.g. pressure, temperature, pH, flow rate, hold time, screw
speed
 Interactions often present --- factor effects not
additive i.e. synergistic or antagonistic effects
 Usually, several responses (outcomes) of interest
e.g. viscosity, yield, assay, color, hardness, modulus,
dyeability
 Tradeoffs between responses often necessary

VARIABLES
LURKING
FACTORS RESPONSES
VARIABLES
variables that are outcome variables variables that are

deliberately controlled that are measured unidentified or
in the experiment during the experiment uncontrolled
synonyms
INDEPENDENT VARIABLES DEPENDENT VARIABLES NOISE
PREDICTORS PROPERTIES UNCONTROLLED VARIABLES
KNOBS CHARACTERISTICS
PROCESS VARIABLES OUTCOMES
TREATMENTS
DOSES

EXPERIMENTAL ERROR
 Experimental error is the “noise” in the

system --- the catchall term used to explain
why results are not identical from replicate
to replicate
 2 types of experimental error:
– systematic or bias error
– random error

RANDOM ERROR
0 0
• Unpatterned variability • Normal error

distribution
• Unpredictable
• Standard deviation
• Multiple unassignable represents “typical”
causes error

SYSTEMATIC OR BIAS ERROR
•Patterned variation
•May be predictable
•Due to single assignable cause
–e.g. shift, raw material lot, day, tool wear, ambient temperature

DEALING WITH EXPERIMENTAL ERROR
EXPERIMENTAL ERROR
RANDOM ERROR SYSTEMATIC ERROR
CAUSE Unassignable Assignable
NATURE Unpatterned Patterned
REMEDY Replication Blocking

Randomization

REPLICATION
true relationship
true relationship
RESPONSE
RESPONSE
random
average
error
random
error
fitted average
fitted relationship
relationship
FACTOR FACTOR
•Through averaging of replicates the impact

of random error is reduced
•Two forms of replication
–Hidden replication --- feature of all good designs
–Pure replication (as above)

BLOCKING
 Used in presence of identifiable source of

potential bias (“blocking factor”)
– e.g. day, raw material lot
 Split the experiment up into blocks
representing different levels of the blocking
factor
 Keep the blocks balanced with respect to
the experimental factors. This prevents
confounding, or confusing, any block effect
with that of an experimental factor.

RANDOMIZATION
 “Insurance” protection against potential

unidentified sources of bias
 Randomize the order of experimental runs
 If experiment is blocked, randomize within
blocks
 May require constrained randomization if
some experimental factors hard to change

EXPERIMENTATION IS A PROCESS
Gather Information
Apply Results
Define Objectives
no Update Information Design Experiment
Go to
next stage of yes
experimentation? Run Experiment
Analyze Experiment
Perform Confirmation Runs
Interpret Results

MULTI-STAGE APPROACH
SCREENING INTERACTION RESPONSE

DESIGNS DESIGNS SURFACE
DESIGNS
Evolution of the Experimental Environment
NUMBER OF 6 or more 3-8 2-6

FACTORS
OBJECTIVE Identify key factors Understand factor Prediction model
interactions Optimization
COMMON Plackett-Burman Full Factorial Box-Behnken

DESIGNS Fractional Factorial Fractional Factorial Central Composite
(resolution 3 or 4) (resolution 5) Face Center Cube

STATISTICAL STRATEGY
VS. ONE-FACTOR-AT-A-TIME
One-Factor-At-A-Time Statistical
Levels Levels
of X2 of X2
Y Y
X1 X1

COMPARING THE STRATEGIES
1-FACTOR-AT-A-TIME STATISTICAL
DESIGN Vary only 1 factor at time, Vary all factors jointly in

in multiple small increments, balanced bite-sized space-
keeping all others fixed filling design
FITTED MODEL Curves fitted through data Simple empirical models

points, separately for each based on low-order
factor polynomials
EXPERIMENTAL Ignored Recognized and estimated

ERROR
INTERACTIONS Not considered; not estimable Dealt with as appropriate

6 B’S OF DOE
 Bite Size
– Just enough runs to meet objectives, achieve desired sensitivity, and
estimate experimental error
 Boldness
– Vary experimental factors over wide range
– Measure all relevant responses
 Balance
– Use balanced designs to maximize efficiency and minimize confounding
 Bias Error
– Take countermeasures such as randomization and blocking
 Blunders
– Avoid through careful planning and execution
 Batting Average
– Improve your odds of success through statistical designs and empirical
modeling

Section 4
FACTORIAL GEOMETRY

EVOLUTION OF THE ENVIRONMENT:
Intermediate Stage

DESIGNS

FACTORS


TWO-LEVEL FACTORIAL DESIGNS
 2k Distinct Runs
 Easy to Plan and Analyze
 Usable for Either Continuous or Discrete
Factors with Two Levels
 Uniformly Spread Through Factor Space
 Permit Estimation of Both Main Factor
Effects and Interaction Effects

23 FACTORIAL DESIGN
(LO, HI, HI) (HI, HI, HI)
(LO, LO , HI) (HI, LO, HI)

X3
(HI, HI, LO)

(LO, HI, LO)
X2
(X1, X2, X3) =
(LO, LO, LO) (HI, LO, LO)
X1

3 FACTOR 2 LEVEL FULL-FACTORIAL
Experiment Data
CATALYST TEMPERATURE ADDITIVE COLOR MODULUS

1.8 190 5 72 114
1.0 190 1 48 54
1.8 130 5 37 105
1.0 190 5 48 34
1.8 130 1 36 125
1.8 190 1 74 134
1.0 130 5 62 25
1.0 130 1 61 45

23 FACTORIAL DESIGN
with Workshop 1 COLOR Data
48 72
62 37
X3 (Additive)
48 7
4
X2 (Temperature)
61 36
X1 (Catalyst)

WHAT IS AN EFFECT ?
An effect is the difference in the averages

of two groups of observations.
Let YLOW = average of values on LOW plane.

Let YHIGH = average of values on HIGH plane.
EFFECT = YHIGH - YLOW

SAMPLE CALCULATION: EFFECT OF X1
24
48 72
-25
62 37
X3 (Additive)
26
48 74
-25 X2 (Temperature)
61 36
X1 (Catalyst)
72 + 37 + 74 + 36 48 + 62 + 48 + 61 = 24 + (-25) + 26 + (-25) = 0
4 4 4

35
48 72
-14
62 37
X3 (Additive)
-13 48 74
38
X2 (Temperature)
61 36
X1 (Catalyst)
48 + 72 + 48 + 74 62 + 37 + 61 + 36 = -14 + 35 + (-13) + 38 = 11.5

4 4 4

48 72
62 37
X3 (Additive)
0 -2
1 1
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
48 + 72 + 62 + 37 48 + 74 + 61 + 36 = 0 + (-2) + 1 + 1
= 0
4 4 4

ONE FACTOR AT A TIME
X3
No Hidden Replication
Not Space Filling
X2
X1

INTERACTION OF X1 AND X2
INTERACTION
High X2 High X2
Y Y
Low X2
Low X2
X1 X1
NO INTERACTION
Low X2
Y High X2 Y High X2
Low X2
X1 X1

INTERACTION GEOMETRY
X1 Effect at High and Low X2
YC
YD X1*X2 Interaction Effect

YB
= [(YD - YC) - (YB - YA)] / 2
YA
(YA + YD) (YB + YC)
=
X2
2 2
X1

X2 Effect at High and Low X1
YC
YD X1*X2 Interaction Effect
YB = [(YD - YB) - (YC - YA)] / 2

YA
(YA + YD) (YB + YC)
=
X2 2 2
X1

SAMPLE CALCULATION:
X1*X2 INTERACTION EFFECT
48 72
62 37
X3 (Additive)
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
72 + 74 + 62 + 61 48 + 37 + 48 + 36 = 25
4 4

SAMPLE CALCULATION:
48 72
62 37
X3 (Additive)
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
37 + 72 + 48 + 61 48 + 62 + 74 + 36 = -0.5
4 4

SAMPLE CALCULATION:
48 72
62 37
X3 (Additive)
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
48 + 72 + 61 + 36 62 + 37 + 48 + 74 = -1
4 4

SAMPLE CALCULATION:
X1*X2*X3 INTERACTION EFFECT
48 72
62 37
X3 (Additive)
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
72 + 62 + 48 + 36 48 + 37 + 74 + 61 = -0.5
4 4

HALF OF THREE-FACTOR INTERACTION
SHOWS BALANCE IN ALL FACTOR PAIRS
X3
X1
X3
X2
X2
X1

USES OF HIGHER-ORDER
 Blocking
– Basis for Splitting Experiment into
Smaller Blocks
– Factors Are Balanced Within Blocks
 Screening
– Cut Experiment in Half by Using Only
One of Blocks
– Factors Are Balanced

FACTORIAL DESIGNS: Summary
 The “Cube” Approach

 Each Dimension Is a Factor
 Coding: Low = “-” and High = “+”
 Effects Are Comparisons of “Planes”
 Hidden Replication
 Efficiency: All Data Used to Calculate
Each Effect
 High-Order Interactions
Section 5
FACTORIAL EXAMPLE:
Design
Du Pont Quality Management and Technology SOE/MTB - 5.1

FACTORIAL DESIGN EXAMPLE
Design Phase
Follow the example to design a 2-level full-factorial experiment for
the workshop 1 problem. Link to Workshop 1 Problem
Factors Range
CATALYST CONCENTRATION 1.0 to 1.8
REACTOR TEMPERATURE 130 to 190
AMOUNT OF ADDITIVE 1 to 5
Responses
COLOR
MODULUS
Design = Full-Factorial with center points

Model = Linear + 2 factor interaction terms

THREE-FACTOR CUBE PLOT
X3
X2
X1

TEST FOR CURVATURE
Y
Difference Is ...
Curvature ...
...
...
..
- 0 +
X
Section 6
ANALYSIS OF TWO-LEVEL
FACTORIAL DESIGNS

SECTION 6 OVERVIEW
 How do we know the effects are real ?
 Computer analysis of the effects
 How well have we explained the behavior

of Y?

HOW DO WE KNOW THE EFFECTS ARE “REAL” ?
If each corner is an average of replicated runs, we can
study the repeatability of the effects.
48 72
Rep. 1
62 37
Rep. 2 X3 (Additive)
48 74
X2 (Temperature)
61 36
X1 (Catalyst)
EXAMINE THE REPLICATES
FOR REPEATABILITY (Case 1)
46 73
48 50 72 71
61 36
62 63 37 38
X3 (Additive)
49 75
48 47 74 73
62 X2 (Temperature)
61 60 36 38
34
X1 (Catalyst)
66 52
48 30 72 92
19 57
62 105 37 17
X3 (Additive)
33 44
48 63 74 104
24 X2 (Temperature)
61 98 36 54
18
X1 (Catalyst)
THE IMPACT OF EXPERIMENTAL ERROR
Response Experimental Error

Average
Relationship
Effect
Low (-) High (+)

Factor

HOW “REAL” ARE THE EFFECTS ?
Must estimate the size of the experimental error.
STANDARD DEVIATION: A measure of variability
(Y1- Y)2 + (Y2 - Y)2 + . . . + (Yn - Y)2

Std. Dev. = s =
n-1

s=2.8 46 73
48 50
72 71
s=1.4
61 s=1.4 36
s=1.4 62 37 38
63
X3 (Additive)
s=1.4
49 75
48 47 74 73
s=1.4
62 X2 (Temperature)
s=1.4 61 60
36 38
34
s=2.8
X1 (Catalyst)
POOLED STANDARD DEVIATION
 Based on a weighted average of individual

squared standard deviations
 Assumes homogeneous error --- size of
experimental error is uniform throughout
design region
 A more reliable estimate of the overall
standard deviation
(n1 - 1) s12 + (n2 - 1) s22 + . . . + (nk - 1) sk2

spooled =
(n1 - 1) + (n2 - 1) + . . . + (nk - 1)

COMPUTE THE POOLED STANDARD DEVIATION
FOR THE 23 EXAMPLE: (Case 1)
1*2 + 1*2 + 1*8 + 1*2 + 1*8 + 1*2 + 1*2 + 1*2

spooled =
1 + 1 + 1 + 1 + 1 + 1 + 1 + 1
28
=
8
= 3.5 = 1.87

INTERPRETING THE STANDARD DEVIATION
-4 -3 -2 -1 0 1 2 3 4 SD units
68%
95%
99.7%

DISTRIBUTION OF AVERAGES
The distribution of the means of samples of

size n is:
–More nearly normal
–Narrower σY = σY n

THE STANDARD ERROR OF A
FACTOR EFFECT
 Represents the PRECISION in an estimated

factor effect
 Can be used to “test” if a factor effect is
“statistically significant”
STD. ERROR = S-Pooled * SQRT(1/n1 + 1/n2)

where
n1 = number of observations forming YLOW
n2 = number of observations forming YHIGH

EXAMPLE CALCULATION OF STANDARD ERROR
 Consider the C*T interaction effect

 The estimated effect = 25
 Standard error = 1.87 * SQRT(1/8 + 1/8) = 0.9
(1.87 is the pooled standard deviation calculated
earlier with 8 degrees of freedom)
 Tabled t-value for 8 df, 95% confidence
level
= 2.31 (t-table on next page)

t-DISTRIBUTION VALUES FOR
TWO-SIDED CONFIDENCE INTERVALS
DF 99% 95% 90%
1 63.70 12.70 6.31

2 9.92 4.30 2.92
3 5.84 3.18 2.35
4 4.60 2.78 2.13
5 4.03 2.57 2.01
6 3.71 2.45 1.94

7 3.50 2.36 1.89
8 3.36 2.31 1.86
9 3.25 2.26 1.83
10 3.17 2.23 1.81
11 3.11 2.20 1.80

12 3.05 2.18 1.78
13 3.01 2.16 1.77
14 2.98 2.14 1.76
15 2.95 2.13 1.75
16 2.92 2.12 1.75

17 2.90 2.11 1.74
18 2.88 2.10 1.73
19 2.86 2.09 1.73
20 2.85 2.09 1.72
21 2.83 2.08 1.72
22 2.82 2.07 1.72
23 2.81 2.07 1.71
24 2.80 2.06 1.71
25 2.79 2.06 1.71
30 2.75 2.04 1.70
40 2.70 2.02 1.68
60 2.66 2.00 1.67
120 2.62 1.98 1.66
∞ 2.58 1.96 1.64

SIGNIFICANCE OF AN EFFECT
BY SIGNIFICANCE TEST

 Observed t-ratio = (estimated effect)/(std error)
= 25.0 / 0.9 = 28
 Compare observed t-ratio to tabled t-value:
Here |t-ratio| > tabled t-value (28 > 2.31)
So the C*T interaction is statistically significant
at the 0.95 (or 95%) confidence level
 Note that the C*T interaction is also significant at the
0.99 confidence level (28 > 3.36).
 How high can we take the confidence level and still be
significant? This is related to the P-value.

P-VALUE
 The P-value of an effect is the chance of having

observed an effect that large purely due to random
experimental error
 The larger the effect, the smaller the P-value
 P-value is commonly displayed by most statistical
packages, usually as a decimal (i.e. between 0 and 1)
 P-value = 1 - (maximum confidence level at which
the effect is significant)
 EXAMPLE: P-value of an effect = 0.02
So the effect is significant at 98% confidence level
 For the C*T interaction effect, the P-value is 0.0000 !

SIGNIFICANCE OF AN EFFECT
BY CONFIDENCE INTERVAL

 Confidence interval is:
Estimated effect +/- (t-value * standard error)
 For this example 95% confidence interval
is
25 +/- (2.31 * 0.9) = 25 +/- 2.1 or 22.9 to 27.1
 Whenever ZERO is not included in the
confidence interval, the effect is significant
at the specified confidence level
 Graphical display of uncertainty
SECTION 6 OVERVIEW
 How do we know the effects are “real” ?

– Estimate the experimental error (s-pooled)
and the standard error of the effect
– Compare the estimated effect with its standard
error
– t-test or
– confidence interval
 Computer analysis of the effects

 How well have we explained the
behavior of Y ?
COMPUTER ANALYSIS OF THE
ESTIMATED EFFECTS
Two common approaches:

 EFFECTS ANALYSIS
– Displays each effect and t-test or
confidence interval
– Follows approach just covered earlier
 REGRESSION ANALYSIS
– Expresses each effect as a slope of a
line or coefficient of a model term
– Statistically equivalent to an effects
analysis

STANDARD FACTOR CODING
Regression coefficient (slope) = 1/2 effect
Response Average
Relationship
Slope = ∆ / 2
Effect
Low (-1) 0 High (+1)

Factor

MEANING OF COEFFICIENTS:
Temperature Effect
75
Slope of this line =
Regression coefficient = 5.75
(using centered/scaled
65 factor settings)
C OLOR
For this example:

Effect = 11.5 = expected
5
55 increase in COLOR from
Temperature=130 to 190
(-1 to +1)
Coefficient = 5.75 = expected
45 increase in COLOR from
Temperature=160 (middle)
to 190 (0 to +1). Uses
orthogonally scaled
35 (-1 to +1) factor settings.
130 140 150 160 170 180 190

(-1) (0) (+1)
Temperature

3 FACTOR EXAMPLE (Case 1)
Minitab Effects Table for COLOR Response
Fractional Factorial Fit: COLOR versus Catalyst, Temperature,

Additive
Estimated Effects and Coefficients for COLOR (coded units)
Term Effect Coef SE Coef T P

Constant 54.7500 0.4488 122.00 0.000
Catalyst 0.0000 0.0000 0.4488 0.00 1.000
Temperat 11.5000 5.7500 0.4488 12.81 0.000
Additive 0.0000 0.0000 0.4488 0.00 1.000
Catalyst*Temperat 25.0000 12.5000 0.4488 27.85 0.000
Catalyst*Additive -0.5000 -0.2500 0.4488 -0.56 0.591
Temperat*Additive -1.0000 -0.5000 0.4488 -1.11 0.294

ITEMS IN THE MINITAB
EFFECTS & COEFFICIENTS TABLE
 Term: Name for model term.

 Effect: Expected change in the response over the entire range of the
term (high “plane” - low “plane” averages).
 Coef: Regression coefficients for the model terms. Expected change

in the response per unit change in the term. Note carefully the centering
and scaling used for these coefficients for interpretation.
 SE Coef: Standard error of the coefficient -- uncertainty around the

coefficients due to experimental error.
 T: The ratio of the coefficients divided by their standard errors.

 P: Probability of observing a coefficient of that magnitude when the true
coefficient is zero. Low values (<.05) imply significance.
– If significant, we are pretty sure that at least the sign (direction) of the
relationship is correct

3 Factor Example (Case 1)
Graphical Representation of Factor Effects
Main Effects Plot (data means) for COLOR
0 0
1.0 1. 8 13 19 1 5
60.0
57.5
COLOR
55.0
52.5
50.0
Catalyst Temperature Additive

3 Factor Example (Case 1)
Graphical Representation of the Interactions
Interaction Plot (data means) for COLOR
1 1.8 130 190 1 5
70
Catalyst
1.8 55
1
40
70
Temperature
190 55
130
40
Additive 70
5 55
1
40

Minitab Cube Plot for COLOR Response
Cube Plot (data means) for COLOR
48 72
48 74
190
Temperature 62 37
5
Additive
61 36
130 1
1.0 1.8
Catalyst

Pareto Plot of Effects on COLOR

Normal Probability Plot for Effects on COLOR
Normal Probability Plot of the Standardized Effects

(response is COLOR, Alpha = .10)
A: Catalyst
AB B: Temperat
C: Additive
1
B
Normal Score
-1
0 10 20
Standardized Effect

SECTION 6 OVERVIEW
 Computer analysis of the effects

– Effects analysis
– Regression analysis
 How well have we explained the behavior

of Y ?

TWO ESTIMATES OF EXPERIMENTAL ERROR
 Pure Error Estimate (PE)

– Based on variability among replicate runs under
fixed X-settings
– Same as the pooled standard deviation
 Lack of Fit Estimate (LOF)
– Based on how well the estimated effects explain
the variation in the observed data.
When both the PE and LOF estimates are available,
they can be used to test if additional effects should
be accounted for.

WHEN ARE P.E. AND L.O.F.
ESTIMATES AVAILABLE ?
Lack of Fit
Not Available Available
df: 2, 0, 0 df: 2, 0, 1
Not
Available Key:
Pure Error
df: #, #, #
df: 2, 2, 0 df: 2, 3, 1 Fit Lack of

Fit
Pure
Error
Available

THE ANALYSIS OF VARIANCE
 Statistical technique for evaluating

PURE ERROR and LACK-OF-FIT
experimental error
 Characterizes how well the estimated

effects explain the data

ONE FACTOR EXAMPLE
Experiment Worksheet
Row FACTOR RESPONSE

1 -1 25
2 -1 35
3 0 60
4 0 80
5 1 45
6 1 55

ONE FACTOR EXAMPLE
Data Graph with Fitted Line
80
70
Response Y
Y = 50 + 10X
60
50
40
30
-1 0 1
Factor X

ONE FACTOR EXAMPLE
Coefficients and Analysis of Variance Results
The regression equation is

RESPONSE = 50.0 + 10.0 FACTOR
Predictor Coef SE Coef T P

Constant 50.000 7.906 6.32 0.003
FACTOR 10.000 9.682 1.03 0.360
S = 19.36 . . .
Analysis of Variance
Source DF SS MS F P
Regression 1 400.0 400.0 1.07 0.360
Residual Error 4 1500.0 375.0
Lack of Fit 1 1200.0 1200.0 12.00 0.041
Pure Error 3 300.0 100.0
Total 5 1900.0

FURTHER EXPLANATION ON SELECTED
COEFFICIENT TABLE ITEMS
 Regression F-test:
Tests whether any of the
model terms explain the behavior of Y (overall test). If the
model has some significant terms, the p-value should be
small (<.05).
 Lack of Fit F-test:

Tests whether model can be
improved with existing data. Might be significant due to
missing terms, outliers, a need for a transformation, lack of
measurement precision, etc. An appropriate model will have
a non-significant lack of fit F-test (high p-value).
 Square Root of MS Pure Error:

Pooled
standard deviation from replicates. Estimates SD of
experimental error at any fixed set of conditions.
REGRESSION AND LACK OF FIT SIGNIFICANCE
Example 1
14
Regression p-value = .0000
12 Lack of fit p-value = .5514
10
RESP.Y1
15 20 25
FACTOR.X

Example 2
5.5
Lack of fit p-value = .9641
RESP.Y2
5.0
4.5
15 20 25
FACTOR.X

Example 3
9.5
8.5 Lack of fit p-value = .0000
7.5
RESP.Y3
6.5
5.5
4.5
3.5
2.5
15 20 25
FACTOR.X

Example 4
12.6
11.6 Regression p-value = .0001
Lack of fit p-value = .0003
10.6
9.6
RESP.Y4
8.6
7.6
6.6
5.6
4.6
3.6
15 20 25
FACTOR.X

BACK TO OUR 3 FACTOR EXAMPLE . . .
Case 1
46 73
48 50
72 71
61 36
62 63 37 38
X3 (Additive)
49 75
48 47 74 73
62 X2 (Temperature)
61 60 36 38
34
X1 (Catalyst)
Coefficients Table for COLOR Response
Fractional Factorial Fit: COLOR versus Catalyst, Temperature,

Additive

Constant 54.7500 0.4488 122.00 0.000
Catalyst 0.0000 0.0000 0.4488 0.00 1.000
Temperat 11.5000 5.7500 0.4488 12.81 0.000
Additive 0.0000 0.0000 0.4488 0.00 1.000
Catalyst*Temperat 25.0000 12.5000 0.4488 27.85 0.000
Catalyst*Additive -0.5000 -0.2500 0.4488 -0.56 0.591
Temperat*Additive -1.0000 -0.5000 0.4488 -1.11 0.294

Analysis of Variance for COLOR Response
Analysis of Variance for COLOR (coded units)
Source DF Seq SS Adj SS Adj MS F P

Main Effects 3 529.00 529.00 176.333 54.72 0.000
2-Way Interactions 3 2505.00 2505.00 835.000 259.14 0.000
Residual Error 9 29.00 29.00 3.222
Lack of Fit 1 1.00 1.00 1.000 0.29 0.608
Pure Error 8 28.00 28.00 3.500
Total 15 3063.00

SECTION 6 OVERVIEW
 Computer analysis of the effects
 How well have we explained the

behavior of Y ?
– Analysis of variance results
–Regression F-test
–Lack of fit test

Section 7
FACTORIAL EXAMPLE:
Analysis

FACTORIAL DESIGN EXAMPLE
Analysis Phase
Recall the workshop 1 problem (summarized below) for which we generated

a design earlier in ECHIP. We already have response data entered so we’re
now ready to analyze the data.
Factors Range Responses

CATALYST CONCENTRATION 1.0 to 1.8 COLOR
REACTOR TEMPERATURE 130 to 190 MODULUS
AMOUNT OF ADDITIVE 1 to 5
Design = Full-Factorial with center points

Model = Linear + two-factor interaction terms

THREE-FACTOR CUBE PLOT OF
EXAMPLE WORKSHOP 1 DATA
34 50 72 114
51 71
25 58 36 105
57 38
Additive
38 41
80
42 43
48 73 134
54
49 74
60 34 Temperature
45
59 35 125
Catalyst
KEY: Numbers inside circles = COLOR values
Numbers outside circles = MODULUS values

Workshop 1 Revisited
COLOR -- Coefficients (circle if significant) Signif. Replicate
of LOF Standard Comments
Team Cata Temp Addi C*T C*A T*A test Deviation
1
2
3
4
5
6
7
8
9
10
Fractional Factorial Fit: COLOR versus CATALYST, TEMPERATURE, ADDITIVE

Constant 51.450 1.535 33.52 0.000
CATALYST 0.125 0.063 1.716 0.04 0.972
TEMPERAT 13.875 6.938 1.716 4.04 0.002
ADDITIVE 0.125 0.062 1.716 0.04 0.972
CATALYST*TEMPERAT 22.875 11.438 1.716 6.66 0.000
CATALYST*ADDITIVE 0.125 0.063 1.716 0.04 0.972
TEMPERAT*ADDITIVE -0.125 -0.063 1.716 -0.04 0.972
CATALYST*TEMPERAT*ADDITIVE -2.125 -1.063 1.716 -0.62 0.547
Analysis of Variance for COLOR (coded units)

Main Effects 3 770.19 770.19 256.729 5.45 0.013
Residual Error 12 565.51 565.51 47.126
Curvature 1 546.01 546.01 546.012 308.01 0.000
Pure Error 11 19.50 19.50 1.773
Total 19 3446.95
Unusual Observations for COLOR
Obs COLOR Fit SE Fit Residual St Resid

4 38.0000 51.4500 1.5350 -13.4500 -2.01R
R denotes an observation with a large standardized residual . . .
Fractional Factorial Fit: MODULUS versus CATALYST, TEMPERATURE, ADDITIVE
Estimated Effects and Coefficients for MODULUS (coded units)

Constant 79.60 0.05774 1378.71 0.000
CATALYST 80.00 40.00 0.06455 619.68 0.000
TEMPERAT 9.00 4.50 0.06455 69.71 0.000
ADDITIVE -20.00 -10.00 0.06455 -154.92 0.000
CATALYST*TEMPERAT -0.00 -0.00 0.06455 -0.00 1.000
CATALYST*ADDITIVE 0.00 0.00 0.06455 0.00 1.000
TEMPERAT*ADDITIVE 0.00 0.00 0.06455 0.00 1.000
CATALYST*TEMPERAT*ADDITIVE 0.00 0.00 0.06455 0.00 1.000
Analysis of Variance for MODULUS (coded units)

Main Effects 3 27524.0 27524.0 9174.67 1E+05 0.000
2-Way Interactions 3 0.0 0.0 0.00 * *
3-Way Interactions 1 0.0 0.0 0.00 * *
Residual Error 12 0.8 0.8 0.07
Curvature 1 0.8 0.8 0.80
Pure Error 11 0.0 0.0 0.00
Total 19 27524.8
. . .

Interaction Plot (data means) for COLOR
CATALYST
1.0
70 1.8
60
Centerpoint
Mean
50
40
130 190
TEMPERATURE

Main Effects Plot (data means) for MODULUS
Centerpoint
0 0
1 .0 1.8 13 19 1 5
120
100
MODULUS
80
60
40
CATALYST TEMPERATURE ADDITIVE

Main Effects Plot (data means) for COLOR
Centerpoint
0 0
1 .0 1.8 13 19 1 5
60
55
COLOR
50
45
40 CATALYST TEMPERATURE ADDITIVE

Section 8
GOOD EXPERIMENTAL
PRACTICE

GOOD EXPERIMENTAL PRACTICE
1. Assess the environment

2. Consider the factors
3. Consider the responses
4. Choose an appropriate design
5. Consider strategies for bias error
6. Create the experimental plan
7. Review the plan for operability
8. Avoid blunders
9. Plan for the analysis
10. Report the recommendations

1. ASSESS THE ENVIRONMENT
 Gather basic information

 Determine current state of understanding
 Define experimental objectives
 Define physical environment and constraints
 Consider experimental error

2. CONSIDER THE FACTORS
 How many and which factors to vary?

 Which factors to keep fixed, and at what
level?
 Are factors continuous or discrete?
 How many levels of each factor?
 How bold in choice of levels?

3. CONSIDER THE RESPONSES
 Consider all responses of potential interest

 Are responses continuous or discrete?
 What is effect size of interest?
 Is measurement error absolute or relative?
 Anticipate measurement issues

4. CHOOSE AN APPROPRIATE DESIGN
 Identify underlying model

 Choose appropriate design class
 Consider desired sensitivity (resolution) in
choosing size of design
 Consider extra runs
 Practical constraints?

NUMBER OF RUNS VS. SENSITIVITY
SOE Rule of Thumb for balanced 2-level factorial designs:
2
7 or 8
n = ( ∆/σ )
∆ = smallest size effect worth detecting
σ = standard deviation of experimental error
∆ / σ is “signal-to-noise ratio”
∆/σ 0.5 1.0 1.5 2.0
n 196-256 49-64 22-28 12-16

5. STRATEGIES FOR BIAS ERROR
 Consider blocking to protect against bias

due to identifiable variables
– e.g. time, material batch, operator
 Use randomization to protect against bias
due to unidentified variables
 Consider possible constraints on
randomization, do restricted randomization
if necessary
 Assess randomization, adjust if necessary

6. CREATE THE EXPERIMENTAL PLAN
 Create experimental worksheet, in

randomized run order
 Express factor levels in actual physical
units
 Include blank columns for responses
 Write detailed protocol with explicit
instructions for every step

7. REVIEW PLAN FOR OPERABILITY
 Include all relevant parties in review

– Planner
– Executor
– Lab analyst
– Local experts
– Statistician
 Examine all runs for operability, adjust if
necessary
 Consider constraints on experiment,
feasibility of schedule
8. AVOID BLUNDERS
 Anticipate problems in advance

 Emphasize importance of adhering to
experimental protocol
 Avoid shortcuts
 Understand importance of each run
 Avoid stopping short
 Record any deviations from plan

9. PLAN FOR THE ANALYSIS
 Consider analysis issues in advance

– Models
– Software
– Method of analysis
– Organization of data
 Plot the data

10. REPORT RECOMMENDATIONS
 Relate recommendations to the objective

 Recognize value of negligible effects
 Avoid extrapolation
 Report raw data for credibility
 Provide direction for future action

Section 9
SCREENING DESIGNS

Early Stage

DESIGNS

FACTORS


CHARACTERISTICS OF SCREENING DESIGNS
 Number of runs n only a few more than

number of factors k
 Factors considered at two levels each
k
 Use a fraction of the full 2 factorial design
 Designs balanced (orthogonal)
 Main factor effects clear of each other
 Interactions generally not estimable
 Two-way interactions may be fully or
partially confounded with main effects

CLASSES OF SCREENING DESIGNS
 Plackett-Burman (P-B) designs
 Fractional Factorial (FF) designs

PLACKETT-BURMAN DESIGNS
 Available in sizes n which are multiples of 4

 Can handle up to n-1 factors, although
recommended maximum is n-5
 Tables available for commonly used sizes of
n=12, 20, 24, 28
 For n equal to power of 2 (e.g. 8, 16, 32,...),
same as Fractional Factorial designs
 Two-way interactions partially confounded
with each main effect (for n not a power of 2)

12 RUN PLACKETT-BURMAN DESIGN
Trial X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11
1 + + - + + + - - - + -
2 + - + + + - - - + - +
3 - + + + - - - + - + +
4 + + + - - - + - + + - each row
5 + + - - - + - + + - + a cyclic
permutation
6 + - - - + - + + - + + of previous
7 - - - + - + + - + + + row
8 - - + - + + - + + + -
9 - + - + + - + + + - -
10 + - + + - + + + - - -
11 - + + - + + + - - - +
12 - - - - - - - - - - -
last row all minuses

BALANCE OF 12 RUN P-B DESIGN
Illustrate using factors X1 and X2
Rows rearranged according to level of X1
Trial X1 X2
1 + +
2 + -
4 + + equal number
5 + + of + and -
6 + -
10 + - SAME BALANCE TRUE
FOR ANY PAIR OF FACTORS
3 - + (COLUMNS ARE “ORTHOGONAL”)
7 - -
8 - - equal number
9 - + of + and -
11 - +
12 - -

PARTIAL CONFOUNDING IN 12 RUN P-B
Illustration showing partial confounding between X1 and X2*X3 interaction.
Rows rearranged according to level of X1.
Trial X1 X2 X3 X2*X3
1 + + - -
2 + - + -
4 + + + + 2 +’s
5 + + - - 4 -’s
6 + - - +
10 + - + -
not completely
3 - + + + balanced
7 - - - +
8 - - + - 4 +’s
9 - + - - 2 -’s
11 - + + +
12 - - - +
If there is an X2*X3 interaction effect, it will slightly bias the estimate of the X1 effect
(as well as all other main effects except X2 and X3).

USING PLACKETT-BURMAN DESIGNS
 If using tabled design, assign factors to any

columns of the design
 If blocking, use additional column(s) to
define blocks
 Save at least 4 columns for estimating
experimental error
 Assess sensitivity of design --- if inadequate:
– Use larger design
– Add reflected design

FRACTIONAL FACTORIAL DESIGNS
 Available design sizes in powers of 2

k-p
 For k factors, 2 FF design is a 1/2p fraction
of a full 2k factorial design
 16 and 32 run designs most useful for
screening
 Confounding between any two effects is
either total or absent
 Degree of confounding determined by
Resolution of design

CONSTRUCTING FF DESIGNS
TO CONSTRUCT 2k-p DESIGN, WHERE k = NUMBER OF FACTORS:
1. Construct a full factorial design in k-p of the factors.
2. Define each of the p additional factors as equal to, or the negative

of, a judiciously selected interaction among some of the first k-p
factors. These are called the defining relations of the fractional
factorial design.
For given values of k and p, tables are available of good choices

for the defining relations which will result in the highest possible
resolution design.

EXAMPLE 1: CONSTRUCTING A 24-1 DESIGN
4 FACTORS --- FOR SIMPLICITY, DENOTE FACTORS BY 1, 2, 3, 4
1. Write down full factorial 23 design in factors 1, 2, 3
2. Define factor 4 equal to 123 interaction
1 2 3 4=123 Defining Relation 4=123 induces
- - - - additional relations:
+ - - + 1=234 2=134 3=124
- + - + 12=34 13=24 14=23
+ + - - 1234=I (I =column of all +)
- - + + NOTE THAT EFFECTS ARE
+ - + - CONFOUNDED IN PAIRS:
- + + - main effects with 3-way
+ + + + 2-way with 2-way
4-way with mean
THIS IS A RESOLUTION 4 DESIGN

EXAMPLE 2: CONSTRUCTING A 25-2 DESIGN
5 FACTORS --- 1, 2, 3, 4, 5
1. Write down full factorial 23 design in factors 1, 2, 3
2. Defining relations: 4 =12 , 5=13
1 2 3 4=12 5=13 Defining Relations induce
- - - + + additional relations:
+ - - - - 1=24=35=12345 2=14=345=1235
- + - - + 3=15=245=1234 4=12=235=1345
+ + - + - 5=13=234=1245 23=45=134=125
- - + + - 25=34=123=145 I=124=135=2345
+ - + - + NOTE THAT EFFECTS ARE NOW
- + + - - CONFOUNDED IN GROUPS OF 4
+ + + + + Each main effect is now confounded
with a 2-way interaction
THIS IS A RESOLUTION 3 DESIGN

RESOLUTION OF A DESIGN
Let R denote resolution of design.
Two effects of order a and b are unconfounded if a+b < R.
However, if a+b ≥ R, then the effects may be confounded.
In particular, a main effect is unconfounded with any effects of

order less than R-1 but may be confounded with an effect of order
equal to R-1.
RESOLUTION PROPERTIES OF DESIGN
3 Main effects clear of each other but confounded with some

2-way interactions
4 Main effects clear of each other and of 2-way interactions

but 2-way interactions confounded with each other
5 or more Main effects and 2-way interactions all clear of each other

AVAILABLE DESIGN SIZES
# OF FRACTIONAL FACTORIAL PLACKETT-BURMAN

FACTORS R=3 R=4 R=5 MIN ≥ 5 DF ERROR
5 8 16 16 8 12
6 8 16 32 8 12
7 8 16 64 8 16
8 16 16 64 12 16
9 16 32 128 12 16
10 16 32 128 12 16
11 16 32 128 12 20
12 16 32 256 16 20
Note: All designs listed assuming no replicates and no center points

Section 10
SCREENING EXAMPLE
DuPont Quality Management & Technology SOE/MTB 10.1

Screening Design Example
This example is a 2-level screening experiment for a product called GLOOP.
Factors Abbreviation Settings
TENSION CONTROL TENS manual, automatic
MACHINE MACH #1, #2
THROUGHPUT TPUT 10 to
20 gal/min MIXING MIX single,
double TEMPERATURE TEMP 200°
to 250° F MOISTURE MOIST 20%
to 80%
Response Abbreviation Expected Range
PRODUCT HARDNESS Hardness 10 to 200 (Gauge)
Design = Plackett-Burman Model = Main Effects only

Special Notes: In the past day-to-day differences with this process have
been observed. Since we can only make 8 product items per day we would
also like to block this design on day. Thus, this study really involves 7
factors including the blocking variable (DAY). A preliminary standard
deviation estimate of 13 for hardness has been obtained. Detecting
changes in hardness of at least 30 units is of interest.
What Design Size Is Needed?
 Using our sample size formula we need at least:
2
 7 
n≥  = 9.2 ⇒ 10
 30/13 
n ≥ 10
 The smallest Plackett-Burman design for n ≥ 10
is the 12 run design
 The smallest Fractional-Factorial design for n ≥
10 is the 16 run design

Fractional Factorial Fit: Hardness versus TENS, MACH, ...
Estimated Effects and Coefficients for Hardness (coded units)

Constant 70.00 3.592 19.49 0.000
TENS 1.67 0.83 3.592 0.23 0.828
MACH 19.33 9.67 3.592 2.69 0.055
TPUT 26.33 13.17 3.592 3.67 0.021
MIX -0.67 -0.33 3.592 -0.09 0.931
TEMP 61.67 30.83 3.592 8.58 0.001
MOIST 7.00 3.50 3.592 0.97 0.385
DAY -20.00 -10.00 3.592 -2.78 0.050
Analysis of Variance for Hardness (coded units)

Main Effects 7 15966.7 15966.7 2281.0 14.73 0.010
Residual Error 4 619.3 619.3 154.8
Total 11 16586.0
. . .

Thought Questions
 What would happen if you analyzed this data

ignoring the day effect?
 Canwe get any information on interactions
among the significant effects with this
data?

Section 11
WORKSHOP 2:
Glyxel Screening

Workshop 2 - Problem Description
Use a screening design to determine which of the following twelve factors are
the most important in their effects on the two observed responses.
RESPONSES Bend Resistance (BR) -- valid range approximately 50-80 psi

goal = target to 68 (acceptable range is 65-71)
Area Shrinkage (AS) -- valid range approximately 0-3 %
goal = minimize (a maximum of 1% is desirable)
FACTORS LOW LIMIT HIGH LIMIT UNITS
Throughput (TP) 200 800 kg/hr
Additive A Concentration (AC) 4 8 %
Additive A Impurity (AI) 0.7 3.2 % impurity
Catalyst Amount (CA) 0.1 0.3 %
Reactor Pressure (RP) 100 150 psi
Dryer Temperature (DT) 120 150 °C
Extruder Temperature (ET) 180 190 °C
Quench Water Temperature (QT) 10 15 °C
Quench Water Flow Rate (QF) 10 20 l/min
Press Temperature (PT) 140 160 °C
Storage Temperature (ST) 10 25 °C
Block (BL) - see description 2 blocks needed week

Workshop 2 - Background Information - I
Your R&D team has been assigned to develop a new product for a critical aerospace use. There is
also a potential market in the military aeronautic and automotive industries.
The product, called Glyxel, will be produced and sold in sheet form. Two critical property goals must
be met.
Bend Resistance (BR) - (acceptable range 65 - 71 psi, target of 68) Gives a proper
balance between end-use strength and customer processing needs.
Area Shrinkage (AS) - (1.0% maximum, lower is better) Low shrinkage is required to
maintain dimensional stability through customers’ processing.
Below are brief variable descriptions and key learnings from early R&D work. First, to meet the
expected market demand, property goals should be met with the highest possible throughput (TP). It
is expected that at least 500 kg/hr will be needed for acceptable profitability.
Preliminary R&D work has indicated two process variables likely to aid in meeting bending resistance
and area shrinkage goals are reactor pressure (RP) and extruder temperature (ET).
Additive A concentration (AC) is suspected to be important for obtaining low area shrinkage. Impurity
levels of this additive (AI) are also believed to affect area shrinkage. Impurity levels are dictated by
the lot number. From our supplier we receive an an accurate estimate of impurity level through a
Certificate Of Analysis for each lot. However, since we have 20 lots to choose from that span the
range of impurity levels listed, you may experiment with impurity level and treat it as a continuous
variable.
Two other variables, quench water temperature (QT) and quench water flow rate (QF) have been
suggested as potentially affecting either bending resistance or area shrinkage.
Workshop 2 - Background Information - II
It is expected that, in operation, quench water would be supplied from a well with an average yearly
temperature range from 10 to 15 deg C. However, you have a temperature controlled water supply
(+/- 1 deg) available for experimental use. Hopefully, your team can better understand how typical
quench water temperature variation will affect the process. Another related variable that may
enhance quenching is quench water flow rate (QF).
The catalyst has been thought to enhance molecular structure development. Since our catalyst is
expensive, a lower level of CA is desirable if property goals can be satisfied.
Materials exiting the reactor are fed into a dryer to remove moisture prior to extrusion. Dryer
temperature (DT) may influence final product properties as well.
Once the product is formed into sheets, it is stored for up to two weeks in a warehouse until shipped.
High storage temperature (ST) may be related to product property deterioration.
After our customer receives our product in sheet form, it is fed into presses for final shaping. The
press design temperature (PT) is 150 deg C. However, due to press-to-press differences and
temperature control, the actual temperature can vary from 140 to 160 deg C. Your R&D team has
acquired a similar press for the duration of the experimental program.
Lastly, due to the complicated nature of this process and time required to make process and recipe
changes, it is only possible to experiment with about 12 run combinations in a week. Because a
knowledgeable technician has suggested product properties vary from week to week, we would like to
test this claim and, more importantly, remove this potential source of variation by blocking (BL) the
experiment. Since it is not possible to complete our 12-factor screening design in one week, you will
need to run it over two weeks or blocks.
Workshop 2 - Screening Design Assignment
 Choose a Plackett-Burman screening design to identify the most important factors affecting BR and
AS. The smallest Plackett-Burman design which could handle 12 factors is the 16 run design which
is actually a fractional-factorial design. As this would only have 3 degrees of freedom to estimate
experimental error and should be treated as a resolution 3 fractional-factorial design (which are
more complex to deal with), the next design might be preferable. This will be a 20-run design (with
zero replicates). Make sure you define factors and responses by the 2-letter abbreviations (factors
=>TP, AC, AI, ... ,BL responses => BR, AS).
 Preliminary standard deviation estimates for BR and AS are 1.5 psi and 0.1 %, respectively.
Assuming we are interested in detecting effects (least important difference) of at least 3.0 psi for BR
and 0.2 % for AS, will our proposed screening design will have adequate sensitivity?
 Type BR and AS as the names of two empty columns in the worksheet.
 Generate the response data using the simulator: %GLYXEL
 Examine the results in detail. Be prepared to answer the following questions and report your teams
results (see team report spreadsheets).
 Which factors have significant effects on product properties? What was the experimental error from
your results? Perhaps the most important question is: What factors have you selected to be
included in further process optimization work? You may identify up to 5 factors to carry forward to
the next design stage. Fill in your team results on the appropriate row of the team report
spreadsheets. Note, we will pick up here in the next workshop where we will follow up with a
response surface design to optimize this process.

Workshop 2: Glyxel Screening - Team Reports
Effects for BEND RESISTANCE - circle if significant Exp. Error

Team TP AC AI CA RP DT ET QT QF PT ST BL (Resid. SD)
10

Effects for BEND RESISTANCE - circle if significant Exp. Error

11
12
13
14
15
16
17
18
19
20

Effects for AREA SHRINKAGE - circle if significant Exp. Error

10

Effects for AREA SHRINKAGE - circle if significant Exp. Error

11
12
13
14
15
16
17
18
19
20

Circle Factors To Be Studied in Workshop 3

COMMENTS
Team (note that a maximum of 5 factors may be circled)
1 TP AC AI CA RP DT ET QT QF PT ST BL

Circle Factors To Be Studied in Workshop 3

COMMENTS
Team (note that a maximum of 5 factors may be circled)

Section 12
RESPONSE SURFACE
DESIGNS

Advanced Stage
RESPONSE
SCREENING INTERACTION SURFACE
DESIGNS DESIGNS DESIGNS

FACTORS


USES OF RESPONSE SURFACE MODELS
 Quantitative Understanding
 Prediction
 Optimization
 Conditions for Stability
 Calibration
 Process Control Adjustments

FACE-CENTERED CUBE DESIGN
for 3 factors
Block 1
(First Half-Fraction)
X3
Block 2
(Second Half Fraction)
Block 3
(Face Points)
Center Points
X2
X1
BOX-BEHNKEN DESIGN
for 3 factors
X3
Edge Centers
Center Point
X2
X1

SETTING UP A RESPONSE
SURFACE EXPERIMENT
 Assess the Environment

 Consider the Factors
 Consider the Responses
 Choose an Appropriate Design
 Consider Strategies for Bias Error
 Rewrite the Experimental Schedule in
Physical Units
 Review the Experiment for Operability
 Avoid Blunders
 Plan for the Analysis
 Report the Recommendations
RESPONSE SURFACE DESIGNS: Summary
 Quadratic Polynomial Models Link to Model

Diagnostics
 Danger of Extrapolation
Link to RS
 Shape of Experimental Region Example
– Cubical: Face-Centered Cube Link to RS

Workshop
– Spherical: Central Composite, Box-Behnken
 Space-Filling, Balanced, and Robust

UNDERLYING ASSUMPTIONS
 Fitted model form is “approximately correct”

– More important for Response Surface models
 Deviations from model have no systematic
component (bias error)
 Experimental error (random error) is
approximately normally distributed
 SD of experimental error is homogeneous
throughout experimental space

DIAGNOSTICS AND REMEDIES
ASSUMPTION DIAGNOSTIC TOOL POSSIBLE REMEDY
Correct model form Lack-of-fit test Model augmentation
Transformation
No bias error Residual plots Bias modeling

Outlier handling
Normal error Residual histogram Transformation

& normal plot
Fundamental knowledge
Homogeneous error Residual plots Transformation

Fundamental knowledge Weighted regression

RESIDUAL PLOTS IN MINITAB
 Minitab can calculate 3 kinds of residuals:

Regular, Standardized, or Studentized
[“Deleted”]. Standardized and “Deleted”
residuals use a standard deviation scale.
 Available plots of residuals include:
 Histogram
 Normal Probability Plot
 vs. Time
 vs. Fits (Predicted Values)
 vs. Variables
Minitab: Transformations
• Transformations in Minitab are calculated

using the Calculator… item in the Calc
pull-down menu.
• Store results of transformations in separate

columns so that original values are retained.

R-Squared & Adjusted R-Squared
Both estimate percent or proportion of

observed variability explained by model
R-squared adjusted is generally a more

honest appraisal as it adjusts for the number
of terms in the model.
Minitab will not display R-Squared values

for the analyses of 2-level designs but will
display them for response surface designs.

Section 14
RESPONSE SURFACE
EXAMPLE

RESPONSE SURFACE EXAMPLE
Problem Statement
A new compound is being developed for a coating process. Three
factors are under consideration:
Factor Range
ADDITIVE AMOUNT 0 to 70 grams
REACTION TIME 20 to 60 minutes
REACTION TEMPERATURE 100 to 180 degrees C
The yield of the compound is measured. The compound is added

to a fixed amount of formula and the coating process completed.
Adhesion is then measured. Specifications on the responses are:
YIELD ≥ 91%
ADHESION ≥ 45 grams
Find settings of the factors for which these conditions can be

achieved. Resources are available for a maximum of 24 runs.
EXAMPLE DATA PLOTTED ON CUBES
YIELD ADHESION
50 90 40 39
77 44
Temperature
Temperature
40 75 37 31
92 41
82,87, 37,41,
68 87,82, 75 24 40,40, 44
85,85 42,42
80 38
81 65 10 48
75 31
68 51 3 40
Additive Additive

Response Surface Regression: YIELD versus Additive, Time, Temperature
The analysis was done using coded units.
Estimated Regression Coefficients for YIELD
Term Coef SE Coef T P

Constant 84.55 0.6964 121.403 0.000
Additive 4.90 0.6406 7.649 0.000
Time 6.40 0.6406 9.991 0.000
Temperat -0.80 0.6406 -1.249 0.240
Additive*Additive -12.86 1.2216 -10.530 0.000
Time*Time 1.64 1.2216 1.340 0.210
Temperat*Temperat -8.36 1.2216 -6.847 0.000
Additive*Time 0.75 0.7162 1.047 0.320
Additive*Temperat 13.50 0.7162 18.849 0.000
Time*Temperat -0.25 0.7162 -0.349 0.734
S = 2.026 R-Sq = 98.9% R-Sq(adj) = 97.9%
Analysis of Variance for YIELD

Regression 9 3746.71 3746.71 416.302 101.45 0.000
Linear 3 656.10 656.10 218.700 53.29 0.000
Square 3 1627.61 1627.61 542.538 132.21 0.000
Interaction 3 1463.00 1463.00 487.667 118.84 0.000
Residual Error 10 41.04 41.04 4.104
Lack-of-Fit 5 15.70 15.70 3.141 0.62 0.694
Pure Error 5 25.33 25.33 5.067
Total 19 3787.75
...

Response Surface Regression: ADHESION versus Additive, Time, Temperature
The analysis was done using coded units.
Estimated Regression Coefficients for ADHESION
Term Coef SE Coef T P

Constant 40.24 0.5847 68.816 0.000
Additive 8.80 0.5378 16.362 0.000
Time 2.90 0.5378 5.392 0.000
Temperat 5.90 0.5378 10.970 0.000
Additive*Additive -6.09 1.0256 -5.939 0.000
Time*Time -0.59 1.0256 -0.576 0.577
Temperat*Temperat -2.59 1.0256 -2.526 0.030
Additive*Time 0.75 0.6013 1.247 0.241
Additive*Temperat -10.25 0.6013 -17.046 0.000
Time*Temperat -0.50 0.6013 -0.831 0.425
S = 1.701 R-Sq = 98.8% R-Sq(adj) = 97.7%
Analysis of Variance for ADHESION

Regression 9 2399.87 2399.87 266.653 92.18 0.000
Linear 3 1206.60 1206.60 402.200 139.04 0.000
Square 3 346.27 346.27 115.424 39.90 0.000
Interaction 3 847.00 847.00 282.333 97.60 0.000
Residual Error 10 28.93 28.93 2.893
Lack-of-Fit 5 11.59 11.59 2.319 0.67 0.665
Pure Error 5 17.33 17.33 3.467
Total 19 2428.80
Unusual Observations for ADHESION

Observation ADHESION Fit SE Fit Residual St Resid
3 37.000 40.236 0.585 -3.236 -2.03R
R denotes an observation with a large standardized residual. ...
THOUGHT QUESTIONS
Part 1
 Do any of the initial design points satisfy the

specified performance criteria for YIELD and
ADHESION ?
 Which model terms are statistically significant ?

What confidence level did you use to judge
significance ?
 How well does the model fit the data ? How did you
assess this ?
 At a given set of experimental conditions (X settings),

what is the experimental error in YIELD and
ADHESION ?
Click on OK and then OK again
in the next frame to get to:

Overlaid contours for desired values of Yield, Adhesion
Lower Bound
180 Upper Bound
White area: feasible region
170 YIELD 91
100
160 ADHESION 45
100
150
Temperature
140
130
120
110
100
0 10 20 30 40 50 60 70
Additive
Hold values: Time: 60.0

Link to RS
Workshop
Section 15
WORKSHOP 3:
Glyxel Response Surface

Workshop 3 - Problem Description
This workshop is a continuation of the Glyxel problem described in workshop 2,

however, now we will use a response surface design to optimize the process. Here
we will study in more detail the key factors identified through your screening design.
The original list of factors examined in workshop 2 are given below. From your
previous assignment you reduced this list down to the critical few variables (up to 5) to
investigate in this response surface design.
Factors (original list) Low Limit High Limit Units
Throughput (TP) 200 800 kg/hr
Additive A Concentration (AC)4 8
% Additive A
Impurity (AI) 0.7 3.2 % impurity
Catalyst Amount (CA) 0.1 0.3 %
Reactor Pressure (RP)
100 150 psi
Dryer Temperature (DT) 120 150 °C
Extruder Temperature
(ET) 180 190 °C
Quench Water Temperature (QT) 10
15 °C Quench
Water Flow Rate (QF)
DuPont Quality Management and Technology
10 20 l/min SOE/MTB - 15.2
© 2000 E. I. du Pont de Nemours and Company Press Temperature (PT) 140 160Revised°C7/02/2000
Workshop 3 - Background Information
If needed, refer back to the original problem description (Section 11 - workshop 2 background
information slides) to refresh your memory on the design variable descriptions.
Recall the two critical property goals.
Bend Resistance (BR) - (acceptable range 65 - 71 psi, target of 68) Gives a proper
balance between end-use strength and customer processing needs.
Area Shrinkage (AS) - (1.0% maximum, lower is better) Low shrinkage is required to
maintain dimensional stability through customers’ processing.
Additionally, recall that based on acceptable profitability and market demand expectations we need to
satisfy property goals with the highest possible throughput. It was previously stated that we needed
to achieve at least 500 kg/hr if possible with higher levels being more desirable.
Several of the factors originally described fall into the category of environmental variables. Such
variables may include uncontrolled, noise, ambient, raw materials, or customer use variables.
Although environmental variables are not typically controlled in operation, we may choose to explicitly
control them within the context of a designed experiment to understand their potential impact on
product properties.
Which factors explored in the Glyxel problem in workshop 2 are environmental variables? It is also
possible that some factors retained for the optimization study in this workshop are environmental
variables. What are they? The importance of Identifying environmental factor(s) and understanding
their nature and how they might be treated during optimization will become clear in the assignment
discussion and thought questions.
Workshop 3 - RS Design Assignment (Part 1)
 Select a response surface design. Choose either a CENTRAL COMPOSITE or BOX-BEHNKEN

design. Design sizes will vary according to the number of factors studied, design type, and the
number of replicates.
 Verify that your intended design will be large enough to detect the size effects we are interested in.
Recall that you want to detect a 3 psi change in BR and a 0.2% change in AS. You now have 2
standard deviation estimates for BR and AS, the preliminary estimates (given in workshop 2) and the
estimates obtained from the screening design. Which estimate would you use in your design size
or sensitivity calculations here and why?
 Type BR and AS as column names for two unused columns in the worksheet.
 Generate the response data with the simulator by typing: %GLYXELRS
in the session window. This simulator will ask for settings for your fixed factors. Enter * for factors
that are in the design.
 Proceed to analyze.
 Examine the results in detail to understand which effects are important and how well the model fits.
Fill in your teams results on the appropriate row in the BR and AS regression model results
spreadsheets and answer all thought questions (slide 7) prior to generating any contour plots.

Workshop 3 - RS Design Assignment (Part 2)
 Generate a few contour plots to get a feel for the behavior of the response surfaces. Keep in mind
that knowing the key model effects can help point you to a more promising portion of the design
space. Try to identify design regions where property goals can be satisfied.
 Initially you may want to assume that any environmental factor(s) will vary across the full design
range. Under this assumption you might begin by generating contour plots that leave
environmental factor(s) as off-axis variables set to their midpoints and then explore the limits of other
design factors to see if our property goals can be met. Next, you may want to investigate the range
of environmental factor(s) either as on-axis or off-axis variable(s). Finally, you may choose to relax
this assumption and explore optimizing with regard to all factors (including environmental ones) , to
see if this makes a difference in meeting process goals albeit recognizing greater control of such
factors(s) may be required.
 Identify recommended factor settings (where predictions satisfy stated goals) to test model
predictions with confirmatory runs. Obtaining independent data for validation is a critical step for
building confidence in the predictive capabilities of our models in the region of interest. Wait for
instructions on how to collect and process the confirmatory runs.
 Finally, answer all thought questions on slide 12 and fill in your results on the team report
spreadsheets with recommended settings, predicted response levels, and results obtained from
confirmatory runs. Also, be prepared to discuss (or assign a spokesperson from your team to
discuss) your teams approach, results, and recommendations.

Flowchart for Experimental Design and Analysis
 Perform initial design size or sensitivity calculations

 Design the experiment
 Run the experiment on the process and enter the data
 Analyze the experimental data
– fit the model to the responses
– determine the important effects and check model adequacy (examine the
various tabular summaries and residuals; any evidence of LOF?)
– model look OK?
 Generate plots describing the model
– 2D and 3D contour plots
– do any predictions from the 2D contour plots satisfy process goals?
– optimize the individual responses and then simultaneously optimize to meet
the combined process goals
 Verify predictions with check point runs

Thought Questions for Workshop 3
(Before contour plotting)
 Do any of your initial design points satisfy the property

and process goals?
 Which model terms are statistically significant? What
confidence level did you use to judge significance?
 How well does the model fit the data? Be prepared to
justify your answer.
 Is there evidence of lack of fit?
 At a given set of experimental conditions (X settings),
what is the experimental error in BR and AS (in terms of a
standard deviation)?
 Based on your results, which factors or set of factors
would you choose as on-axis factors in contour plots?
Workshop 3 - Response Surface Model Results (BR)
Design Significant Effects (list factor abbreviations) Resd Rep 2 Resids

RAdj LOF?
Used Main Effects Inter. Effects Quad. Effects SD SD OK?
Team
1
10

Workshop 3 - Response Surface Model Results (BR)

RAdj LOF?
Team
11
12
13
14
15
16
17
18
19
20

Workshop 3 - Response Surface Model Results (AS)

RAdj LOF?
Team
1
10

Workshop 3 - Response Surface Model Results (AS)

RAdj LOF?
Team
11
12
13
14
15
16
17
18
19
20

Thought Questions for Workshop 3
(After contour plotting)
 Which contour plot(s) were the most informative?

 What is the most promising region of the design space for
satisfying the property goals? Are throughput levels OK?
 What confirmatory (check point) runs did you make?
Were your predictions supported? Requirements met?
 What conclusions can you make, if any, on the impact the
environmental factor(s) may have on meeting your process
goals?
 What recommendations would you make, if any, to improve
the process by exercising greater control of the
environmental factor(s)?
 What plot(s) and / or tables would you include in a report
of your results?
Workshop 3 (Glyxel Response Surface) - Team Reports
Recommended Settings (use MP for MidPoint of excluded factors) Predicted Obtained

Team TP AC AI CA RP DT ET QT QF PT ST BR AS BR AS
10

Workshop 3 (Glyxel Response Surface) - Team Reports
Recommended Settings (use MP for MidPoint of excluded factors) Predicted Obtained

Team TP AC AI CA RP DT ET QT QF PT ST BR AS BR AS
11
12
13
14
15
16
17
18
19
20

Section 16
OTHER EXPERIMENTAL
ENVIRONMENTS

SOME OTHER TYPES OF
EXPERIMENTAL SITUATIONS
 Categorical factors with more than 2 levels

 Constrained regions
 Mixture problems
 Incomplete block designs
 Split plot designs
 Nested designs
 Supersaturated designs
FOR THESE TYPES OF SITUATIONS
CONSULT AN EXPERT!

CATEGORICAL FACTORS
WITH MULTIPLE LEVELS
 Catalysts
 Electronic components
 Suppliers
 Operators
 Machines
 Brands or types of formulation ingredients

MIXTURE EXAMPLES
 Drugs  Paints
 Gasoline Blends  Dyes
 Metal Alloys  Textile Fiber Blends
 Rocket Propellants  Concrete
 Aerosol Propellants  Cake Mixes
 Herbicides  Composite Materials

A CLASSIC MIXTURE
5 Parts Gin 1 Part Vermouth

MIXTURE CONSTRAINT
0 ≤ Xj ≤ 1 Σ X =1
j=1
j
q-1
so Xq = 1 - ΣX
j=1
j
 Mixture Components Cannot Be Varied

Independently
 Factorial and Response Surface Designs
Cannot Be Used

FACTOR SPACE IN TWO VARIABLES
Independent Mixture
+ 1
X2 X2
- 0
- X1 + 0 X1 1
FACTOR SPACE IN THREE VARIABLES

Independent Mixture
X3
X3
X1+X2+X3=1
X2 X2
X1 X1

FACTOR SPACE FOR A
FOUR-COMPONENT MIXTURE
0,1,0,0
0,0,1,0 1,0,0,0
0,0,0,1

QUADRATIC RESPONSE-SURFACE
MODEL FOR TWO FACTORS
Y = a0 + a1X1 + a2X2 + a12X1X2 + a11X12 + a22X22

The Mixture Constraint
X1 + X2 = 1
X12 = X1*X1 = X1(1 - X2) = X1 - X1*X2
X22 = X2*X2 = X2(1 - X1) = X2 - X1*X2
Quadratic Mixture Model (Scheffé)
Y = b1X1 + b2X2 + b12X1X2

TWO-COMPONENT MIXTURE
Scheffé Linear & Quadratic Model
Y
b2
d = 0.25*b12
b1
X1 1.0 0.5 0.0
X2 0.0 0.5 1.0

THREE-COMPONENT MIXTURE DESIGN
X1 = 1
Pure Component
Binary Blend
Ternary Blend
Check Points
X2 = 1 X3 = 1
X1 = 0

ROCKET PROPELLANT CONTOUR PLOT
0.0
*1.0
0.1
0.9
0.2
0.8
0.3
0.7
0.4
* 700
0.6
800
0.5
* 900 * 0.5 Maximum Near
0.6
0.4 (0.2, 0.3, 0.5)
0.7 * 1000
0.3
0.8
0.2
0.9
* *
800
0.1
X2 1.0 1000
* * * 0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
X3

MINIMUM COMPONENT LEVELS
 Concrete
 Cake
 Steel

MINIMUM COMPONENT LEVELS
0.0
X2 = 0.14 X1 = 1
Requirements :
X1 ≥ 0.18
X2 ≥ 0.14
X3 ≥ 0.11
0.5
0.5
X1 = 0.18
X2 = 1 0.0
0.0 0.5 X3 = 1
X3 = 0.11

MINIMUM AND MAXIMUM COMPONENT LEVELS
Requirements : 0.0
X2 = 0.14 X1 = 1
0.18 ≤ X1 ≤ 0.51
0.14 ≤ X2 ≤ 0.52
0.11 ≤ X3 ≤ 0.58
X2 = 0.52
0.5
0.5 X1 = 0.51
X1 = 0.18
X2 = 1 0.0
0.0 0.5
X3 = 1
X3 = 0.11 X3 = 0.58

FLARE EXPERIMENT: DESIGN
X2 = NaNO3
Vertices
Face Centers
Center
X3 = SrNO3 X1 = Mg
X4 = Binder

AEROSOL PROPELLANT STUDY
X1 = 1
Mixture
Highly
Flammable
X2= 1 X3 = 1

EXAMPLE OF CONSTRAINED REGION:
Petroleum Fractionation Process
Amount
of Toluene
In Solvent Equipment Fouling:
No Phase Separation
Region
of Interest
Unfavorable
Economics
Solvent/Solute

STEPS FOR DESIGNS IN IRREGULAR REGIONS
 Define Region
 Incorporate the Principles of Good Design
 Identify Candidate Runs
– Include extreme points
 Select Runs
– By inspection, if geometry simple
– Using computer-aided algorithmic design
(e.g. D-Optimal Design)

OTHER EXPERIMENTAL ENVIRONMENTS:
Summary
 Discrete Factors
 Mixture Designs
 Constrained Factor Spaces

Section 18
SUMMARY

EXPERIMENTAL DESIGN VERSUS
ONE-FACTOR-AT-A-TIME EXPERIMENTS
 Drawbacks of one-factor-at-a-time experiments

– Not space-filling
– Ignores interactions
– Ignores experimental error
– Inefficient due to lack of hidden replication
– Limited system understanding
– Potential bias error due to lack of randomization

EXPERIMENTAL DESIGN VERSUS
ANALYSIS OF HISTORICAL DATA
 Drawbacks of historical data analysis

– Correlations between factors
– Unrecorded control actions may create misleading
effects, confusion of cause and effect
– Typical lack of boldness in factor settings
– Data collection problems
– Missing data
– Bad observations
– Large bias errors due to lack of randomization
– At best describes “What is” instead of “What is
possible”
WHEN SHOULD I USE DOE ?
 Discovery Research and Scouting

 Product/Process Design and Development
 Process Scale-up, Startup, and
Qualification
 Process Control and Calibration
 Product/Process Improvement

PRINCIPLES OF GOOD
EXPERIMENTAL STRATEGY
 Diagnosis of the Environment (objectives,

prior knowledge, number & nature of factors)
 Balanced Statistical Designs
 Measure All Relevant Responses
 Bite-Sized Experiments
 Boldness
 Randomization and Blocking
 Estimate Experimental Error
 Avoid Blunders
 Plan Ahead for Statistical Analysis

EXPERIMENTAL EVOLUTION
Screening Designs Interaction Designs Response Surface Designs

Many factors Fewer factors Small number of factors (3-6)
Distinguish “critical few” Identify/exploit interactions Used for prediction, optimization,
from “trivial many” Model contains linear terms modelling, . . .
Linear model: and at least some interactions: Quadratic model:
Y = b0 + b1X1 + b2X2 Y = b0 + b1X1 + b2X2 Y = b0 + b1X1 + b2X2
+ b3X3 + b4X4 + . . . + b3X3 + b12X1*X2 + b3X3 + b12X1*X2
Plackett-Burman and + b13X1*X3 + b23X2*X3 + b13X1*X3 + b23X2*X3
Fractional Factorial Full and Fractional Factorial + b11X12 + b22X22 + b33X32
Usually only a few more Face-centered cube, Box-Behnken,
runs than factors others

DOE APPLICATION PROCESS
Strategy of Experimentation
Identify Gather Information

Business
Needs Define Experimental Objectives
Design the Experiment
Run Experiment Update

Information
Analyze Experiment
Interpret Results
Perform Confirmation Runs
Go to Next Stage of Experimentation?
Apply Results Assess, Document

& Communicate
Business Results

ANALYZING EXPERIMENTS: Flowchart
Screening Inter action Response Sur face
Designs Designs Designs
ENTER DATA
DISPLAY DATA
PLOT/VERIFY DATA
FIT MODEL
Main Effects Main Effects Quadratic

and Interactions Model
ASSESS SIGNIFICANCE
VALIDATE ANALYSIS
PLOT RESULTS
Rank Effects Exploit Find “ Optimum”

Interactions

WHEN DO I NEED A
D-OPTIMAL DESIGN ?
You may need a d-optimal design for the following

situations:
– Discrete/Qualitative factors at more than 2 levels
– Constrained regions (including mixtures)
– A special model (mixed number of levels of factors or models
with some terms excluded due to your assuming that they have
negligible effects on the responses of interest)
– To augment an existing design to be able to estimate a larger
model (in some cases) -- assuming no change in process other
than possibly a level shift (use blocking) between when the
existing and new data are collected
Do not overuse d-optimal designs. Use standard

designs whenever they are appropriate.

Section 19
MIXTURES IN MINITAB

TYPES OF MIXTURE DESIGNS
AVAILABLE IN MINITAB
 Simplex centroid
– use when components have no upper bounds (lower bounds are OK)
– includes:
• 2q-1 runs for q components
• all pure component (100% of component) runs, all binary (½, ½) blends,
all ternary ( 1 3 , 1 3 , 1 3 ) blends, …
 Simplex lattice
– use when components have no upper bounds (lower bounds are OK)
– includes:
• degree 1 design: pure component runs
• degree 2 design: pure component runs, binary blends
• degree 3 design: pure component runs, binary blends, ternary runs, all
( 1 3 , 2 3 ) blends
 Extreme vertices
– use when components have upper bounds
CREATING MIXTURE DESIGNS
IN MINITAB
 Start a new Minitab project
 From the Stat pull-down menu, select
DOE
Mixture
Create Mixture Design
 Select the number of components
 Select the type of mixture design
 Click on Designs… to select the points to be
included

EXAMPLE WITH LOWER AND UPPER
BOUNDS ON COMPONENTS:
Choose type of design
Effects Ranges
X1 .18 to .51
X2 .14 to .52
X3 .11 to .58

Extreme Vertices Design Options:
Vertices and Center Point only

Vertices, Center Point, Axial Points

Vertices, Center Point, Binary Blends

Defining factors in Minitab
Effects Ranges
X1 .18 to .51
X2 .14 to .52
X3 .11 to .58

Minitab worksheet from extreme vertices
design with degree=2 and center point

VISUALIZING THE MIXTURE
DESIGN IN MINITAB
After producing the design:

 Select from the Stat pull-down menu
DOE
Mixture
Simplex Design Plot
 Click OK to use
the default graph
settings

ANALYSIS OF MIXTURE DATA
IN MINITAB
 Select from the Stat pull-down menu
DOE
Mixture
Analyze Mixture Design
 Select the response column to analyze
 The default model is quadratic -- click OK

Minitab analysis results
Regression for Mixtures: Response versus X1, X2, X3
Estimated Regression Coefficients for Response (component proportions)
Term Coef SE Coef T P VIF

X1 44.14 24.42 * * 243.89
X2 4.60 20.66 * * 168.54
X3 -31.81 13.44 * * 82.35
X1*X2 194.74 77.83 2.50 0.041 280.33
X1*X3 209.88 62.86 3.34 0.012 160.19
X2*X3 165.30 49.35 3.35 0.012 77.67
S = 2.0298 PRESS = 85.656

R-Sq = 98.26% R-Sq(pred) = 94.85% R-Sq(adj) = 97.02%
Analysis of Variance for Response (component proportions)

Regression 5 1632.89 1632.8886 326.5777 79.26 0.000
Linear 2 1561.86 79.9464 39.9732 9.70 0.010
Quadratic 3 71.03 71.0328 23.6776 5.75 0.027
Residual Error 7 28.84 28.8406 4.1201
Total 12 1661.73
Unusual Observations for Response
Observation Response Fit SE Fit Residual St Resid

1 46.000 49.697 1.321 -3.697 -2.40R
R denotes an observation with a large standardized residual

Minitab Response Trace Plot
Minitab’s response trace
plot is similar to an
effects plot for mixture
components.
 From the Stat pull-
down menu, select
DOE
Mixture
Response Trace Plot
 Click OK for default
settings

Minitab contour plot
 Produce a contour plot of
the model fit by selecting
from the Stat pull-down
menu:
DOE
Mixture
Contour/Surface (Wireframe)
plots
 Click on Contour plot
Setup
OK (for default settings)
 3-D plots and optimizations
Note that this plot was produced by
are available also requesting specific contour levels.

THOUGHT QUESTION
What if sample sizes test tells you only need 8 design

points -- and each experiment will be very
expensive.
What would you do?

MIXTURE EXAMPLE 2
 Create a Mixture Design with the Following

Constraints
– Variable 1 must be less than 0.60.
– Variable 2 must be less than 0.70.
– Variable 3 must be GREATER than 0.20.
 You will need an extreme vertices design due to the

maximums given for variables 1 and 2.

MIXTURE EXAMPLE 2:
2 Design Alternatives
Minitab’s default Minitab’s degree=2

design with axial design without axial
points points

MIXTURE EXAMPLE WITH LINEAR
CONSTRAINTS
Four factors with their ranges.

Poly 0.10 to 0.30
Comp1 0.00 to 0.15
Comp2 0.00 to 0.15
Filler 0.55 to 0.85
We have the following additional constraints
B + C <= 0.15
B + C >= 0.05.

CONSTRAINTS: Define Variables

CONSTRAINTS: Define Constraints

CONSTRAINTS: Extreme Vertices
with Axial Points
Run Poly Comp1 Comp2 Filler
1 0.15 0.050 0.025 0.775
2 0.25 0.050 0.025 0.675
3 0.25 0.100 0.025 0.625
4 0.15 0.025 0.050 0.775
5 0.10 0.000 0.050 0.850
6 0.25 0.025 0.100 0.625
7 0.20 0.050 0.050 0.700
8 0.10 0.050 0.000 0.850
9 0.30 0.000 0.050 0.650
10 0.10 0.000 0.150 0.750
11 0.30 0.050 0.000 0.650
12 0.15 0.025 0.100 0.725
13 0.15 0.100 0.025 0.725
14 0.10 0.150 0.000 0.750
15 0.30 0.000 0.150 0.550
16 0.25 0.025 0.050 0.675
17 0.30 0.150 0.000 0.550

Section 20
GLOSSARY OF TERMS

GLOSSARY OF TERMS
algorithmic designs
Designs that are computer-generated for very specific situations, often using an algorithm
tied to some particular optimality criterion, such as D-optimality.
analysis of variance (ANOVA)
Procedure for analyzing data which involves partitioning the total variation into portions
explainable by a model and unexplainable. Through appropriate partitioning, the statistical
significance of particular model terms or groups of terms can be tested using F-ratios.
axial points
Points in a central composite design having the property that all but one of the factors are
set at their middle level. Syn: star points.
balance
Desirable characteristic of an experimental design wherein design points are allocated in
a manner that is balanced with respect to the center of the factor space.
boldness
The recommended practice in experimentation of investigating factors over wide ranges.
bias error
Variability in the response data that is of a systematic, patterned nature, often due to a
single assignable cause. Syn: systematic error.
blocking
Running the experiment in specially chosen subgroups, called blocks, within which the
experimental conditions or material is expected to be more homogeneous than between
blocks.
Box-Behnken designs
A particular class of response surface designs which are spherical in shape.
categorical factor
A factor that can assume only a finite number of possible values or levels. The levels may
be numerical or non-numerical labels, and are usually treated as unordered. Syn:
discrete factor.
centering
The procedure of re-expressing the scale of a continuous factor by subtracting a central
value, such as the midpoint of the factor’s experimental range, from the factor values.
Fitting polynomial models in the centered factors results in more easily-interpreted
coefficients, and reduces correlations among the coefficients.
central composite design
A class of response surface designs which consist of corner points, axial points, and the
overall centroid.
central composite in cube design
The same as a face-centered cube. These are a special class of central composite
designs in which the axial points are located in the centers of the faces of the cube.
central composite in sphere design
A special class of central composite designs in which the axial points are located outside
the faces of the cube, on the surface of the circumscribed sphere.
confidence interval
An interval within which an unknown population parameter is estimated to lie. The
confidence level (e.g. 95%) associated with the interval represents the long-run
percentage of times that the interval will actually include the population parameter.

GLOSSARY OF TERMS
confounding
A property of a design wherein the estimates of certain effects are correlated with each
other. When two effects are totally confounded they are inseparable from each other --
they are both estimated with the same contrast. Syn: aliasing.
continuous factor
A factor which can take on any value over some numerical range.
contour plot
A two-dimensional plot of the relationship between two continuous factors and a
response, in which the plot axes represent the two factors, and points of constant
response are connected by curves, called contour lines.
correlation
A measure of the degree to which values of one variable change in concert with values of
another variable.
D-optimal design
A design that minimizes the volume of the region of uncertainty of the unknown model
parameters. Usually generated algorithmically.
degrees of freedom (d.f.)
The number of independent pieces of information used to fit a model (model d.f.) or
estimate experimental error (residual or replicate d.f.)
design
The set of specific factor combinations to be run in the experiment. Usually specified in a
design table with factors assigned to columns and runs to rows which indicate the factor
combinations to be run.
design of experiments (DOE)
A strategic process, with supporting methods and tools, for guiding the planning,
execution, analysis, and application of results of experimental or developmental
programs.
discrete factor
see categorical factor
duplicate
A repeated run that does not repeat all elements of the 'run' process. Examples: making
one piece of product and measuring it more than once, or setting conditions once and
making multiple pieces of product and measuring each. Note distinction from replicate.
effect
A difference of averages: high level average - low level average. The expected change in
the response as you go from the low to high level of the factor/interaction/etc.
efficiency
A comparison of the current design vs. a theoretical optimum. May be based on any of
several commonly-used criteria (e.g. D-efficiency or G-efficiency). Usually expressed as a
percentage between 0 and 100; 100% efficiency may not always be achievable.
environmental factor
Factor that may affect product functionality but is not controlled during normal production
or use. Syn: noise factor.
experimental error
Lack of repeatability (variability) in the experimental outcomes.
extrapolation
Making predictions outside the range covered by the current data.
F-test
A ratio of variances or mean squares used to compare means, or to compare variances,
or to test the significance of terms or groups of terms in models.

GLOSSARY OF TERMS
face centered cube design
See central composite in cube design.
factor
A variable which is deliberately manipulated in an experiment. Syn: independent
variable, knob, predictor, input variable, controlled variable.
factorial design
Experimental design that is generated by using all possible combinations of each of the
levels of the factors; most commonly used in cases where the factors each have two
levels.
foldover design
A design obtained by adding the reflection of a design to the original design (thereby
doubling the number of runs); the reflection is obtained by reversing all ‘+’ and ‘-’
signs in the coded design table. A foldover Plackett-Burman design will isolate the main
effects from two-way interactions. Syn: reflected design.
fractional factorial design
Subset of a full-factorial design that is formed by totally confounding factor effects with
certain high-order interactions.
Gaussian distribution
See normal distribution.
hidden replication
The apparent repeating of factor combinations when a balanced design is collapsed over
variables not involved in the effect of interest.
high-order interactions
Interaction terms that involve several (typically 3 or more) variables simultaneously.
Frequently used as a basis for blocking or determining fractional factorial designs.
historical data
Data taken during the normal operation of a process where factors are not varied in a
deliberate, planned manner.
influence
A numerical measure of the importance of an observation in determining the fitted model.
Various commonly-used measures are available, which may depend on either the location
of the point in the design space, or the response value at that point, or both.
inoperable region
A subset of the design space in which response data can not be collected/used.
interaction
A condition involving two or more factors in which the effect of one factor depends on the
levels of the other(s).
interaction model
A model which consists of main effects and 2-factor interactions (higher-order interactions
generally not included).
interpolation
Making predictions within the range covered by the current data.
lack-of-fit test
An analysis that compares residual variability versus replicate variability to assess
whether the model could somehow be significantly improved using the current data.
linear model
A model which includes main effects only.
mean
The average.

GLOSSARY OF TERMS
mixture experiment
An experimental environment where the factors of interest are the proportions of various
ingredients in a formulation.
model
A mathematical representation of the relationship between the factors and the response.
normal distribution
A symmetric, bell-shaped function that represents the expected frequency of data values,
often used to model the distribution of random error. It is specified by a mean and
standard deviation. Syn: Gaussian distribution.
one-factor-at-a-time experimentation
An experimental strategy which involves holding all factors constant except one, which is
varied across a range. This process is then repeated in turn for each factor.
orthogonal design
A design in which the columns of the design matrix are uncorrelated with each other.
orthogonal coding or scaling
Expressing the range of a factor on a -1 to +1 scale.
outlier
An observation that appears to be far removed from the range of variation of the other
observations in the data set. May suggest a possible error or anomaly.
p-value of an estimated effect
The probability of observing an effect that large purely by chance i.e. when the true effect
is actually zero. The smaller the p-value, the stronger the evidence of a real effect.
parameter
An unknown constant associated with the population.
Plackett-Burman design
A class of two-level screening designs that exist in multiples of 4 runs. Factor effects are
not completely independent of 2-factor interactions. Designs where the number of runs is
a power of 2 should be treated as fractional factorials.
pooled standard deviation
A combined estimate of experimental error variability based on replicating more than one
set of experimental conditions. Assumes that the true variance is about the same for all
factor combinations in the experiment.
population
The hypothetical set of all possible data values of a variable.
practical significance
The degree to which an estimated effect or parameter represents something meaningful
or useful in the context of the experimental environment.
pure error standard deviation
See replicate standard deviation.
quadratic model
A continuous factor model which consists of main effects, 2-factor interactions, and
curvature terms in each factor.
quality
The totality of features and characteristics of a product or service that bear on its ability to
satisfy stated or implied needs.
R-squared
The proportion of the total variation of the response explained by fitting the model. Also
known as the coefficient of determination. Increases with the addition of model terms,
regardless of their significance.

GLOSSARY OF TERMS
R-squared adjusted
An adjusted form of R-squared that takes into account the number of terms in the model
vis-a-vis the total number of observations. Provides a more equitable measure for
comparing models of different sizes than the unadjusted R-squared. Can be negative for
models that have no value.
random error
Variability in the response data which exhibits no systematic pattern. It can not be
attributable to any single cause.
randomization
The deliberate scrambling of the run order of the design so that any bias present is
unlikely to be confounded with any factor effects, but will appear as random variation.
reflected design
See foldover design.
regression (least squares)
A method of fitting a model to a set of data by minimizing the sum of squares of the
deviations from the model.
regression F-test
A test that assesses the overall significance of the model. Compares the variance
explained by the model to the variance unexplained by the model.
replicate
A repeated run that includes repetition of all components of the 'run' process (i.e.changing
factor settings, making product, measuring product, etc.) Note distinction from duplicate.
replicate standard deviation
A pooled standard deviation computed using all replicated sets of runs. Provides an
estimate of the standard deviation of experimental error. Syn: pure error standard
deviation.
residual
The difference between an observed response value and the response predicted from the
model.
residual standard deviation
A standard deviation based on the set of all residual values. Under the assumption that
the fitted model form is valid, provides an estimate of the standard deviation of
experimental error.
resolution (of fractional-factorial design)
A number which indicates the level of confounding in a fractional-factorial design. Higher
numbers imply less confounding. A design of resolution R is one in which no p-factor
effect is confounded with any other effect containing fewer than R-p factors.
response
A variable which is observed/measured whose value may depend upon the settings of the
design factors. Syn: dependent variable, property, characteristic.
response surface experiment
A stage of experimentation where the experimental data is to be used for optimization,
calibration, prediction, etc. Quadratic models are typically used as the fitted models.
robust
Insensitive to changes in environmental conditions.
screening experiment
A stage of experimentation where the experimenter needs relatively crude information
about which of a relatively large number of factors are important. Models containing main
effects only are typically used here.

GLOSSARY OF TERMS
signal-to-noise ratio
Delta/s where delta is the minimum change in the response that is desired to be detected
and s is the standard deviation of experimental error.
standard deviation
A measure of the variability in a population or set of data. The square root of the
variance.
standard error
The variability associated with an estimated effect or coefficient.
standardized residual
The residual divided by the estimated standard deviation of the residuals where all
observations contribute to the standard deviation.
statistic
A numerical characteristic of a sample; for example, mean and standard deviation.
statistical significance
A conclusion made from statistical analysis of data that a difference or effect is real.
studentized residual
The residual divided by the estimated standard deviation of the residual where the current
observation is omitted from the standard deviation calculation.
transformation
A function applied to a variable (typically a response) to help improve the fit of the model,
or to make the statistical analysis more valid.
variable
A measurement or observation for which any of various possible data values can occur.
variance
A measure of the variability in a population or set of data. The square of the standard
deviation.

Section 21
REFERENCE

© 2000 E. I. du Pont de Nemours and Company Revised: 5/08/2000
CONTENTS
 Catalogue of Designs
 Blank Cube Diagrams
 Defining Relations for Fractional-Factorial Designs
 Miscellaneous Formulas
 Selected DOE Bibliography
 DOE Related Accession Reports
 Consultant List (hand-out)

DESIGN CATALOGUE: Two-level Designs
Number of Confounding
Design Distinct Points of Model Terms
Full Factorial 2k none
Fractional Factorial 2k-m either total or none - see pages

later in this section
Plackett-Burman multiples of 4 main effects partially

that are not powers of 2 confounded w/interactions
Plackett-Burman multiples of 8 main effects clear of 2-factor

plus Reflection that are not powers of 2 interactions
k = number of factors in design

m = degree of fractionation

DESIGN CATALOGUE:
Response Surface Designs
Factor Shape of
Design Levels Design Space
Face-Centered Cube 3 Cubical
Box-Behnken 3 Spherical
Spherical Central- 5 Spherical

Composites

23 FACTORIAL DESIGN

TWO 23 FACTORIAL DESIGNS

33 FACTORIAL DESIGN

TWO 33 FACTORIAL DESIGNS

THREE-FACTOR FACE-CENTERED CUBE

TWO THREE-FACTOR FACE-
CENTERED CUBES

THREE-FACTOR BOX-BEHNKEN DESIGN

TWO THREE-FACTOR BOX-
BEHNKEN DESIGNS

FRACTIONAL-FACTORIAL DESIGNS
AVAILABLE WITH VARIOUS RESOLUTIONS
Number of runs required for:
Number of Resolution Resolution Resolution Full
Factors III IV V or more Factorial
3 4 - - 8
4 - 8 - 16
5 8 - 16 32
6 8 16 32 64
7 8 16 64 128
8 - 16 64 256
9 16 32 128 512
10 16 32 128 1024
11 16 32 128 2048
12 16 32 256 4096

MISCELLANEOUS HANDY FORMULAS
Sample Mean Standard Error of a Factor Effect
ΣY
n
√
Y1 + Y2 + . . . Yn
Y= = i=1
i
sFE = sp 1 1
n n +
n1 n2
Sample Variance where n1 and n2 are the number of observations in
the low and high “halves” of the factor effect.
(Y1-Y)2 + (Y2-Y)2 + . . . + (Yn-Y)2
s2 = Confidence Interval
n-1
estimate +/- t * standard error
Sample Pooled Variance t is a tabled Student-t quantile whose value depends
on the degrees of freedom & confidence used
(n1-1)s12+(n2-1)s22+ . . .+(nk-1)sk2
sp2 = When zero is not included in the confidence interval, the effect is
statistically significant
(n1-1)+(n2-1)+ . . .+(nk-1)
where s12 , s22 , . . . sk2 are the individual variances Experiment Size
and n1 , n2 , . . . nk are the number of replicate For 2-level designs
measurements at each combination
Sample Standard Deviations

s=√ s2 sp = √ sp 2
(
n = 7 or 8
∆/s
2
For 3-level designs

) where ∆ is effect size desired to
detect and s = estimate of std dev.
Estimated Effect about 50% more than for 2 levels

Yhigh - Ylow

MISCELLANEOUS HANDY FORMULAS
(continued)
Explained Variation (%) in Regression
(R-Squared)
_
Σ
n
100 (Ypredicted - Y)2

i=1
R2 = _
Σ
n
(Yi - Y)2
i=1
Explained Variation in Regression (R-

Squared adjusted)
Σ(Y - Y
n
2
pred) /(n-k)
( )
i
i=1
R2(adj) = 100 1- _
Σ
n
(Yi - Y)2 /(n-1)
i=1
where k = number of model terms (including constant)
R2(adj)= 100 (1 - (varianceresidual/ variancetotal))

SELECTED DOE BIBLIOGRAPHY
 Design and Analysis of Experiments, Douglas C. Montgomery, 3rd ed.,
1990, John Wiley and Sons, Inc.
 Empirical Model Building and Response Surfaces, George E. P. Box
and Norman R. Draper, 1987, John Wiley and Sons, Inc.
 Experiments with Mixtures, John Cornell, 2nd ed., 1990, John Wiley and
Sons, Inc.
 Response Surface Methodology, Raymond H. Myers, 1976, Virginia
Polytechnic Institute and State University.
 Statistical Design & Analysis of Experiments With Applications to
Engineering and Science, Robert L. Mason, Richard F. Gunst, James L.
Hess, 1989, John Wiley & Sons, Inc.
 Statistics for Experimenters, George E. P. Box, William G. Hunter and J.
Stuart Hunter, 1978, John Wiley and Sons, Inc.
 Strategy of Experimentation, Course Text (1988), DuPont Quality
Management & Technology Center, Wilmington, DE.

AVAILABLE DOE-RELATED MATERIALS
 Design of Experiments: A Competitive Advantage
– Introduction in question and answer format
– Accession Report #17934
 Design of Experiments Application Guide
– Outline of DOE as a strategic process. How and when to apply the
tools and methods effectively.
 Design of Experiments Quick Reference Guide
– Quick reference guide for trained users of DOE as a strategic
process. Includes tools and methods, examples and catalogs.
 Design of Experiments Overview (presentation)

TABLE OF CONTENTS
SECTION TITLE
1 Introduction
2 Workshop 1
3 Foundations of the Strategy
4 Factorial Geometry
5 Factorial Example: Design
6 Analysis of Two-Level Factorial Designs
7 Factorial Example: Analysis
8 Good Experimental Practice
9 Screening Designs
10 Screening Example
11 Workshop 2 - Glyxel Screening
12 Response Surface Designs
13 Model Diagnostics
14 Response Surface Example
15 Workshop 3 - Glyxel Response Surface
16 Other Experimental Environments
17 Algorithmic Design
18 Summary
19 Mixtures in Minitab
20 Glossary of Terms
21 Reference
Agenda Additions:
Questions & Answers

Your Data Session
DuPont Quality Management and Technology SOE/MTB


Design of Experiment Using Minitab Book - 1

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Design of Experiment Using Minitab Book - 1

Diunggah oleh

Hak Cipta:

Format Tersedia

Section 1

DuPont Quality Management and Technology SOE/MTB - 1.1

DOE is a strategic process, with supporting

DuPont Quality Management and Technology SOE/MTB - 1.2

 Highest leverage quality tool available --- use to

DuPont Quality Management and Technology SOE/MTB - 1.3

 DOE is useful in every stage of product life

DuPont Quality Management and Technology SOE/MTB - 1.4

 Define and translate customer needs

DuPont Quality Management and Technology SOE/MTB - 1.5

 Reduce time to process qualification

DuPont Quality Management and Technology SOE/MTB - 1.6

 Develop process models

DuPont Quality Management and Technology SOE/MTB - 1.7

 Reduce product variability

DuPont Quality Management and Technology SOE/MTB - 1.8

 An appreciation of the basic underlying

DuPont Quality Management and Technology SOE/MTB - 1.9

DuPont Quality Management and Technology SOE/ECHIP - 2.1

DuPont Quality Management and Technology SOE/ECHIP - 2.2

Your problem is:

Your boss’s best guess of a good place to start is:

 Experimental Variability Is a Fact of Life

DuPont Quality Management and Technology SOE/ECHIP - 2.6

DuPont Quality Management and Technology SOE/ECHIP - 3.1

DuPont Quality Management and Technology SOE/ECHIP - 3.2

 Almost always more than one factor of interest

DuPont Quality Management and Technology SOE/ECHIP - 3.3

variables that are outcome variables variables that are

DuPont Quality Management and Technology SOE/ECHIP - 3.4

 Experimental error is the “noise” in the

DuPont Quality Management and Technology SOE/ECHIP - 3.5

• Unpatterned variability • Normal error

DuPont Quality Management and Technology SOE/ECHIP - 3.6

DuPont Quality Management and Technology SOE/ECHIP - 3.7

RANDOM ERROR SYSTEMATIC ERROR

CAUSE Unassignable Assignable

NATURE Unpatterned Patterned

REMEDY Replication Blocking

DuPont Quality Management and Technology SOE/ECHIP - 3.8

•Through averaging of replicates the impact

DuPont Quality Management and Technology SOE/ECHIP - 3.9

 Used in presence of identifiable source of

DuPont Quality Management and Technology SOE/ECHIP - 3.10

 “Insurance” protection against potential

DuPont Quality Management and Technology SOE/ECHIP - 3.11

no Update Information Design Experiment

DuPont Quality Management and Technology SOE/ECHIP - 3.12

SCREENING INTERACTION RESPONSE

NUMBER OF 6 or more 3-8 2-6

COMMON Plackett-Burman Full Factorial Box-Behnken

DuPont Quality Management and Technology SOE/ECHIP - 3.13

DuPont Quality Management and Technology SOE/ECHIP - 3.14

DESIGN Vary only 1 factor at time, Vary all factors jointly in

FITTED MODEL Curves fitted through data Simple empirical models

EXPERIMENTAL Ignored Recognized and estimated

DuPont Quality Management and Technology SOE/ECHIP - 3.15

DuPont Quality Management and Technology SOE/ECHIP - 3.16

DuPont Quality Management and Technology SOE/ECHIP - 4.1

SCREENING INTERACTION RESPONSE

NUMBER OF 6 or more 3-8 2-6

COMMON Plackett-Burman Full Factorial Box-Behnken

12 + 12 + 18 + 12 + 18 + 12 + 12 + 12