Perspective on Use of Statistical Tools

in Pharmaceutical Manufacturing

Karthik Iyer (CQE, CSSBB)

Senior Policy Advisor
CDER/OC/OMPQ
March 9th, 2012
AOAC Conference
* This presentation reflects the views of the author and should not
be construed to represent FDAs views or policies.

Agenda

Enforcement Action Examples

CGMP References
ASTM Standards
Conclusions

Recent warning letters and other compliance issues

Examples involving
Incorrect application of sampling plans
Equipment changes and process
capability
Container closure - determining baseline
defect rates
Recall example application of ASTM
E2709
3

1. Warning Letter sampling plans

Firm using sampling plans incorrectly

Pooled X vials, used only 1 reportable value, but used
n=X in sampling plan.

.based your lot or batch acceptance/rejection

criteria on a single reportable value averaged from a
pooled sample.

For .., you are collecting 3 pooled samples (each pool

= 10 vials). This equates to a lot disposition action on
3 reportable values with corresponding AQL of X%
and LQ of X% respectively. This is not equivalent to
an X or X plan as claimed in your SOP.
4

1. Warning Letter sampling plans

Response to 483 indicated firm did not know
how to use and interpret sampling plans
correctly.
Firm did not understand concepts of
Acceptable Quality Level (AQL) and Limiting
Quality (LQ) and Operating Characteristic
Curve (OC) of a specific sampling plan.

2. Warning Letter equipment comparability and process

capability

Four (4) tablet products, various strengths

Initial process qualification used a single-sided tablet
press. During routine production, however, these
products were also being manufactured using a
double-sided tablet press.
Compression using the double sided press was not
qualified.
Firms response to the FDA 483 attempted to show
statistical equivalence between the single and double
sides presses.
6

2. Warning Letter equipment comparability and process

capability
The firms written response referenced the Cpk values for processes
using a double-sided tablet press and the single-sided tablet press.
FDA evaluation of the FDA 483 response
The Cpk value alone was not an appropriate metric to demonstrate
statistical equivalence. Cpk analysis requires a normal underlying
distribution and a demonstrated state of statistical process control. The
firm did not address these issues in their response.
Statistical equivalence between the two presses could have been
shown by using either parametric or non-parametric (based on
distribution analysis) approaches and comparing means and variances.
Neither of these approaches was employed. Firm did not use the
proper analysis to support their conclusion that no significant differences
existed between the two compression processes.

2. Warning Letter equipment comparability and process

capability

Issues
Data did not support proper statistical
conclusions.
Firm did not understand underlying
assumptions required to conduct Process
Capability calculations.
Firm did not conduct proper statistical
analysis to demonstrate equivalence between
two operations.
8

3. Warning Letter container closure quality and

baseline defect rates

Product LVP in a dual chamber bag

Numerous complaints of leaks, bursts, and premature activation
during 2 year period.
Root cause - variability in the film thickness that influenced the
sealing of the bags. Bags have two seals and their strength (or
weakness) relative to each other led to different failure modes.
Critical defects compromising sterility and stability.

Poor history for the supplier of this container closure

system.
Incoming acceptance activities, as well as in-process
and finished product release activities, were found
inadequate.
9

3. Warning Letter container closure quality and

baseline defect rates

Issues
In responding to the 483, the firm equated customer
complaints to true manufacturing defect rate. They did
not understand that market incident data may not
track with the quality of the product prior to release.
The proposed sampling plans to identify these known
potential defects were not based on appropriate
statistics.
Firm did not understand sampling plan used for lot
release.
Firm could not justify using a riskier sampling plan
(higher probability of accepting bad product).
10

Application of ASTM E2709 - Standard Practice for Demonstrating Capability

to Comply with an Acceptance Procedure.
Tablets,

Q value: 70%

Background: Firm was having recall issues due to dissolution failures on

stability. Dissolution data was analyzed using ASTM 2709. Sample data
below shown for 2 lots (Each row is a different lot). If evaluated correctly,
these lots would have been flagged as high risk for failure.

Unit
6

Unit
7

Unit
8

Unit
9

Unit
10

Unit
11

Unit
12

USP PASS
or
FAIL

ASTM E2709
Probability @
95%
confidence

Unit
1

Unit
2

Unit
3

Unit
4

Unit
5

96%

72%

82%

74%

102%

70%

97%

63%

71%

78%

74%

60%

78%

14%

17%

Pass

0.14%

77%

73%

90%

95%

92%

59%

73%

94%

60%

72%

62%

85%

78%

13%

17%

Pass

0.14%

Mean

SD

RSD

11

CGMP References

211.110(a) & (b)

211.165(d)
211.180(e)
Preamble for 21 CFR 210, 211

12

Key elements in these requirements

Control procedures
Monitor the output
Performance
Variability in the characteristics of in-process
material and the drug product
.derived from previous acceptable process
average and process variability estimates
(where possible)
.determined by...suitable statistical
procedures (where appropriate)

13

Statistics
Can sample tablets at any stage of process and analyze for:

Weight.
Content Uniformity.
Dissolution.
Other critical quality attributes and or parameters of interest.

Can make decisions at any stage of process with respect to:

Ability for a lot to pass USP UDU and or Dissolution tests in the future.
(ASTM E2709)
Confidence in sampling. (ASTM E2334 & ASTM E122)
Capability and Performance analysis. (ASTM E2281)
Statistical Process Control Charts. (Monitor Variation, ASTM E2587)

Following tools illustrate making inferences about untested units on

a particular attribute, variable and or parameter with respect to
sample size and an associated confidence.
14

Voluntary Consensus Standards:

US Government Agencies
OMB Circular A119
Federal Participation in the Development and Use of
Voluntary Consensus Standards and in Conformity
Assessment Activities (Rev. Feb 10, 1998)
directs agencies to use voluntary consensus
standards in lieu of government-unique standards
except where inconsistent with law or otherwise
impractical
intended to reduce to a minimum the reliance by
agencies on government-unique standards.
http://www.whitehouse.gov/omb/circulars/a119/a119.html
15

Sample Size Effect on RSD Limit @ 95% Confidence / 95% Probability

ASTM E2709

7.00

Standard Practice for Demonstrating

Capability to Comply with an Acceptance
Procedure

n=10

5.00
RSD

n=20
4.00

n=30
n=100

3.00

n=500
2.00

n=1000

1.00
0.00
85.1

87.5

90

92.5

95

97.5

100

102.5

105

107.5

110

112.5

114.9

Sample Mean

Sample Size Effect on RSD Limit @ 99% Confidence / 95% Probability

7.00
6.00
5.00
RSD

One tool to analyze

Uniformity of Dosage
Units

6.00

n=10
n=20

4.00

n=30
n=100

3.00

n=500
2.00

n=1000

1.00
0.00
85.1

87.5

90

92.5

95

97.5

100

102.5

Sample Mean

105

107.5

110

112.5

114.9

16

ASTM E2709 Explanation

Standard Practice for Demonstrating Capability
to Comply with an Acceptance Procedure

Slide shows the relationship between sample size and

tolerance for variability. As sample size increases, so does
the tolerance for variability.
The analysis was performed using ASTM E2709-10. The
RSD limits on the y-axis represent the maximum variability a
lot can possess to ensure with 95 or 99% confidence that
there is at least a 95 or 99% probability a lot will comply with
the USP Uniformity of Dosage Units test based upon a given
sample size, confidence level, and sample mean.
For example: If you sampled 30 units and had a sample
mean of 95%, then the maximum RSD value for those 30
units would be ~3.0% to be 95% confident that there is at
least a 95% probability a future sample from the lot would
pass the USP UDU test.
17

ASTM E2334
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample

Confidence vs Sample Size

100
90
80

Confidence

70
60

Pd=1%

50

Pd=0.5%

40

Pd=0.065%

30
20
10
0
6

10

12

24

30

100

150

300

500

1000

Sample Size

18

ASTM E2334 Explanation

Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample

Slide shows the relationship between Confidence and

Sample Size. As sample size increases, so does
confidence demonstrated.
The analysis was performed using ASTM E2334-09.
Keeping the maximum percent defective constant (1,
0.5, and 0.065%) a line was generated to show how
sample size effects the confidence demonstrated in
having no more than the maximum percent defective. A
zero response was assumed (that is zero defects in the
sample) and a binomial distribution was used.
For example: If you desire a percent defective of no more than
0.5% and sample 30 units, then you are only ~15% confident
that your lot has no more than 0.5% defects.
19

ASTM E2334
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample

Sam ple Size vs Percentage of Non Conform ities

40

Percentage Non Conforming

35
30
25
20

99%
Confidence

15

95%
Confidence

10
5
0
10

20

30

50

Sam ple Size

500

1000

20

ASTM E2334 Explanation

Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample

Slide shows the relationship between the upper

confidence bound on percent defects and sample size.
As sample size increases the upper confidence bound
on percent defects decreases.
The analysis was performed using ASTM E2334-09.
Keeping the confidence level constant (95 and 99%) a
line was generated to show how sample size effects the
upper confidence bound percent defects. A zero
response was assumed (that is zero defects in the
sample) and a binomial distribution was used.
For example: If you want to be 99% confident that there is no
more than 1% defective units in your lot, then you must sample
~460 units with a zero response.
21

ASTM E122
Standard Practice for
Calculating Sample Size to Estimate, With Specified
Precision, the Average for a Characteristic of a Lot or
Process

Estimate Precision vs Sample Size

10.00
9.00

Precision Range

8.00
7.00

=1.0

6.00

=3.0

5.00

=5.0

4.00

=6.0

3.00

=10.0

2.00
1.00
0.00
10

20

30

50
Sample Size

500

1000
22

ASTM E122 explanation

Standard Practice for
Calculating Sample Size to Estimate, With Specified
Precision, the Average for a Characteristic of a Lot or
Process

Slide shows the relationship between sample

size and precision. As sample size increases,
so does your estimate precision.
The analysis was done using ASTM E122-09.
Lines were generated using different sample
sizes to show the effect it has on your estimate
precision.
For example: If you sampled 30 units and your sigma
value was 6, then your sample average is ~ +/-3.5%
of your true population average.
23

ASTM E2281
Standard Practice for
Process and Measurement Capability Indices

Confidence in Manufacturing Capability

3.50

Manufacturing Capability

3.00
2.50
2.00

95% Confidence
99% Confidence

1.50
1.00
0.50
0.00
6

10

12

20

24

30

50

Sample Size

100

250

500

1000

24

ASTM E2281 Explanation

Standard Practice for
Process and Measurement Capability Indices

Slide shows the relationship between a reported

Process Capability Index (Cpk (3.14)) and
sample size. As sample size increases, so does
the reported Cpk.
When reporting a Cpk, a lower 95 or 99%
confidence bound should always be the value
reported. As this value accounts for the sample
size in which the Cpk was estimated.

For example: If you sampled 30 units and estimated

a Cpk of 3.14, then the value reported should be ~2.5
(that is I am 99% confident that the Cpk for my
process is at least 2.5). The analysis was done using
ASTM E2281-08.
25

ASTM E2281
Standard Practice for
Process and Measurement Capability Indices

Confidence in Manufacturing Performance

3.50

Manufacturing Performance

3.00
2.50
2.00

95% Confidence
99% Confidence

1.50
1.00
0.50
0.00
6

10

12

20

24

30

50

Sample Size

100

250

500 1000

26

ASTM E2281 Explanation

Standard Practice for
Process and Measurement Capability Indices

Slide shows the relationship between a reported

Process Performance Index (Ppk (2.79)) and
sample size. As sample size increases, so does
the reported Ppk.
When reporting a Ppk, a lower 95 or 99%
confidence bound should always be the value
reported. As this value accounts for the sample
size in which the Ppk was estimated.

For example: If you sampled 30 units and estimated

a Ppk of 2.79, then the value reported should be ~2.2
(that is I am 99% confident that the Ppk for my process
is at least 2.2). The analysis was done using ASTM
E2281-08.
27

ASTM E2587

Standard Practice for Use of Control Charts in

Statistical Process Control

SPC (Statistical Process Control) Charts are a

collection of very effective statistical-graphical
tools which can be used to:

Understand and diagnose your data.

Track performance to identify problems, or shifts in
performance (good or bad).
Control or adjust the process to maintain desired
performance.

Can be applied for data based on Incoming, Inprocess, or Lot release samples.
28

ASTM E2587

Standard Practice for Use of Control Charts in Statistical Process Control

Variable X-bar-R chart

Voice of the Process

Is the process in a state of control?

Sample Mean

2 5

103

UCL=103.048

5 5

_
_
X=101.868

102

101

LCL=100.689
1

100

11

21

31

41

51
Sample

61

71

81

91

Sample Range

4.5

UCL=4.326

3.0
_
R=2.046
1.5

0.0

LCL=0
1

11

21

31

41

51
Sample

61

71

81

91

29

ASTM E2587
Standard Practice for Use of Control Charts in Statistical Process Control

Chart is used to detect special causes of

variation during manufacturing.
Control is determined against standard 8
rules established by Dr. Walter Shewhart.
Preceding chart is called X-bar-Range with
Subgroup size of 5 tablets (each point is
an average of 5 individual results).
Control limits reveal true variability of the
process.
30

ASTM E2587

Standard Practice for Use of Control Charts in Statistical Process Control

Attribute nP chart

Pass/Fail test for incoming tablet bottles (nP chart)

The tablet bottle is either pass or fail (Binary response)
Sample 100 bottles per incoming lot
12

UCL=11.38

Sa mple Count

10
8
6

__
NP=4.9

4
2
0

LCL=0
1

10

13

16

19

22

25

28

Sample
Here we assume the failure rate is 5 bottles/100 bottles
Need to know historical defect rate from supplier

31

Conclusions
What have we covered?
Enforcement action examples and CGMP references.
Use of statistics to quantify relationship between
confidence associated with attribute, variable and or
parameter of interest with respect to the sample size
collected.
To make inferences on untested units.
Can be applied to In-coming, In-process, or Finished
samples.
Can be used for real time manufacturing and or
annual/periodic product reviews.
32

Conclusions
Other Statistical Tools
Sampling plans
Do they describe Consumers Risk?
How are true defect rates calculated to use a particular
sampling plan?

Confidence, Prediction, and Tolerance Intervals

Is the correct statistical tool being applied for the

right analysis?
Do the tools help answer questions about
product quality and process performance?
33

Conclusions
The specific statistical tools and analysis
depends on what variables, attributes,
parameters are being used to monitor process
performance and product quality.
The preceding example is just one set of
statistical methods available to monitor process
performance and product quality.
Statistics is a tool to elicit information to confirm
that a specific manufacturing process is
producing quality product.
34

Acknowledgements
Alex Viehmann CDER/OPS
Grace McNally CDER/OC

35