Anda di halaman 1dari 17

Risk Management

Analysis Techniques
For Validation Programs
B Y D AV I D W. V I N C E N T & B I L L H O N E C K

Introduction different risk management tools however, this article will
In recent years, the subject of quality risk management be based on those most commonly used in the healthcare
has become a major focus of the Food and Drug Adminis- industry. The following is a list of the most commonly
tration (FDA). On April 9-11 2002, the FDA held a public used risk management tools, and a brief description of
meeting in Washington, D.C. The purpose of the meeting their practical usages:
was for the public to comment on the following three FDA
concept papers: Cause and Effect
Premarketing Risk Assessment, Risk Management Pro- Fault Tree Analysis (FTA)
grams, and Risk Assessment of Observation Data: Good Hazard Analysis and Critical Control Points
Pharmacovigliance Practices and Pharmacoepidemiologic (HACCP)
Assessment.1 Failure Modes and Effect Analysis (FMEA)
It is only matter of time before the FDA and other regu-
latory agencies will expect the same quality risk assessment Cause and Effect
to be applied to all areas of the biotechnology and pharma-
ceutical industry. Cause-and-effect diagrams were developed by Kauro
Ishikawa of Tokyo University in 1943, and thus, are often
Quality Risk Management called Ishikawa Diagrams. They are also known as fishbone
diagrams because of their appearance (in the plotted form).
Quality risk management is not a new concept. It has Cause-and-effect diagrams are used to systematically list the
been used in the medical device and other industries for different causes that can be attributed to a problem (or an ef-
many years, and is now becoming more accepted within the fect). A cause-and-effect diagram can aid in identifying the
pharmaceutical and biotechnology industries. For exam- reasons why a process goes out of control.
ple, Failure Mode and Effect Analysis (FMEA) techniques A fishbone diagram is one technique used to illustrate
have been around for over 30 years. Its only recently, how- cause-and-effect. The following is an example of a fishbone
ever, that FMEAs have gained widespread acceptance out- diagram technique:
side the safety area, thanks in large part to QS-9000.
The purpose of this article is to discuss several risk as- FISHBONE DIAGRAM TECHNIQUE
sessment techniques, and how they can be utilized to sup- 1. The diagram, like other problem solving techniques,
port the development of user requirement specifications, is a heuristic (verify) tool. As such, it helps users orga-
commissioning, and validation activities. Before a risk as- nize their thoughts and structure the quality improvement
sessment technique can be utilized in any quality assess- process. Of course, the diagram does not provide solu-
ment, it is first important to understand each technique and tions to quality problems.
how to implement them into your system. There are many 2. The final diagram does not rank causes according to

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 235
David W. Vincent & Bill Honeck

their importance. Put differently, the diagram does not sible causes of that event are considered. The analysis pro-
identify leverage points, that when manipulated, will sig- ceeds by determining how these system-level failures can be
nificantly improve the quality of the process at hand. caused by individual or combined lower level failures or
3. The diagram is a very attractive tool. On the face of it, events. The tree is continued until the subsystem at fault is de-
it is easy to learn and apply. However, it is a mistake to ap- termined. By determining the underlying causes, corrective
proach it without mastering at least some organizational actions can be identified to avoid or diminish the effects of the
learning skills, such as working together with others, seek- failures. FTA is a great lead-in to robust experimental design
ing the truth, being open to different ideas, and seeing oth- techniques. For example, the following is a top down approach
ers who might oppose you as colleagues with different to understanding a basic sterilization model (See Figure A)
ideas. Without such skills, internal politics can dominate
the process (e.g., the most powerful opinion dominates; Hazard Analysis and
team members bring to the diagram construction process a Critical Control Points (HACCP)
political agenda).
HACCP is a management system in which product
Fault Tree Analysis safety is addressed through the analysis and control of bio-
logical, chemical, and physical hazards from raw material
A Fault Tree Analysis (FTA) is a deductive, top-down production, procurement and handling, to manufacturing,
method of analyzing system design and performance. It in- distribution, and consumption of the finished product. For
volves specifying a top event to analyze (such as a sterilization successful implementation of a HACCP plan, management
process), followed by identifying all of the associated ele- must be strongly committed to the HACCP concept. A firm
ments in the system that could cause that top event to occur. commitment to HACCP by top management provides com-
FTA is a top down approach to failure mode analysis. It as- pany employees with a sense of the importance of produc-
sumes a system level failure, and identifies critical failure ing safe products. While HACCP is traditionally used in the
modes within that system. The undesirable event is defined, food industry, one can see the value of using this technique
and that event is then traced through the system to identify in determining the critical control point in the manufactur-
possible causes. One event is addressed at a time, and all pos- ing of biological or pharmaceutical drugs.

Figure A
A top down approach to understanding a basic sterilization model

Steam Sterilization Not

Achieving Correct Fo?

Steam Sterilizer Problems Steam Sterilizer Incomplete?

Temperature Steam Pressure Sterilization

Thermocouple Gauge Timer
Problem Problem Defective

236 Journal of Validation Technology

David W. Vincent & Bill Honeck

Failure Mode and Effect Analysis (FMEA)

Potential Failure Mode and Effect Analysis (FMEA) Improve productivity and yields
have recently emerged as a powerful tool for avoiding Improve product reliability through improved design
costly product performance failures. Both product/design control
FMEA and process FMEA can help you improve product Reduce waste and rework in both product design and
reliability, and reduce design and manufacturing costs. manufacturing processes
FMEA is a bottom up approach to failure mode analysis. It Improve manufacturing process reliability and reduce
is an inductive process, which is performed after the de- process variability
tailed design phase of the project. It is used to evaluate a de- Change the focus of your engineering and manufactur-
sign or process for possible failure modes. FMEA consid- ing efforts from putting out fires to improving product
ers each mode of failure of every component of a system, design and manufacturing
and ascertains the effects on system operation of each fail- Reduce design and manufacturing costs
ure mode in turn. The failure effects may be considered at Predict potential, unavoidable design and manufactur-
more than one level. For example, the subsystem and the ing problems, and implement corrective actions
overall system may be considered. This technique helps Reduce product development and design cycle times
eliminate poor design and process features. FMEA is com- Comply with FDA GMP regulations
plementary to each stage of a product design, development,
and validation. The next section of this article will describe how to im-
HHS Publication FDA 90-4236 states, Failure mode plement FMEA methodology into your risk assessment
effect analysis (FMEA) should be conducted at the begin- program
ning of the design effort and as part of each design review
to identify potential design weaknesses. The primary pur- How to Implement a FMEA Methodology
pose of FMEA is the early identification of potential design into Your Risk Assessment Program
inadequacies that may adversely affect safety and perfor-
mance. Every company is unique therefore, it will require dif-
FMEA is a method to evaluate possible ways failures ferent methods when developing and implementing a risk
can occur in a design, process, system, or service. FMEA management program. This section of the article will focus
uses teams to improve the design, product, process, on developing a risk assessment program from the early
and/or service. stages of process development, developing a user require-
A second tool, process FMEA, helps in analyzing man- ment specification for equipment and utilities, new facility
ufacturing processes and identifying the processes that are construction and commission, and into manufacturing
important for manufacturing a trouble-free, functioning process and routine process monitoring. While there are
product. The intent is to identify and correct known or po- many different risk assessment methods, this section of the
tential failure modes that can occur during process devel- article will be dedicated to developing and implementing
opment. It has the greatest impact in the early stages of FMEA and developing User Requirement Specifications
process design, before any machines, tools, or facilities are (URS).
purchased. This level of analysis is completed before set-
ting the final product design and, therefore, before produc- Early Stage Process Development
tion begins. After identifying the critical processes, correc- Using Design FMEA
tive actions and appropriate process controls, such as Sta-
tistical Process Control (SPC), are evaluated and imple- A FMEA is systematic method of identifying and pre-
mented. Process FMEA can also be used to define impor- venting product and process problems before they occur.
tant process variables (as compared to product variables) to FMEAs are focused on preventing defects and contamina-
maximize the application success of SPC. Process FMEAs tion problems, enhancing safety, and increasing customer
should be living documents. satisfaction. Usually, FMEAs are conducted in the product
FEMA cause and effect should be used during process design or process development stages, however, FEMAs
validation to determine worst-case conditions. can be performed on existing processes and products.

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 237
David W. Vincent & Bill Honeck

Review the Process

Design FMEA
Design FMEA is used to analyze product design before To ensure that everyone on the FMEA team has the same
they are released to manufacturing. A design FMEA fo- understanding of the process that is being worked on, the
cuses on failure modes caused by design deficiencies. team should review all the necessary documentation relating
Process FMEA to the product. If the team is conducting a process FMEA, a
Process FMEA is used to analyze manufacturing detailed flowchart of the operation should be reviewed.
processes. A process FMEA focuses on failure modes The documentation will assist the FMEA in understand-
caused by deficiencies or potential problems with the ing all the requirements and issues related to the process
actual process. and/or product.

FMEA PROCESS STEPS Set up Initial Brainstorming Meeting

The following are general steps that should be followed:
Once the team has an understanding of the process (or
1. Select the team. product), team members can begin planning for potential
2. Review the process failure modes that could affect the manufacturing process
3. Set up initial brainstorming meeting. or the product quality. A brainstorming session will uncover
4. Construct a FMEA worksheet. (See Figure 1) all ideas. Team members should come to the brainstorming
5. Prioritize the steps, define the definition of key terms, meeting with a list of their ideas regarding the process or
and agree on the criteria for ranking severity, occur- product.
rence, and detection. Most manufactured products and manufacturing
6. Investigate and evaluate available information. processes are complex, therefore, it is best to conduct a se-
7. Team approved action(s). Assign appropriate personnel ries of brainstorming sessions, each focused on a different
to each corrective action. element (for example; process, personnel, methods, proce-
8. Complete and document corrective actions, as required. dures, components, equipment, materials, and the environ-
ment) of the product or process.
Selecting the Team Once the brainstorming is complete, the ideas should be
organized by category. The team must decide the best cat-
It is important to select team members who have differ- egories for each group, as there are many different ways to
ent familiarity with the product and process. Team mem- form group failure modes. The type of failures e.g., materi-
bers must understand customer expectations and be knowl- als, mechanical, and personnel can be used to categorize a
edgeable of the system, its controls. The success of FMEA grouping. The type of failure can indicate the stage of the
in selecting a team leader who will be responsible for set- product or the process point at which the failure is most se-
ting up meetings, ensuring that the team has resources to rious. Grouping the failures will make the FMEA process
support FMEA, and managing to the successful completion easier to analyze.
of the FMEA. Depending on whether the FMEA is design For accuracy and completeness, it is important to record
or process, different team members will be required. Rep- all decisions made during the brainstorming meeting. The
resentatives from the following list should be chosen de- development of standardized assessment forms to capturing
pendent upon the nature of the project. valuable information is one tool that is very effective in ac-
Research & Development cumulating information.
Regulatory Affairs Failure Modes
Quality Control A failure mode is a physical description of a defect, con-
Quality Assurance dition, or characteristic related to the specific function,
Project Management component or operation identified. List the potential effects
of each failure mode. There may be several of these for
each item identified in the list below:

238 Journal of Validation Technology

Steam Sterilization Not
Achieving Correct Fo?

David W. Vincent & Bill Honeck

Figure 1
Figure 1
FMEA Team Start-Up Worksheet
Start-Up Worksheet

FMEA Number: ____________________________ Date Started:________________________________

Date Completed:____________________________

Team Members:_______________________________ _______________________________

Team Leader:____________________________________________

1. Are all affected areas represented?

YES NO Action: ___________________________________________________________________

2. Are different levels and types of knowledge represented on the team~

YES NO Action: ___________________________________________________________________

3. Is the customer involved?

YES NO Action: ___________________________________________________________________

4. Who will take minutes and maintain records? _______________________________________________________

FMEA Team Boundaries of Freedom

5. What aspects of the FMEA is the team responsible for?

(FMEA Analysis ) (Recommendations for Improvement) (Implementation for Improvement)

6. What is the budget for the FMEA? __________________________________

7. Does the project have a deadline?__________________________________

8. Do team members have specific time constraints?__________________________________________________

9. What is the procedure if the team needs to expand beyond these boundaries?

10. How should the FMEA be communicated to others?_________________________________

11. What is the scope of the FMEA?

Be specific and include a clear definition of the process or product to be studied:

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 239
David W. Vincent & Bill Honeck

START WITH KNOWN FAILURE MODES: more than one cause for each failure mode.
Customer complaints Design FMEAs
Process control reports The focus is specifically on design weaknesses and de-
Validation failures ficiencies, or possible customer use/misuse situations
Test results that could lead to the failure.
Product quality data Process FMEAs
The focus is on process aspects, controls, variables, or
Potential Effects (System and End User) conditions that can result in the failure.

Effects are any conditions that can occur in the early Occurrence
process development phase, clinical setting, and/or manu-
facturing conditions, potentially brought about by a failure Occurrence is the probability that the cause listed will
mode, if it were present in the product used by the cus- happen and create the failure mode described. Historical data
tomer. In the case of process FMEAs, also include potential on this or similar designs/processes may be used to estimate
effects on subsequent operations in the manufacturing how often an occurrence will transpire. The probability of oc-
process. There may be several effects for each failure mode. currence may be defined on a scale from one to five. There is
an occurrence rank for each cause identified. (See Figure 4)
Assigning Severity, Occurrence,
and Detection Ratings Detection
In most FMEA, the rating is based on a 10-point scale, Detection ranking is specific to Current Controls. A
with one (1) being lowest and ten (10) being highest. Fig- ranking score of one is assigned to represent the combined
ure 2 is an example of a typical ranking system for Sever- impact of all controls identified for a given cause. If there are
ity, Occurrence, and Detection. no controls for a cause, assign a high rank (5) in the detection
It is important to establish clear and concise descriptions column for that cause.
for the points on each of the scales, so that all team members Design FMEAs
have the same understanding of the ratings. The scales Detection is based on the ability that routine testing and
should be established before the team begins the ranking of inspection will detect the failure or cause of the failure
the FMEA. prior to manufacturing.
In a typical ranking system, each of three ratings (sever- Process FMEAs
ity, occurrence, and detection) is based on a five-point Detection is based on the probability that the process
scale, with one (1) being the lowest rating and five (5) being controls/inspections identified will prevent or remove the
the highest. This ranking method was selected because it cause prior to manufacturing or customer use.
best suited the process analysis.
Risk Priority Number (RPN)
The Risk Priority Number, (RPN), is a measure of the
Severity ranking is an assessment of the seriousness of overall risk associated with the failure mode. The RPN is
the effect, assuming the affected product is actually being obtained by multiplying the rating for severity, occurrence,
used. This is depicted using a numbering scheme. The and detection. It will be a number between 1 and 125. The
Severity is estimated on a scale of one through five. Figure higher the number, the more serious the failure mode will
3 may be used as a reference for scaling. There will be a be. Each failure mode may have several RPNs, because
severity rank for each effect identified. there may be multiple effects (i.e., severity, occurrence, and
detection ranks) and, therefore, several combinations of
Potential Causes of Failure those numbers.

For each failure mode, list all the possible mechanisms Severity x Occurrence x Detection = RPN
and causes that could bring about the failure. There may be 5 x 5 x 5 = 125

240 Journal of Validation Technology

David W. Vincent & Bill Honeck

Figure 2?
Figure 2
Figure 2?
2? ranking
A typical SeverityOccurrence, and Detection.
system for Severity, Occurrence Detection
10 Dangerously High Very High: Failure is almost Absolute Uncertainty
Rating Severity
Severity Occurrence
inevitable Detection
High High VeryRemote: Failure
Very High: is isalmost
Failure unlikely Almost
is almostAbsolute Certainty
Absolute Uncertainty
10 1 10 Dangerously High: Failure Uncertainty
1 1 None None Remote:
Remote: FailureFailure is unlikelyAlmostAlmost
is unlikely Certainty

Figure 3?
Figure 3
A severity Figure
Figure 3?
Leveleffect identified.
for each
3? Description Ranking
Very High Any failure that could reasonably result in a 5
Severity LevelLevel Description
Description issue (potential harm to worker or customer)Ranking Ranking
Very High
Very High Anyand/or
failure may
that result
Any failure could in
that could a regulatory
reasonably reasonably issue.
result result
in a in a 5 5
High Major
safetysafety failure
issue issue that may
(potential harm torender
harm the
worker system
to worker or customer)
or customer) 4
and/or and/or may
may result or result
result in
in significant
a regulatory
in a regulatory issue.reduction
issue. in
High High performance
Major Major
failurefailure or
that may quality
thatrender of
may render the product
the system the system 4 4
Moderate Moderate
inoperable or failure
inoperable result likely
or result
in resulting
significantsignificant in reduction
reduction in inin
reduction 3
performance or orquality
or quality quality
of the of of
the the
Moderate failures
Moderate Moderate are noticeable
failurefailure to the
likely resulting
likely resulting end user, and
in arein
in reduction
in reduction 3 3
likely to
performance generate a moderate
or quality
or quality of the of level
the product.
product. of
failures failures
are noticeable or complaints
are noticeable
to the to end thefromendthe
user, customer.
and and are
Low Minor
likely tolikely failure,
generate not
to generate noticeably
a moderatea moderate affecting
level of level of functional 2
dissatisfaction however,
dissatisfaction may
or complaints generate
complaintsfrom the complaints
from due
the customer.
Low Low Minorto Minor
failure,failure, For
not example,
not noticeablynoticeably
affecting defects
functional 2 2
quality, increased
however, maintenance.
however, may generate
may generate complaints complaints
due due
None Minor
to annoyance. failure,
to annoyance. unlikely
For example, to be
For example, noticed
cosmetic by customers
defects defects 1
or generate
and increased
and increased complaints.
None None Minor Minor
unlikelyunlikely to be noticed
to be noticed by customers
by customers 1 1
or generate
or generate complaints.

Figure 4
An occurrence rank for each cause identified.
Figure4? of Failure
Figure4? Description Ranking

Very high
Probability of Failure
of Failure Failures occur regularly and one could reasonablyRanking
Description 5
expect the failure to occur for each component or
Very high
Very high Failures
Failures eachoccur
occur regularly
regularly andcould
and one could reasonably 5
reasonably 5
High expect
expect the failure
the failure onoccurto occur
a frequent for
for each each component
These failures
component or or 4
during during
do not
each each
occur process
process step.
time, however, they do occur at
High High Failures
a rate
Failures occur
to produce
occur on a frequent
on a frequent basis.
basis. ThesetoThese failures 4
the product
failures 4
do not do
occurandoccur every
every time, however,
time, however, they do they do occur
occur at at
Moderate a rate
a rate to produce significant
to produce only occasionally, concern
concern however, to the product
at a rate
to the product 3
does and performance.
not significantly impact production, but
Moderate canFailures
Failures be occur
a nuisance.
occur only occasionally,
only occasionally, however, however, at a rate 3
at a rate 3
Low that not
does does
significantly impactimpact
These failure production,
rates but
production, 2
canfew can be a
be aproduction nuisance.
LowRemote Failures
A failure
Failures occurof occur rarely.
the component
rarely. failure
or system
failure rates
rates iscreate create
extremely 2 1 2
few unlikely.production
production problems. problems.
RemoteRemote A failure
A failure of the of the component
component or system
or system is extremely 1
is extremely 1
unlikely. 241
M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3
David W. Vincent & Bill Honeck

Figure 5
A detection 5?
Figure rank for each failure identified.

Probability of Failure Description Ranking

None No detection methods in place to prevent the 5

release and use of the product or process.
Detection of problems are not feasible.
Low No intentional inspection techniques in place; 4
however, the failure is such that it may render the
product useless upon receipt.
Moderate Indirect inspection techniques on various aspects 3
of the product/process, such that an abnormality
could reasonably result in a true failure occurring.
High Statistical sampling inspection techniques in place 2
to directly look for the failure.
Very high 100% inspection techniques in place to directly 1
look for the failure

Prioritize the Failure Modes for Action above 50 creates an unacceptable risk.
As a general guideline, RPN numbers with a severity of Once a corrective action is determined by the teams, its
three (3) or greater, and an overall RPN of 50 or greater, important to assign an individual or group to implement the
should be considered as potentially critical, and actions required action. Selection should be based on experience
should be taken to reduce the RPN. However, this threshold and expertise to perform the corrective action. Its also im-
number may vary from process-to-process, and the project portant to assign a target completion date for the action item.
team must make the final decision. This will help in insuring the timely close of any problem.
Pareto analysis can be applied. The top 20% of the
ranked RPN numbers should account for approximately Reassessing the Risk Mode
80% of the anticipated frequent failure modes. These 20% after CorrectiveAction
should be a top priority in corrective action.
Once action has been taken to improve the product or
RECOMMENDED ACTIONS process, a new rating for severity, occurrence, and detection
To reduce severity: change design or application/use. should be determined, and the resulting RPN calculated.
To reduce occurrence: change process and/or product design. For failures modes where action was taken, there should be
To improve detection: Improve controls as a temporary significant reduction in the RPN. If not, that means the ac-
measure. Emphasis should be on prevention e.g., de- tion did not reduce the severity, occurrence, and detectabil-
velop controls with alarms. ity. The final RPN can be organized in a Pareto diagram and
compared with the original. You should expect at least a
By ranking problems in order, from the highest risk pri- 50% or greater reduction in the total RPN after the FMEA.
ority number to the lowest, you can prioritize the failure After the action has been implemented, the severity, oc-
modes. A Pareto diagram is helpful to visualize the differ- currence, and detection ratings for the targeted failure
ences between the various ratings, and to assist in the rank- modes are re-evaluated. If the resulting RPN is satisfactory,
ing process. The FMEA team must now decide which items you can move onto other failure modes. If not, you may
to work on first. Usually, it helps to set a cut-off RPN, wish to recommend further corrective action.
where any failure modes with an RPN above that point are Use the example of a typical FMEA worksheet that ap-
attended to. pears on the prior page.
Those below the cut-off are left alone for the time being. The FMEA risk assessment method listed above is just
For example, an organization may decide that any RPN one example of how implementing a risk management tool

242 Journal of Validation Technology

David W. Vincent & Bill Honeck

Description of FMEA Worksheet
System _________________________ FMEA Number ___________________

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3
Subsystem _________________________ Potential Prepared By ___________________
Component _________________________ Failure Mode and Effects Analysis FMEA Date ___________________
(Design FMEA)
Key Revision Date ___________________
Team Lead _________________________ Date _________________________ Page _______of__________
Core Team _________________________________________________________________
Action Results
Potential Potential Responsibility

Potential Current Recommended

New Occ
New Sev
Effect(s) Cause(s)/

New Det
& Target



Item / Design
Failure of Mechanism(s) Action(s) Completion Actions
Function Controls
Mode(s) Failure of Failure Date Taken
Coolant Crack/break. Leak 3 Over pressure 3 Burst, 2 18 Test included John Scientist Install 3 1 1 3
containment Burst. validation in prototype 2/27/04 durable
in Product. Bad seal. pressure and Jim Engineer hose
Poor hose Poor hose cycle. production 5/1/04 material
connection material validation with
testing. pressure
to prevent
Response Plans and Tracking pressure.
Occurance - Write down the new design
Write down each failure mode potential cause(s), and on a
Risk Priority Number - The
and potential consequence(s) scale of 1-5, rate the likelihood
combined weighting of Severity,
of that failure. of each failure (5= most likely).
Occurance, and Detectability.
RPN = Sev X Occ X Det
Severity - On a scale of 1-5 Detectability - Examine the
Corrective Action
rate the Severity of each current design, then, on a scale
Implemented New
failure (5= most severe). of 1-5, rate the Detectability of
RPN assigned
each failure(5= least detectable).
See Detectability sheet.
David W. Vincent & Bill Honeck

can decrease the potential for quality problems. The next ceived, installed, commissioned, and validated. However,
topic will cover establishing risk management for systems before URSs and protocols can be developed, a component
that require validation. impact assessment should be performed on that system.
In order to decrease the cost and potential delays in a pro-
User Requirement Specification Procedure ject, Good Engineering Practices (GEP) should be imple-
Getting Started mented. The ISPE Baseline Commissioning and Qualifica-
tion guideline defines Good Engineering Practice as follows:
This section of the article will describe how to develop Established engineering methods and standards that
a URS system for direct and indirect impact systems. How- are applied throughout the project life cycle to deliver ap-
ever, before a detail URS document can be developed, a propriate cost-effective solutions
system impact assessment process should be performed for The proper design and selection of system can be criti-
each system. Figure 6 is a brief overview of how to perform cal to any manufacturing operations. By implementing
an equipment impact assessment. GEP, the risk of problems occurring during the design and
selection can be decreased substantially.
Equipment Impact Assessment Direct Impact systems are expected to have an impact
Figure ? on product quality, whereas, indirect impact systems are not
An equipment impact assessment should be performed expected to have an impact on product quality. Both systems
on any system or equipment before they are purchased, re- will require commissioning; however, the Direct Impact

Identify System Figure 6

Steps in performing an equipment impact assessment

Develop System

Does the System

have a direct impact on YES
product quality?


Is the System "Indirect Impact" "Direct Impact"

linked to a Direct Impact YES
system system


"No Impact" Develop Supporting

system Rationale

244 Journal of Validation Technology

David W. Vincent & Bill Honeck

system will be subject to qualification practices to meet ad- Complete the impact assessment challenge table (See
ditional regulatory requirements of the FDA and other regu- next page). Use the seven listed challenges to evaluate the
latory authorities. system and place an X in the appropriate Yes or No
See Figure 6 for an outline of the impact assessment block.
process based on the ISPEs Commissioning and Qualifica- Classify the system as Direct Impact, Indirect Im-
tion guidelines. pact, or No Impact on the system classification line.

Benefits Impact Assessment If the response to any of challenges numbers one

through six is Yes, the system shall be classified as a
Execution of an impact assessment accomplishes the Direct Impact system.
following: If the response to challenges numbers one through six is
Provides classification of a system as either Direct Im- No, but the response to challenge 7 is Yes, the sys-
pact, Indirect Impact, or No Impact based on the systems tem shall be classified as an Indirect Impact system.
effect on product quality If the response to challenges numbers one through seven
Identifies systems that, if improperly, designed or in- is No, the system shall be classified as a No Impact
stalled, could present a risk to product quality system.

System Impact Procedure Complete the system classification rationale section with
a brief explanation as to why the classification was assigned.
You must first identify the system and enter system This is to ensure understanding by subsequent reviewers and
name and system number, on the system impact assessment approvers as to why the classification was chosen.
Table 1. This information can usually be obtained from the Attach the P & IDs to the system impact assessment
P & ID drawings or other system documentation. table, fill in the page numbers, and fill in the prepared by
Complete the system description section with a general and date fields.
narrative of the system and its major components, de- Impact Table 1 is a system impact assessment for nitro-
sign, operation, functional capabilities, and critical func- gen air distribution system used in the operations of manu-
tions. facturing process.
Mark on the system P & ID drawing(s) to clearly iden-
tify the system boundaries and all components of the Component Criticality Assessment Process
system included within the boundary. After you have established that a system is direct or in-
Specify system boundaries by inserting a horizontal or direct, you then perform a component impact assessment.
vertical line at the boundary. These lines should be However, this is usually performed after the impact assess-
placed to clearly identify whether or not the adjacent ment is performed, and URSs have been developed. The
component is part of the system. component criticality assessment process requires the Pip-
To help in establishing the system boundary, utilize the ing and Instrument Drawings (P&IDs) and system instru-
following general guidelines (there may be exceptions to ment list be reviewed in detail.
these guidelines): The components within Direct Impact, Indirect Im-
pact, and in some cases No Impact systems should be
If the component number of a valve etc. is labeled as assessed for criticality. This is suggested to ensure that sys-
part of the main system being assessed, then it generally tems previously judged to be Indirect Impact or No Im-
will be part of that system. pact in the early, high-level assessment, have not subse-
The control system I/O for a given system will become quently acquired a critical function, as the detailed design
part of that system. has progressed to conclusion.
Disposable flexible piping connectors and/or portable
tanks etc. should not be highlighted as part of the sys-
tem, and should be noted either on the drawing or in the
comments section of the form, so it is clear that they
were not highlighted on purpose.

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 245
David W. Vincent & Bill Honeck

Applicability of any of the following listed criteria to a 6) The component controls critical process elements that
given component will provide an indication that the com- may affect product quality without independent verifi-
ponent is critical: cation of the control system performance.
7) The component is used to create or preserve a critical
1) The component is used to demonstrate compliance with status of a system
the registered process
2) The normal operation or control of the component has a Evaluation of each criticality of components within
direct effect on product quality each system with respect to their role will assure product
3) Failure or alarm of the component will have a direct ef- quality.
fect on product quality or efficacy After the impact assessments have been performed, the
4) Information from this component is recorded as part of qualification phase of the systems can be performed. The
the batch record, lot release data, or other GMP-related use of risk assessment methods, as described above, can as-
documentation sist in developing validation protocols that are logically de-
5) The component has direct contact with product or prod- signed to insure proper qualification of a system.
uct components

Table 1
Impact Assessment Challenge Table

Impact Challenge Table Yes No

1. Does the system have direct contact with the product (e.g., air quality) X
or direct contact with a product contact surface (e.g., CIP solution)?
2. Does the system provide an excipient, or produce an ingredient or X
solvent (e.g., water for injection)?
3. Is the system used in cleaning, sanitizing, or sterilizing X
(e.g., Clean Steam)?
4. Does the system preserve product status X
(e.g., Nitrogen purge for air sensitive products)?
5. Does the system produce data that is used to accept or reject product X
(e.g., electronic batch record system, critical process parameter chart
recorder, or release laboratory instrument)?
6. Is the system a process control system (e.g., PLC, DCS) or contain X
a process control system that may affect the product quality, and
there is no system for independent verification of control system
performance in place?
7. Is the system not expected to have a direct impact on product quality, X
but does support a Direct Impact System?
System Classification: (Direct Impact, Indirect Impact, or No Impact): This system was
defined as Direct Impact because it meets the requirements based on the above
risk assessment criteria.
System Classification Rationale: The function of the nitrogen air distribution system is
to provide a continuous overlay of product. Since nitrogen air does impact status, the system
impact is considered Direct. The problem with nitrogen quality is that it will have a direct
impact on product quality.

246 Journal of Validation Technology

David W. Vincent & Bill Honeck

Applying Risk Management

probably raise the level of success when qualifying a sys-
When Developing User Requirement
tem. The URS procedure should have forms, which must be
Specifications for Systems completed by the end user, and reviewed and approved by
Based on my experience, most companies have not yet the various functional groups. The URS is usually submit-
developed a risk assessment system for their validation pro- ted to the purchasing department as part of the purchasing
gram. They normally rely on industry standards or previous specifications. The following is an example of a method for
experience to determine which type of qualification proto- developing a URS with a validation risk assessment.
cols (Installation, Operational and Performance) need to be
developed. Most often, they have either developed an ultra
conservative or minimalist approach in assessing the types The use of risk assessment
of qualification protocols required for a system. Each ap-
proach can be very costly and time consuming for a com-
methods can assist in
pany. If you take an ultra conservative approach, it will add developing validation
additional cost and time to the project. By taking a mini-
malist approach, it could also be very costly and/or time
protocols that are logically
consuming, especially if an inspector notes GMP deficien- designed to insure proper
cies in the validation program.
For years, equipment qualification was an activity that
qualification of a system.
was addressed, if at all, only after equipment was designed,
purchased, and installed. Companies view the generation, Equipment Description Section
execution, and detail of this documentation as a black
box; its something they need for compliance, but do not The equipment description section of the URS is used to
fully understand. The development of User Requirement describe design specifications. The following is a brief
Specifications (URS) is usually one of the most critical el- overview of that section:
ements in the compliance documentation process.
What is a URS? A URS is a detailed document used to Description: Briefly describe the equipment type, size,
specify the requirements of the user for individual aspects capacity, etc. Include manufacturer and model, if
of the facility, equipment, utility, and system, in terms of known.
function, throughput, operation, documents, and applicable Location: Indicate where the equipment will be installed
qualifications need. and/or used. Include room number, floor footprint, lab
How do you develop an appropriate URS? The follow- bench, etc., as applicable.
ing is an example of how to develop an acceptable URS, Contact Person: The end user or other individual who
and applying a risk assessment to determine the type of will coordinate the commissioning process.
qualification needed for a particular piece of equipment User Department: The department that is responsible for
using a ranking tool. the equipment.
The URS documents the design qualification and ratio- Other Affected Departments: Departments whose activi-
nale equipment selection. The URS describes critical in- ties will be affected by operation of the equipment, or
stallation and operating parameters and performance stan- who will be involved in the selection, installation, opera-
dards that are required for the intended use of the equip- tion, qualification, and/or maintenance of the equipment.
ment, and provides the basis for qualification and mainte- Intended Use: Describe the intended use of the equip-
nance of the equipment. The URS should be prepared by ment in relation to cGMP operations or processes.
the equipment owner in collaboration with representatives Functional Requirements: Describe critical functions
of departments that will participate in qualification and that the equipment must perform to support the intended
maintenance of the equipment, and departments that will be use.
affected by the operation of the equipment. Calibration Requirements: Describe calibration specifi-
In order for a URS system to be successful, you will cations and schedules for instrumentation and controls
need to develop a procedure that describes in detail the associated with the equipment.
function of the URS. While a URS is not necessary, it will Maintenance Requirements: Describe preventive main-

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 247
David W. Vincent & Bill Honeck

Table 2
Component Impact Assessment

A. Quality Impact Score

No impact: Equipment will not be directly or indirectly associated with 0

cGMP activity.
Minimal impact: Equipment indirectly affects cGMP processes or procedures. 1
(Non Direct Product Impact)
Potential Impact: Equipment performs or directly supports a cGMP process 2
or procedure; failure could potentially affect product quality. Equipment failure
could negatively impact operational efficiency or costs. (Indirect Product Impact)
Direct Impact: Equipment is an essential component of a cGMP process or 3
procedure, or is in direct contact with drug substance or drug product.
Equipment failure could result in loss of product; safety hazard; damage to
materials; equipment or facility; or negative inspection findings. (Direct
Product Impact)
B. Quality Risk Management Score

No risk control necessary. 0

Failure of the equipment would be detected immediately and be corrected 1
before affecting a cGMP process or procedure.
Failure could not go undetected. Systems and procedures are in place to detect 2
negative impact on product quality safety or purity before a significant loss
of productivity.
Failure could potentially go undetected and cause failure of other processes 3
or procedures.
C. Technology Risk Score

Very simple system; minimal chance of failure. 0

Commonly understood technology, rugged equipment; low probability of failure. 1
Somewhat complex equipment, generally reliable technology, components 2
and/or controls.
Highly complex and/or sensitive equipment, sophisticated technology, unique 3
components or processes.

D. Technology Risk Management Score

Control and repair possible without impacting cGMP activities. 0

Equipment requires minimal training, simple maintenance procedures;
back-up, repair, or like-for-like replacement is readily available. 1
Requires trained operators and maintenance technicians. Backup systems,
repair, maintenance, and replacement are readily available. 2
Operators and maintenance technicians must be highly trained. Maintenance,
repair, or replacement requires specialized and/or time-consuming effort. 3
Backup systems, repair, maintenance, and/or replacement are not readily

248 Journal of Validation Technology

David W. Vincent & Bill Honeck

tenance tasks and schedule. Identify any additional by the company or vendors. Include manuals, factory
maintenance requirements to ensure that the equipment acceptance test, site acceptance, commissioning docu-
continues to operate as required. ments material construction, parts lists, drawings, gov-
Requalification Requirements: Describe requalification ernment inspections, certificates, SOPs, etc.
requirements to ensure that the equipment remains in a Training: Indicate training requirements for operators
validated state. and maintenance personnel. Identify any special certifi-
cations, educational, or physical requirements, for oper-
System Requirements Definition Section ation or maintenance of the equipment.

Identify the specific attributes that are necessary for the Systematic Risk Assessment for
equipment to satisfy the requirements for the equipments System Qualifications
intended use. Provide acceptance criteria and acceptable
ranges that can be verified to document that the equipment The risk assessment section discusses the potential im-
is appropriate for its use, and capable of functioning reliably, pact on cGMP operations associated with use of the equip-
as required. This section provides the basis for qualification ment, and the steps that will be taken to reduce those risks.
protocols, and for ongoing maintenance and calibration pro- Identify conditions that could lead to failure of the equip-
cedures. List only those characteristics that will provide spe- ment, and the effects of failure on cGMP operations. Evalu-
cific evidence relevant to the equipments intended use. In- ate the degree of risk to product quality, company opera-
clude the following requirements, as appropriate: tions, and safety of personnel and equipment. During the
Procurement: Identify any special shipping, delivery, risk assessment, its important to perform an impact assess-
preliminary testing, certification, or other requirements ment on the system. Impact assessment is the process by
for acquisition of the equipment, as necessary. which the impact of the system on product quality is evalu-
Installation: Identify requirements for installation, oper- ated, and the critical components within those systems. The
ating environment, and support utilities. Indicate any risk assessment for systems should fail within three cate-
qualification testing and/or documentation required for gories: direct product impact, in-direct product impact, and
utilities or peripheral equipment prior to installation of no direct product impact.
the subject equipment. By performing a design impact assessment, companies
Operation: List the critical operating parameters and can reduce the scope of the systems and component subject
ranges, capacity requirements, etc., that are required for to qualification, and allowing appropriate focus to be placed
the intended function. Do not include measures that do on the components that may present a potential risk to the
not affect the required functionality of the equipment. product.
Performance: Identify measurable product or results The following is one example of how applying risk as-
that are required when operating the equipment under sessment for a validatable system can be beneficial in de-
expected conditions. Include operating limits and veloping a scientific rationale, and justification for selection
ranges, and worst-case scenarios that may be encoun- of the different types of qualification needed to support a
tered during normal use. system. Summarize risks and associated controls in an im-
Safety Features & Controls: Identify safety features and pact/complexity analysis, as follows:
controls that the equipment and installation must supply.
Instrumentation, Operating Controls, and Peripherals: Impact Analysis: Rate the impact of the equipment on
Identify the required instrumentation, control compo- product quality, safety and purity, and on safety of per-
nents and peripheral equipment that monitor and control sonnel and equipment. Evaluate the systems in place to
the equipment. Provide necessary operating ranges, sen- control those risks.
sitivity, and calibration requirements.
Consumables: Identify consumables required for opera- Complexity Analysis: Describes the technological risks
tion of the equipment. Identify whether supplied by and controls associated with the equipment. The com-
manufacturer or user. plexity analysis evaluates the risk of failure due to techni-
Documentation: List the documentation that will be cal sophistication of the equipment, and the relative diffi-
supplied with the equipment, and that must be created culty of maintaining the equipment in a state of control.

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 249
David W. Vincent & Bill Honeck

Table 3
Figure ? Requirements

Risk Score Qualification Requirements Validation Maintenance Requirements

0 Document installation Documentation maintained by Users or
and commissioning Facilities Department.
1 to 3 IQ Installation, commissioning, maintenance, and
change control documentation maintained by QA.
4 to 6 IQ/OQ Operate, maintain, and calibrate according to
written SOPs.
Document preventive and corrective maintenance
and calibration according to SOPs
Apply change control procedures according to SOPs
and change control programs.
7 IQ/OQ/PQ Perform operation, maintenance, calibration, and
performance verification procedures according to
written procedures.
Document preventive and corrective maintenance
and calibration according to SOPs.
Apply change control procedures according to
SOPs and change control programs.

Risk Score: This section is a calculation used to evalu- and time in the long run. Most project cost overruns and de-
ate the overall risk of the equipment by combining the lays have been contributed to not performing Good Engi-
individual impact and complexity scores in the follow- neering Practices and Risk Assessment at the beginning of
ing formula: project. Also, implementing a risk assessment program
within firms Quality Function will insure that the final
(A + B) x (C + D) product quality will be achieved.

Where: A = Quality Impact

B = Quality Risk Management About the Author
C = Technology Risk
D = Technology Risk Management David W. Vincent has over 24 years experience in the
health care industry with 15 years in field of validation.
Validation Requirements He has BS degree in Industrial Microbiology and Me-
chanical Engineering Technology degree; he has con-
Identify the qualification requirements for the equipment sulted for many companies, national and international.
based on the impact/complexity analysis as shown in the fol- Mr. Vincent has expertise in many areas of Quality As-
lowing table. For smaller, less complex system qualifica- surance, Regulatory Affairs, and Validation, including
tions, protocols can be combined into I/OQ or IQ/OQ/PQ BLA submission preparation, facility and equipment
protocols. Any additional information to support and justify design review, process development and validation,
the validation requirements should be included. project management, and utility and process equip-
ment qualification. He has been involved in the vari-
Conclusion ous aspects of bringing many new drug manufactur-
ing facilities on-line, from design concept and engi-
The implementation of a risk assessment program within neering, through construction and start-up, to the
a firm can decrease the cost and time it takes to perform a qualification/validation, and licensing phases. He has
system qualification. Spending the time upfront performing presented many training seminars and written many
a risk assessment will save a company a great deal of cost articles regarding validation topics. He teaches Vali-

250 Journal of Validation Technology

Statistics for
David W. Vincent & Bill Honeck

dation Program for Pharmaceutical, Biotechnology

and Medical Device Industries RA 776 at San Diego
State University (SDSU) for their Regulator Affairs
Master Degree program. Currently, he is the CEO of
Validation Technologies, Inc. a nationwide Validation
Services Company.

William Honeck is the Senior Validation Manager at

Scios, Inc. He has over 13 years experience in the
Video Series
biopharmaceutical industry in the areas of quality 5 Volumes Approximately 3 hours
assurance, quality control, method development
and validation, equipment and facility validation, This series of videotapes reviews the basics of sev-
and computer validation. William holds a B.S. de- eral statistical techniques utilized for conducting ef-
gree in Biochemistry from the University of Califor- fective validations. These subjects include:
nia at Davis. He can be reached by phone at 510- Quantifying and calculating measurement error
248-2760, by fax at 510-248-2454, or by e-mail at Bias The views expressed in this Rules for a machine capability study
article are strictly those of the authors and are not Definitions of Cp and Cpk
intended to represent the perspectives of Scios Inc., Normality
Johnson & Johnson, or this publication. Installation Qualification (IQ), Operational Quali-
fication (OQ), and Performance Qualification
Calculating sample sizes utilizing power, alpha,
Reference beta risks, and binomials
Variable samples
1. Journal of GXP Compliance, FDA Conference Report
Design of experiments viewers are taken step-by-
Public Meeting: Risk Assessment Management and Phar-
step through an actual designed experiment. The re-
macovigilance, Volume 7 Number 4, July 2003, Warren
sults of the experiments are reviewed and discussed
Campbell, Independent Consultant.
Attribute sampling
2. ISPE Pharmaceutical Engineering Guide, Commissioning
and Qualification, January, 2001.
Call for more de
Article Acronym Listing other training vid on these or
eo titles
cGMP: current Good Manufacturing Practice
FDA: Food and Drug Administration
FMEA: Failure Mode and Effect Analysis
FTA: Fault Tree Analysis
GEP : Good Engineering Practices
GMP: Good Manufacturing Practice
HACCP: Hazard Analysis and
Critical Control Point
ISPE: International Society of
Pharmacalogical Engineeers
200 Business Park Way, Suite F
P&ID: Piping and Instrument Drawing
Royal Palm Beach, FL 33411-1742
RPN: Risk Priority Number
SOP: Standard Operating Procedure Phone: 561-790-2025 U.S. Only: 800-276-4242
SPC : Statistical Process Control Fax: 561-790-2065 E-Mail:
URS: User Requirement Specification Web Site:

M a y 2 0 0 4 Vo l u m e 1 0 , N u m b e r 3 251