Anda di halaman 1dari 28

SIX SIGMA RISK ANALYSIS

Six Sigma Tools and Metrics


Learn what ISO standards, CLSI guidelines, and Joint Commission protocols recommend for Risk Analysis Identify hazards and failure modes in your own processes Apply the appropriate ranking scale (qualitative, semiquantitative, and quantitative) to your process Use Risk Analysis tools to assess and judge the acceptability of risks in your processes.

Risk Analysis is coming to medical laboratories. But for too many labs, Risk Analysis is a buzzword without meaning, an approach without defined technique. In this book, Dr. Westgard surveys the ISO standards (ISO 14971, ISO 22367) as well as the CLSI guidelines (EP18, EP23) and the Joint Commission methodology for Proactive Risk Reduction. After providing an overview of the general approach to Risk Analysis, Dr. Westgard explains how to adapt the principles for the medical laboratory, using data-driven tools and practical implementation tips: Process maps, flowcharts and fishbone diagrams Risk Acceptability matrices Assessment of hazards through Failure Mode Effect Analysis (FMEA) Fault Tree Analysis (FTA) and Failure Reporting, Analysis and Corrective Actions System (FRACAS) Six Sigma metric integration into the Risk Analysis techniques

WESTGARD QC, INC.

Using Six Sigma metrics, Dr. Westgard shows how Risk Analysis can be converted from an arbitrary and qualitative technique, into something concrete, quantitative, and relevant to medical laboratories and the patients they serve. For laboratories serious about adopting Risk Analysis in their operations - and manufacturers eager to provide industry-leading support of their instruments - this is an essential reference.

7614 Gray Fox Trail Madison WI 53717


Copyright 2011 Westgard QC Inc.

http://www.westgard.com

Six Sigma Risk Analysis


Designing Analytic QC Plans for the Medical Laboratory

James O. Westgard, PhD

Copyright 2011 7614 Gray Fox Trail, Madison WI 53717 Phone 608-833-4718 HTTP://WWW.WESTGARD.COM

Library of Congress Control Number: 2011906056 ISBN 1-886958-27-0 ISBN-13 978-1-886958-27-2 Published by Westgard QC, Inc. 7614 Gray Fox Trail Madison, WI 53717 Phone 608-833-4718
Copyright 2011 by Westgard QC, Inc. (WQC). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior written permission of Westgard QC, Inc..

Westgard QC, Inc. Copyright 2011

Preface
With manufacturers building control mechanisms into their analytical systems, laboratories are interested in customizing their quality control systems on the basis of risk analysis and the remaining failure modes, i.e., errors that still may occur and affect the quality of laboratory testing. The focus in this book is on the development of Analytic Quality Control Plans to fit the needs of a particular analytic instrument as operated in an individual medical laboratory. The QC issues discussed in this book trace their origin to CMSs 2004 interpretative guidelines for Equivalent QC (EQC), which allow laboratories to reduce the frequency of QC from 2 levels of controls per day to 2 per week or even 2 per month. There is no scientific evidence to support the equivalence of these practices to traditional QC procedures. There is no proof that these reduced QC frequency protocols provide adequate error detection or patient safety, even though CMS prescribed validation protocols in order to qualify for reduced QC frequency. Those validation protocols themselves are not valid. The EQC guidelines seem to be driven by a desire to simplify QC practices, particularly for Point-of-Care applications where operators have little experience in doing laboratory tests and minimal knowledge of traditional QC practices. Because of concerns and urgings of both manufacturers and laboratories, CLSI initiated a project to develop a new guideline for QC procedures based on risk analysis. That guideline is known as EP23 Laboratory Quality Control based on Risk Management. Manufacturers are generally familiar with risk analysis because of practices recommended in ISO 14971 Application of risk management to medical devices. Medical laboratories, on the other hand, have little or no experience with formal risk analysis. We, like others, had to start from scratch to study risk analysis in order to understand its potential application for developing Analytic QC Plans. In the process of learning about risk analysis, we examined the risk models and the reliability of particular techniques for estimation or calculation of risk. We assessed the practicality of the risk analysis methodology that was being recommended. On the

Page i

Six Sigma Risk Analysis

basis of our studies, we concluded there could be serious problems in implementing the CLSI guidance unless additional educational materials and training programs are made available to laboratories. This book is part of our efforts to provide more practical and quantitative guidance for the application of risk analysis in medical laboratories. In Part I, First, do no harm!, we consider the problems and issues that must be addressed within the overall framework for managing analytical quality in a medical laboratory. In Part II, ISO and CLSI Guidance, we review several international and national consensus standards, including ISO 14971, ISO 22367, ISO 15198, CLSI C24, CLSI EP18, and CLSI EP23. In Part III, Methodology and Tools, we recommend adoption and adaptation of the JC (Joint Commission) Proactive Risk Reduction methodology and illustrate the application of many of the tools that are useful for developing Analytic QC Plans. Our resulting methodology employs a more rigorous risk analysis model and a more quantitative approach for assessment of the residual risk of an Analytic QC Plan. As quality management systems evolve, risk analysis should be integrated with existing practices, particularly with Six Sigma concepts, principles, tools, and metrics. That is the basis of our approach in Six Sigma Risk Analysis. Six Sigma is inherently risk oriented in its definition of tolerance limits, estimation of defects, and characterization of defect rates. Risk models, when properly applied, can provide estimates of the number of defective test results that may produced by a laboratory. Such defective test results are potentially harmful or hazardous to the health of our patients. Given that Six Sigma Risk Analysis is a more advanced and complex subject, it will require more extensive and more thorough study than some of our previous books. This book assumes a basic knowledge of quality control, such as found in our book on Basic QC Practices that describes the principles and procedures for implementing Statistical QC to monitor the performance of the methods in your laboratory. It assumes knowledge of the experimental and statistical techniques for evaluating the analytical performance of measurement procedures, such as found in our book Basic Method Validation. It also builds on Six Sigma concepts, principles, and tools, as found in our book on Six Sigma Quality Design and Control

Page ii

Westgard QC, Inc. Copyright 2011

and in our book on Assuring the Right Quality Right, which focuses on the design of Statistical QC procedures to verify the attainment of the intended quality of test results (the technical requirement for QC as stated in ISO 15189).

A Note on ISO standards and CLSI guidelines


As of the time of this printing, the CLSI EP23 was not yet finalized and accepted as a guideline. Nevertheless, the final shape of the document is complete. But in that spirit, readers should note that each standard, guideline, and regulation is inevitably a moving target. ISO and CLSI continuously review and attempt to improve their documents, and every few years they issue an update. The regulatory and accreditation bodies (CLIA, CAP, JC, etc.) do the same. Thus, the specific language of some of these standards will change. However, as you are probably well aware, large changes in regulatory policy are rare. It is unlikely that major changes will occur that change the goals of these organizations and their recommendations. It is also important to note that this book is NOT meant to replace or substitute for ISO standards or CLSI guidelines. Laboratories are strongly encouraged to purchase the specific documents that they intend to use in their operations. For a manufacturer or laboratory that intends to implement Risk Analysis, it will not be sufficient to read just this book. Where this book can be helpful is to give an overview and a comparison of ISO, CLSI, and JC recommendations. Laboratories may be able to decide which documents to purchase, as well as how to reconcile the differences between the different standards and guidelines. The other unique feature of this book is the Sigma-metric approach. While the other recommendations tend to be vague on how to rank and judge the acceptability of risk, this book is very quantitative and data-driven. Assessing your risk on the Sigma-scale will give you a concrete estimate of its residual risk and impact on patient care. The combination of Six Sigma and Risk Analysis can provide powerful tools and techniques to the medical laboratory.

Page iii

Six Sigma Risk Analysis

Acknowledgments
First and foremost, Sten Westgard made this book possible through his perspective on risk analysis in the world outside the laboratory, his interest that we find a way to help laboratories do a better job of risk analysis, and his commitment to supporting publication of these new materials, both as Internet courses and this book. I am fortunate that once these words are typed into a word processor, they can be transformed into educational materials and quickly made available to the laboratory community. Sten makes that happen and these materials wouldnt exist without him. In developing Six Sigma Risk Analysis, we have been stimulated by discussions with many people, including Jan Krouwer, Don Powers, Greg Cooper, Tina Krenc, Mike Noble, and Jim Nichols. We have also been stimulated by the evolving standards and guidelines for application of risk analysis with medical devices and medical laboratories, and appreciate the time, effort, and hard work that has gone into their development. We hope that our distillation of the current guidelines and our recommendations for adapting the JC methodology will support the implementation of a more rigorous and objective approach for risk analysis in medical laboratories, particularly for the design of Analytic QC Plans that will verify the attainment of the intended quality of results, as recommended in ISO 15189.

But wait, theres more!


This book is not long enough to contain all the tools and lessons that we want to make available to you. So weve put some more online. For readers who have purchased this book, additional tools available online. For example, a set of power function curves that show the error detection and false rejection capabililites of Average of Normals (AoN) QC procedures. Visit http://www.westgard.com/risk-extras.htm for more details. Youll be given a link to a special location, along with password instructions.

Page iv

Westgard QC, Inc, Copyright 2011

Table of Contents
1. Controlling Quality ..................................................................................................................................... 1 2. Managing Analytical Quality
......................................................................................................

13

3. Analyzing and Assessing Risk .................................................................................................25 4. A Safety Net to Catch Analytical Errors ........................................................................51 5. ISO 14971 Risk Management for Medical Devices .........................................67 6. ISO 15198 and CLSI C24 Guidance for Safe Use and QC .......................95 7. EP18 & EP22 Guidance for Risk Analysis and QC Plans .....................113 8. ISO 22367 Guidance for Risk Management .......................................................... 137 9. Adopting the JC Risk Analysis Methodology .................................................... 147 10. Diagramming a Laboratory Process .......................................................................... 163 11. Identifying Failure Modes ........................................................................................................ 177 12. Prioritizing Failure Modes ....................................................................................................... 189 13. Determining Root Causes ....................................................................................................... 203 14. Mitigating Risks with an Analytical QC Plan.................................................... 213

Six Sigma Risk Analysis

15. Estimating Detection and Evaluating Residual Risks.......................... 233 16. Monitoring Failures and Measuring Performance .................................... 259 17. Implementing Analytical QC Plans .............................................................................. 275 18. Integrating Six Sigma into Risk Analysis
...........................................................

283

Index

................................................................................................................................................

291

Westgard QC, Inc. Copyright 2011

3: Analyzing and Assessing Risk


with Sten Westgard, MS What is Risk Management? Existing ISO guidelines and emerging CLSI guidelines recommend that risk management be applied in healthcare laboratories. Laboratories face a learning curve that includes basic concepts and terminology, as well as practical applications for the total testing process. Our particular interest here is the application of risk management to the analytical portion of the testing process, particularly the development of an Analytical QC Plan that takes into account the potential risk of a particular test, its analytical methodology, and its application in a laboratory environment.

We want to be forthright about our concern about the application of risk management in healthcare laboratories. We agree with the principles of risk management to prevent problems from occurring and reduce harm to patients when problems do occur. It would be ideal if errors can be prevented by manufacturers in their design of analytic systems and minimized by built-in controls and instrument checks, but laboratories are still responsible to verify the attainment of the intended quality of test results, according to ISO 15189 [1]. We believe that means Statistical QC should be a major part of any Analytical QC Plan. Also, we have reservations about practice guidelines that are emerging, particularly in the US where the drive has been to reduce the amount of QC performed, rather than to optimize QC to guarantee the quality of test results. Our purpose here is to demonstrate that the principles of risk management can be related and applied quantitatively as part of analytical quality management and with the application of traditional SQC. To accomplish this, we will review the principles of risk management, define the terms to clarify the risk management Process, focus on the steps involved with risk assessment and a

Page 25

Six Sigma Risk Analysis

commonly-used tool Failure Mode Effects Analysis (FMEA), then demonstrate how its application to analytical testing can be related to the Sigma performance of analytical methods and systems.

An all-too-brief history of Risk Management


Peter Bernstein, author of Against the Gods: The Remarkable Story of Risk [2], places the birth of Risk (as we know it) in a coffee house in London around the year 1696. Edward Lloyd, the enterprising owner of the Starbucks of his day, noticed his customers mostly sailors and ship captains, since he was near the docks had an insatiable thirst for information about the arrivals and departures of ships, as well as details on sea conditions and news in foreign countries. He published all of this information in what became known as Lloyds List. All of this information, assembled together, dramatically improved the ability to launch new ventures. [A]nyone who was seeking insurance would go to a broker, who would then hawk the risk to the individual risk-takers who gathered in the coffee houses or in the precincts of the Royal Exchange. When a deal was closed, the risk-taker would confirm the agreement to cover the loss in return for a specified premium by writing his name under the terms of the contract; soon these one-man insurance operators came to be known as underwriters. In 1771, nearly one hundred of the underwriters formally created the Society of Lloyds. Over 300 years since Edward Lloyd began serving information along with coffee, Lloyds of London has grown into a leading supplier of specialist insurance. Of course, you might protest, thats the birth of insurance, not risk management. But the two concepts are intertwined. Insurance is an answer to the problem of risk. Its illustrative to see that at the very birth of the risk and risk management, information was critical to the process. Even if sea merchants lacked the information or ability to mitigate their hazards by redesign (i.e. change the weather so their ships would avoid storms), they gained some idea of what to avoid.

Page 26

Westgard QC, Inc. Copyright 2011

As we return to the history of risk, we need to connect the birth of risk and risk management to its formal adoption and implementation in healthcare. To do that, we have to leap forward several hundred years to the malpractice crisis of the 1970s and 1980s. During that time, verdicts and settlements against hospitals and other healthcare institutions threatened to overwhelm their financial reserves. Risk management became a way to address the threat of litigation. While at first this was a defensive, reactive move, a way to shield the institution from litigation and losses, eventually healthcare institutions began to use risk management proactively [3]: Professionals with clinical experience were hired with the hope that they could identify the systemic problems in specific clinical areas (primarily obstetrics, anesthesia, and the emergency department), engage clinicians and educate them about the need to modify specific behaviors, and work collaboratively with others on the clinical and administrative teams to help design environments that would be more conducive to the delivery of safe care. When the reports of the Institute of Medicine, To Err is Human [4] and Crossing the Quality Chasm [5] were issued in 2000 and 2001, respectively, both the public and healthcare professionals alike were shocked by the frequency and severity of medical errors. That heightened awareness motivated the healthcare field to search for new tools to combat and prevent medical errors. Risk management became one of the new tools to address the problem. In healthcare, the Joint Commission recommended the use of risk management as part of the Patient Safety Movement. Its accreditation guidelines for 2002 included a requirement that healthcare organizations should perform at least one Failure Mode Effects Analysis (FMEA) each year [6]. Also in response to patient safety issues, the Institute for Healthcare Improvement (IHI) began providing education, training, and support for FMEA via its website [7]. In addition, the Veterans Affairs National Center for Patient Safety supported the use of FMEA throughout its healthcare institutions[8].

Page 27

Six Sigma Risk Analysis

Finally, still another thread of history helped bring risk management into healthcare institutions global standards courtesy of ISO. Long accepted by industry, ISO sets rigorous guidelines for processes and products marketed worldwide. Adherence to ISO standards is often a de facto requirement for businesses to compete globally. As ISO standards were expanded and applied to more and different segments of industry, they developed standards for risk management in Medical Devices (ISO 14971)[9]. The medical device industry, already an industry where litigation worries mandated a robust analysis of potential design flaws and device hazards, found that the risk management techniques married well with their existing efforts to improve quality. From the medical device industry to the medical device marketplace was only a small step. Already ISO 15189 had specified particular requirements for quality and competence in medical laboratories. A further standard, ISO 22367 [10], specified techniques for the Reduction of error through Risk Management and continual improvement. For laboratories outside the US, ISO standards often replace, supplement, or substitute for local government regulations. Some countries simply point to ISO standards and adopt them in their entirety for accreditation of medical laboratories. In the US, however, ISO standards have not been widely adopted because the CLIA regulations have been dominant. With the CLIA Final Rule in 2003 and the subsequent proposal for Equivalent Quality Control practices, the door opened wide for alternative quality regulations. Faced with scientific and professional debate about the adequacy of the new EQC guidelines, CLSI began to develop an alternate approach for defining QC Plans based on risk management guidelines that adhered to the ISO standards. These CLSI guidelines are intended to supplement, if not replace, the CLIA guidelines for EQC. CMS will decide whether the new risk-based QC guidelines can be used to provide equivalent quality testing.

What is Risk?
Risk is both a noun and verb, a concept and an action. We can take risks and we can risk disaster. We can speculate that analytical error is one of the biggest risks in laboratory testing. We can try to

Page 28

Westgard QC, Inc. Copyright 2011

11. Identifying Failure Modes


What tools can be used to identify failure modes? An essential part of risk analysis is identifying what might go wrong, i.e., the potential failure modes in the process under study. The process should first be diagrammed to clarify how the process works. Next, each step of the process should be examined to identify what might go wrong. The technique for doing this begins with brainstorming by the project team to identify potential failure modes, then summarizing and organizing those potential failure modes using a cause-and-effect diagram, also known as a fishbone diagram.

In any risk analysis methodology, the process of interest must first be diagrammed to identify the critical steps or operations, as described in the previous chapter. The next step in the methodology is to identify potential failure modes, i.e., what might go wrong and cause a delay or an error in reporting a test result. The identification of failure modes often makes use of brainstorming, followed by a graphical summary by fishbone diagram two tools that will be described in this chapter. Later chapters will consider the prioritization of failure modes to assess their relative importance and guide efforts to reduce the risks of failures, the identification of root causes of the high priority failure modes, and the utilization of risk control option analysis (ISO/CLSI terminology) [1-4] or process redesign strategies (JC terminology) [5] to eliminate causes when possible, detect failures and implement corrective actions for recovery, and reduce the risk of patient harm by providing information for the safe use of test results.

Page 177

Six Sigma Risk Analysis

Patient testing process


The overall patient testing process involves many steps and can be extremely complicated. To simplify any risk analysis project, it will be critical to confine the project to a part of the patient testing process, otherwise the project may become unmanageable (i.e. the scope will be so large as to be overwhelmed by a multitude of failure modes). An overview of the patient testing process is shown in Figure 11-1, which outlines two major processes the clinical testing process and the laboratory testing process, the latter often called the total testing process by the personnel in a medical laboratory. This laboratory process consists of the pre-analytic, analytic, and postanalytic phases of the patient testing process. It typically begins with the receipt of an order for a test and ends with the report of results. This is the way personnel outside the laboratory usually see the laboratory a department or black box with an input for test requisitions and an output for test results. Their focus is on the clinical testing process, or the pre-pre-analytic and post-postanalytic phases of the patient testing process, though they wont necessarily describe it using this terminology. In applying risk analysis in a medical laboratory, it will be critical to delineate the scope of the project, particularly whether it crosses the departmental boundaries between the clinical and laboratory processes. Across-department projects will require a more diverse and larger team. For example, projects on patient identification and test turnaround time are inherently difficult because of the number of people and departments that are involved. Even within the laboratory, it is necessary to carefully delineate the start and stop steps to narrow the focus when possible to a primary phase. For example, when the purpose of the risk analysis project is to develop an Analytic QC Plan, the focus will be primarily on the analytic phase of the laboratory testing process. Still, there could be overlap with the pre-analytic phase related to sample quality and overlap with the post-analytic phase related to implementation of control mechanisms (e.g., delta checks), review of test reports, and provision of information for safe use via the LIS or HIS. The project team may need some representation from different areas in the laboratory department.

Page 178

Westgard QC, Inc. Copyright 2011

Clinical Testing Process


START Pre-Pre-Analytic
Collect patient info Select test Prepare patient Enter test orders
YES

Need Laboratory Test?


NO

Post-Post-Analytic
Review test reports Interpret test results Plan treatment Treat patient

END

Pre-Analytic
Receive test order Identify patient Collect specimen Transport specimen Process specimen Prepare samples Distribute samples

Analytic
Receive & Inspect samples Prepare reagents & controls Setup analyzer Calibrate method Analyze samples Verify analyzer operation Check QC Release test results

Post-Analytic
Review test results Monitor quality Add safety information Enter in patient record Transmit test reports

Laboratory Testing Process


Figure 11-1. Diagram of the patient testing process, showing the clinical testing process (composed of the pre-pre-analytic and post-post-analytic phases) and the laboratory testing process (composed of the pre-analytic, analytic, and post-analytic phases). This big picture of the patient testing process should provide the frame of reference for narrowing the focus of any risk analysis project. Next, a more detailed description in the form of a flowchart is needed of the specific process of interest. This chapter focuses on the analytic phase of the patient testing process and the identification of potential failure modes that lead to delayed test results and delayed patient diagnosis or treatment.

Page 179

Six Sigma Risk Analysis

Analytic testing process


The analytic phase begins with the receipt of samples and ends with the release of test results, as shown in Figure 11-2.
Receive Samples

Prepare reagents , calibrators, & controls Setup analyzer & Perform function checks

Inspect samples

Samples adequate?
YES

NO

Re-process, Re-collect

Calibrate & Verify analyzer performance

NO

Analyzer Ready?
YES

Analyze samples & controls

Repeat tests

Analyzer performance OK?


YES

NO

Trouble-shoot, Take corrective action

Release test results

Post-analytic review

Figure 11-2. Flowchart for the analytic phase of an example laboratory testing process.

Page 180

Westgard QC, Inc. Copyright 2011

15. Estimating Detection and Evaluating Residual Risks


How should the risk mitigations be tested and evaluated? After identifying the risk mitigation strategies or redesign options for addressing the high priority failure modes, the effectiveness of the redesigned process should be tested and evaluated. For laboratory applications where the redesigned process is actually an Analytic QC Plan, the residual risks should be evaluated to determine acceptability. The Joint Commission and ISO provide some guidance on how to test the process, but little useful advice for judging the acceptability of residual risks. The laboratory should assure that the risks are acceptable for the particular clinical or medical applications of the tests performed. In the case of laboratory tests, quality requirements can be defined for the intended use of each test, the detection of control mechanisms can be characterized by their probabilities of rejection, risk can be estimated in the form of a defect rate, and the residual risks can be described in terms of the number of potentially harmful patient test results. This step is described as analyzing and testing the redesigned process in the Joint Commission (JC) methodology [1]. The general approaches recommended include pilot testing, simulations, and paper testing which in this case means preparing a second FMEA. All these approaches have some utility in the evaluation of an Analytic QC Plan in a medical laboratory if they are focused on estimation of detection and evaluation of the risks that remain after implementation of the controls, i.e., residual risks. ISO 15198 [2] provides additional guidance on how this might be done, recommending studies that include the introduction of failure modes or error conditions, or simulated challenges, and in the case of SQC procedures, validation may be based on statistical evaluation of the simulated effects of imprecision and/or bias on actual performance data obtained in routine operating conditions.

Page 233

Six Sigma Risk Analysis

The combined JC and ISO guidance suggests the following approaches for analyzing and testing an Analytic QC Plan: 1. Evaluate a manufacturers information on the performance of built-in controls, which should be based on experimental studies to characterize their detection. 2. Characterize QC performance for stable sample and patient data control procedures on the basis of simulated effects of errors and estimate detection for medically important errors. 3. Prepare a second FMEA using a 3-factor risk model that includes the estimates of detection for the various controls in order to evaluate residual risks. It is important to distinguish the two uses of FMEAs. In the earlier step, a FMEA was used to prioritize the failure modes that were then addressed by risk mitigation actions. In the latter step, the purpose is to evaluate the effectiveness of those risk mitigation actions and determine the acceptability of the residual risks that remain. JC provides little guidance on how to make this decision. ISO 14971 [3] and CLSI [4,5] describe risk acceptability matrices that are qualitative (and somewhat arbitrary). And though they talk about clinically acceptable risk or medically acceptable risk, the guidelines provide no quantitative relationship between the acceptability matrix and definitions of the quality needed for the intended clinical and medical use of laboratory tests. Despite the risk numbers, it boils down to a personal decision (or a committee vote). You (or the team) decide that the risk is okay, based on your judgment. There should be a more quantitative, objective way to do this. Analytic QC Plans should incorporate a definition of the allowable total error for the quality required for a test. The expected defect rate provides a rational metric for estimating the risk due to delayed and/or erroneous test results. Defect rate can be used to calculate the number of test results that are potentially harmful to patients, which provides an understandable estimate of residual risk. In this way, laboratories can make an informed judgment on the acceptability of residual risks, rather than resorting to an arbitrary risk acceptability matrix.

Page 234

Westgard QC, Inc. Copyright 2011

1. Evaluate the manufacturers information about built-in controls


Ideally, a manufacturer will disclose the risk analysis studies that describe the effectiveness of its built-in controls, but remember a manufacturer is not required to do this. If such information is not available, it may be possible for a medical laboratory to estimate detection for some of these controls by performing the appropriate experimental studies. For example, the effectiveness of serum indices for detection of abnormal sample conditions could be determined in the laboratory by preparing a series of samples having different levels of hemolysis, lipemia, and bilirubin. Such experiments would be a natural part of the method validation studies that should be performed in the laboratory. Experimental testing of other built-in controls might be possible but will be more difficult. It would be much better if the manufacturer performed that testing and provided clear documentation. If this information is not available from the manufacturer, then the laboratory must implement at least one independent control. And if this risk information is not presented in a practically useful manner (for instance, in terms of a defect rate), laboratories will have little ability to understand the residual risks again pointing to a need for at least one independent control mechanism whose detection can be determined and optimized for the intended use of the test and the performance of the method. The easiest independent control mechanism in your laboratory is SQC. With a statistical QC procedure, you can determine its detection and verify the attainment of the intended quality of results in your laboratory.

2. Characterize QC detection of statistical control procedures


In evaluating the effectiveness of a QC Plan, a laboratory should optimize the probability of detecting medically important errors (true alarms) and minimize the probability of false error detection (false alarms) [5]. Manufacturers are expected to provide information about the performance of their electronic checks, built-in controls, and integrated liquid controls. Laboratories, likewise, must consider

Page 235

Westgard QC, Inc. Copyright 2011

17. Implementing Analytical QC Plans


How do you implement a QC Plan? Implementation of an Analytic QC Plan may be done using manual tools, offline computer tools, or online data processing tools. The available tools will depend on software that resides in an analytic system, computerized work stations, middleware, Laboratory Information System (LIS), or Hospital Information System (HIS). Implementation may involve qualitative, semi-quantitative, or quantitative applications of Six Sigma Risk Analysis. The complexity of the analytic systems and the capabilities of the laboratory analysts will affect how risk analysis is applied and how QC Plans are implemented in the medical laboratory.

Applications of Six Sigma Risk Analysis depend on the resources available in an individual laboratory, particularly the data analysis programs and information systems for managing the testing processes. Manual systems may be sufficient for low volume laboratories and Point-of-Care applications, but computer support will be needed for high volume testing and complex analytic systems. Computer support may be found in the analytic system itself, offline and/or online programs. As the volume and complexity of the testing processes increase, online tools that are integrated into the quality management process become increasingly important. Given the many different analytic systems and the many different data processing configurations, the capabilities for implementing QC Plans will vary with the systems available to your laboratory. In the discussion here, we use the terms qualitative, semi-quantitative, and quantitative to refer to the rigor of the risk analysis methodology and the objectivity in the estimation of risk, not to identify the type of laboratory tests (i.e. qualitative or quantitative lab tests).

Page 275

Six Sigma Risk Analysis

Qualitative risk applications


Lets start with the simplest case of a manual method that performs only one test. This category applies to many Point-of-Care devices and Physician Office Laboratories. The test systems are often waived, which means there are minimal regulatory requirements, usually only that the laboratory must follow the manufacturers instructions for use. Nevertheless, a good quality system requires that method performance be validated, which should include the definition of the quality requirements for the intended clinical use of the test. With the information on the quality requirement for the test and the precision and accuracy observed for the method, the SQC procedure should be designed to assure detection of medically important errors. Next particular failure modes and events that require testing of control materials should be identified to determine when controls need to be analyzed (the basic safety net for catching errors). The manufacturers recommended control mechanisms should be reviewed, along with regulatory and accreditation requirements, to define a QC Plan. The Plan should specify frequency of monitoring, corrective action and recovery procedures, and reporting guidelines that include information for safety. For simple applications, method validation should include experiments for reportable range, replication, comparison of methods, and verification of reference intervals. Data analysis may be performed using an electronic spreadsheet or Internet tools, such as the method validation calculators provided on the Westgard website (www.westgard.com). Decisions on acceptable performance depend on the definition of the quality required for the intended use, which may be different in Point-of-Care applications and Physician Office Laboratories. Calculation of a Sigma-metric helps the decisionmaking process, as well as the selection of an appropriate SQC procedure using the Sigma-Metric QC Selection tool from CLSI C24 [1]. Control charts can be prepared for manual plotting of control measurements and should include a tabular record of any changes to the process and all corrective actions. Other control mechanisms recommended by the manufacturer should be integrated into the QC Plan and documented in the control records. Operator training should include training on all of the control mechanisms included in the QC Plan.

Page 276

Westgard QC, Inc. Copyright 2011

Index
A Acceptability matrix, see risk acceptablity matrix AdvaMed 7 Allowable total error (TEa) 21, 58, 222, 285 Analytical performance characteristics 52 Analytical QC Strategy 19, 22 Analytical quality requirements 56 Analytic errors 143 Analytic QC Plan 19, 21, 151, 178, 191, 222- 226, 233, 259261 Developing 213230 Implementing 275281 The plan, the plan, the plan, the plan! 270 Analytic QC strategy 53, 283 Analytic system checks 130 Analytic testing process 180181 A safety net to catch analytical errors 5166 Autoverification programs 280 B Block diagrams and Top-Down Flowcharts 166169 Blood transfusion process 150 Brainstorming 33, 152, 168 basics 181183 Brooks 99 C Carey 131 Cassidy 284 Cembrowski 131 CLIA 1, 2, 7, 10, 18, 53, 124, 156, 157, 219, 261 QC possibilities 35 CLIA criteria for acceptable performance 21 CLIA Final Rule 28 clinical decision interval 21 CLSI 5, 9, 17, 28, 32 CLSI C24 10, 92, 95 CLSI C24 Guidance for SQC Procedures 101111 CLSI EP15 266

CLSI EP18 9, 34, 36, 45, 67, 113 123, 153, 186, 214, 278, 280 CLSI EP22 83, 132 CLSI EP23 9, 34, 45, 67, 97, 113-114, 124-132, 163, 183, 199, 213, 219, 280 Cognitive error 139 Competency evaluation 268269 Continual improvement 138 Control procedure definition 96 Corrective action 138 Corrective Actions and Preventive Actions (CAPA) 264 Criticality 31, 36, 117, 118, 153, 191 definition 117 Criticality matrix 36, 117 Critical systematic error 240241 Crossing the Quality Chasm 27 D Default QC (DQC) 5 Defect rates 285, 287 converting ISO/CLSI ratings 247 Delta checks 216 Deming 14 Design for Six Sigma 17 Detailed process flowchart 172173 Detailed process outline 169 Detection 33, 42, 43, 132, 153, 189 200, 214, 216220, 245, 246, 286 estimating 233256 Developing Analytical QC Plans and Quality Systems 1924 Device risk mitigation features 132 Diagramming a Laboratory Process 152,163 174 Disclosure 221, 262263 E Electronic QC 18 Eliminating or reducing occurrence 215216 Equivalent QC (EQC) 5, 28, 124, 219 options 7 Option 4 7 Equivalent quality testing 5, 28 Error assessment 57 Error budgets 16

Page 291

Six Sigma Risk Analysis


Estimating Detection 233256 Examination failure 86 External Quality Assessment (EQA) 217, 267 F Failure Mode and Effects Analysis (FMEA) 17, 27, 32-33, 114-115, 143, 147, 148, 153-154, 189 200, 272 3-factor model 245 definition 115 example for Turnaround Time 194 example tables 250253 preparing a second FMEA 234 Preparing a table 192 Failure modes 33, 168, 189-200, 213, 270 Identifying 177187 List of potential failures and specific causes 120123 Potential failure modes causing a delay in reporting test results 182 Failure Reporting, Analysis, and Corrective Action Systems (FRACAS) 114, 263 definition 115 False rejection rate 38 Fault Tree Analysis (FTA) 114, 155, 207 212 definition 115 example FT for identifying root cause of delayed test analysis 211 FDA 1, 6, 7 Fishbone diagram 33, 177187, 183186 Five-Whys 155, 204, 206207 Flowcharts 15, 33 basics 164165 Formulation of an Analytical Quality Control Strategy 6166 Fraser 262 Frequency of controls 260 G Guidance to Manufacturers on Validating User QC Procedures 96 H Harm 29 Hazard 29, 69 definition 71 related to performance 8788 Hazard analysis 31, 140 Hazard Identification 127 Hazard matrix 117 Hazard score 117 Healthcare FMEA (HFMEA) 117 hemolysis 157, 197, 221, 235 HFEMA risk scoring matrix 117 High risk methods 66 Hubbard 286 I Identification and control of non-conformities 138 Improving detection 216220 Inadequate volume 197 Information for safety 9091 Inspection 16 Instructions for use definition 97 Integrating Six Sigma into Risk Analysis 283289 Intended purpose 71 Intended use 71, 86 ISO 17, 34 ISO 14971 6, 28, 29, 34, 36, 67 93, 129, 143, 213, 221 General Guidance and Approach for Risk Management 70 Informative annexes 6869 ISO 15189 3, 9, 22, 25, 28, 36, 52, 67, 9 2, 137, 221, 262, 266 ISO 15198 6, 91, 9599, 233 Validation of QC Procedures 98100 ISO 22367 6, 28, 35, 36, 153, 264, 269 ISO/TS 22367 131, 137145 IVD failure 86 J JC Risk Analysis Methodology 147161 Joint Commission 27, 147, 233 rating scale for severity, occurrence, and detection 154 Juran 14

56,

Page 292

Westgard QC, Inc. Copyright 2011


K Koshy 132 Krouwer 32, 207, 261 L Laboratory error 139 Latent error 139 Lean 17, 159 Lipemia 235 Lloyds 26 Low risk methods 64 M Management Responsibilities 140 Manufacturers Risk Report 131 McDermott 45 Measurement uncertainty 262, 266 Measuring Performance 259272 Medically important systematic errors Method Decision 285 Moderate risk methods 65 Monitoring Failures 259272 Multistage QC 46 N Need for objective and quantitative estimates of risk 285287 Need for scientific management of quality 284286 Need for uniform standards of quality in laboratory testing 287289 Non-cognitive error 139 Non-statistical QC procedures 53 O Occurrence 31, 33, 38, 39, 117, 128, 153, 189200, 216, 245, 246, 286 qualitative ranking 78 semi-quantitative ranking 78 Operating point 61 OPSpecs chart 61, 285 P Pareto analysis 31 definition 117 Pareto diagrams 15 Parvin 46, 99, 261 Patient data QC 130, 242, 280 Patient Safety 27 Patient testing process 178179 Peer comparison programs 267, 280 Performance specifications 53 Periodic review of QC data 267 Post-analytic errors 143 Power function graph 236-237 Powers 32, 35, 36, 45 Pre-analytic errors 142 Prevention 214 Preventive action 138 Prioritizing Failure Modes 168, 189200 Proactive Risk Reduction 147 161, 163, 166, 221, 259 Probability for error detection 236 Probability for false rejection 236 Process capability index, Cpk 39 Process redesign options 214 Proficiency Testing (PT) 217, 267 Protective measures 90 Q QC Design and Planning 15, 102105 QC Tools example assessment of feasibility of different tools 227 Qualitative ranking 34 Qualitative risk applications 276277 Quality Assessment (QA) 15 quality by design 266 Quality Compliance 2 Quality Control (QC) definition 2, 101 event-driven 46 Quality Control Plan (QCP or QC Plan) 114, 169, 216, 235 Quality Control rule (QC procedures) 53 definition 101 Quality Control Strategy (QC strategy) definition 102 high error detection 62 low error detection 63 moderate error detection 62 Sigma-metric relationship 63 Quality Goals (QG) 15, 52, 5466

240

Page 293

Six Sigma Risk Analysis


Quality Improvement (QI) 15 Quality Indicators 17, 269 Quality Laboratory Processes (QLP) 15 Quality Requirements Biologic variation 56 European recommendations for biologic goals 56 Medically important changes 57 Stockholm Consensus Conference 54 Quality triology 14 Quantitative risk applications 278279 R Recovery 214, 261 Redesigning the process 156157 Redesign options 156 Reducing severity 221222 Re-engineering 17 Reference change value 262 Residual risk 19, 30, 191, 199, 248 249, 254 definition 73 evaluating 233256 Risk 28 definition 71 Risk acceptability matrices 36, 82, 118, 143, 191, 234, 255 comparison of ISO and CLSI 129 qualitative 80 semi-quantitative 81, 144 Risk analysis 30-31, 143 comparison of JC and EP23 methods 168 definition 72 integrating Six Sigma 283289 outline of integrated JC/CLSI/ISO approach 170172 Risk assessment 30-31, 35 number of factors 34 prioritization 35 ranking scales 34 Risk control 30, 32, 168 definition 73 Risk control option analysis 155, 213 Risk estimation 31, 168 definition 72 Risk evaluation 30, 35, 36 definition 72 Risk information 45, 285 Risk management 17, 26-50 comparison of different models 44 Risk management 17 Risk Management QC Plan (QCP) 5 Risk matrix qualitative 79 semi-quantitative 80 Risk Mitigation 213230, 286 Risk Monitoring 32 Risk Priority Number (RPN) 31, 33, 118, 132, 153, 191, 248 Risk Reduction 32 Risk Score 191 Robust Process Improvement 159 Root Cause Analysis (RCA) 155, 168, 183, 203212, 215 differences between JC and CLSI 204205 Roubini 287, 288 S Safety by design 8990, 156 Selection of Statistical QC procedures 5966 Semi-quantitative ranking 34 Semi-quantitative risk applications 277278 Sentinel event 151 Severity 29, 31, 33, 37, 117, 118, 128, 153, 189 200, 245, 246, 286 qualitative ranking 77 reducing 221222 semi-quantitative ranking 78 Shadow testing 288 Sigma-metric 40, 79, 223225, 240 241, 245, 285 equation 58 priority of QC tools in relation to Sigma performance 225 QC recommendations 59 Sigma-Metrics QC Selection Tool 60, 105109, 285 Six Sigma 16, 17, 275 Specific monitors or quality indicators (list) 265 Stable sample controls 130 Stamatis 32, 45, 236 State Operations Manual (SOM) 7 Statistical QC 18 Statistical QC (SQC) 2, 5, 219

Page 294

Westgard QC, Inc. Copyright 2011


220, 235, 261, 272, 285 definition 102 Surrogate QC 3 Surrogate sample 3 T Tague 207 To Err is Human 27 Total Quality Management (TQM) Total testing process 178 Trueness 10 Trueness controls 217, 266 Trueness controls 268 Truth in labeling 1 Turbid sample 197 U Uncertainty 10 Use error definition 71 Use errors 8687 V Veterans Administration National Center for Patient Safety 117 W Wrong specimen type Y Young 262 197

14

Page 295

Westgard QC Order Form Westgard QC, 7614 Gray Fox Trail, Madison WI 53717 CALL 1-608-833-4718 if you wish to pay by purchase order or other means. Fax: 608-833-0640 or 203-389-6753
Item Price (US$) Quantity Subtotal

Westgard software and reference manuals


EZ Rules 3 QC Design software $695.00 $90.00 $70.00 $90.00 $90.00 $80.00 $80.00 $60.00 $60.00

NEW! Six Sigma Risk Analysis (save $20 if you order before September 30th)
Six Sigma Quality Design & Control, 2nd Edition Assuring the Right Quality Right (Advanced Quality Management) Basic QC Practices manual, 3rd Edition (most popular) Basic Method Validation, 3rd Edition Basic Planning for Quality CLIA Final Rules for Quality manual

Westgard Online Courses


Introduction to Risk Analysis for the Laboratory (4 credits) Quality Management and Design of Analytical Systems (5 credits) "Westgard Rules" and Levey-Jennings Charts mini-course (3 credits) Basic QC Practices complete online course (14 credits) Basic Method Validation complete online course (15 credits) $155.00 $75.00 $75.00 $135.00 $175.00 Grand Subtotal Shipping & Handling: Within US = $9 Canada & Mexico = Add 10%; Europe, Asia, & all other countries = Add 20% Sales Tax (Add 5.5% in WI, 6.0% in CT) GRAND TOTAL

Save more when you order online! Visit http://www.westgard.com/store.htm to see specials and coupons available only online
Your Name Institution Department Street Address City Country Business Phone/Business Fax E-mail Address Credit Card Type (circle one) Credit Card Number Signature VISA Mastercard American Express Exp. Date State Zip Code

**Westgard QC ships domestic orders via US Priority Mail with Delivery Confirmation. For other shipping methods, please call. Fax this to 608-833-0640 or 203-389-6753.

Anda mungkin juga menyukai