Quality Indicators:
Past and Present
History Quality Indicators and ISO Characteristics of Indicators strong and weak Quality Indicator Inventories USA and BC Examples of Quality Indicators Summary
Amazon.com $30.00
Date
1920s
Cycle
Plan-Do-SEE Plan-Do-CHECK-Act
Bob Galvin
1980s
Define-MEASUREAnalyze-ImproveControl
Act
Do
CHECK
Quality Indicators
A workable definition
Established measures used to determine how well an organization meets needs and operational and performance expectations.
Objective Measurable Repeatable
So
Quality Indicators are measured information that
Indicates the performance of a process. determines quality of services. highlights potential quality concerns, identifies areas that need further study and investigation, and track changes over time.
Measuring Performance
Mark Graham Brown
Fewer is better. Link measures to the factors needed for success. Measures should be based around customer and stakeholder needs. Measures should start at the top and flow down to all levels of employees. Measures should change as the environment and strategy changes Measures should have targets or goals established that are based on research rather than arbitrary values.
Keeping Score
Many organizations spend thousands of hours collecting and interpreting data. However many of these hours are nothing more than wasted time because they analyze the wrong measurements, leading to inaccurate decision making.
Mark Graham Brown.
RAND
JCAHO
OECD
WHO
Leapfrog
Achievable
Timed
short and long term
contained
Engaging
all levels
Good Metrics
Interpretable
specific
Balanced
full cycle
Actionable
action oriented
Acceptance Criteria
IQLM Indicators
Diabetes monitoring (system) Hyperlipidemia screening (system) Test Order Accuracy and Appropriateness Patient Identification (pre-analytic) Adequacy and Accuracy of Specimen Information (pre-analytic) Blood Culture Contamination (pre-analytic / system)
Accuracy of point-of-care testing (analytic) Cervical cytology/biopsy correlation (analytic) Critical Values Reporting Turnaround time (postanalytic) Clinician satisfaction (system/postanalytic) Clinician followup (system/postanalytic)
Percent
25 5 10 20 30 5
5
X100
[NUPA hr] * [Telephone minutes]
Just because a computer can calculate a value, doesnt mean that it should.
Crude Turn-Around-Time
A Cautionary Note
Measures that drive the wrong performance.
Measuring professionals is tough because intellectual work is difficult to measure objectively. Looking for factors that can be counted may not be what is really important. Meaningful outputs such as ideas, information, and problems avoided may be difficult but more relevant.
Mark Graham Brown
David Hsia
Medicare Quality Improvement Bad Apples or Bad Systems? JAMA. 2003;289:354-356.
Quality Inventory:
US Medical Laboratories 2004
In 2004 the Institute for Quality in Laboratory Medicine (IQLM) and the Clinical Laboratory Managers Association (CLMA) undertook an online quality inventory of laboratories with CLMA members. Approximately 400 laboratories responded. The study was voluntary, self-reported, with a validated questionnaire. Information provided was not verified by a second method
In British Columbia
The Program Office for Laboratory Quality Management and the Provincial Laboratory Coordinating Office have organized to perform a similar, but improved inventory in 2005.
96 94 92 90 88 86 84 82 80 78 76
QC EQ A Co PT nta ID Sa i n e mp l e I r OK nte Sa grity m La pl e I Wr b Inj D uri Co i tten e ll ec Or s Sp tion ders eci me Timi n nS t or g age
78 76 74 72 70 68 66 64 62
r po e R acy r cu c tA p Re a itic es u l a lV rn u T A nd u ro e m i T
Cr t or
Satisfaction Monitoring
BC Quality Inventory 2005
60 50 40 30 20 10 0
Phys ician lebot Emp loyee Patie nt omy) Patie nt (P
Developing Indicators
Objective Methodology
Limits Presentation Interpretation Limitations Action Plan Exit Plan
What are you trying to measure 1. 2. 3. How to capture the data Who (or what) to capture the data How often to capture the data
Acceptable, Concern, Unacceptable Critical Graphic or Text What does it mean? Does it reflect on YOUR quality? Unintended variables What will I do if it indicates acceptable performance? What will I do if it does not? When can I stop measuring?
80 70 60 50 40 30 20 10 0 1st Qtr 2nd Qtr 3rd Qtr 4th Qtr High Low Average
Microbiology Indicators
Collected and Monitored by Vancouver General Hospital Division of Medical Microbiology and Infection Control
Many thanks to: Diane Roscoe Anita Kwong Medical Microbiology team
Objective:
Limits: Below 2% Interpretation: Meeting accepted limits all the time Limitations Definition may include some true infections and
may miss others Presentation: Linear time graph Action plan: Identify and educate blood collector group. Exit plan: Reactivate with cause
Objective:
Limits: Below 2% (?) Interpretation: Wards with inexperienced collectors have problems Limitations Some frail and elder people have very weak veins and may be
impossible to collect Presentation: Linear time graph Action plan: Identify and educate blood collector group. Exit: Continue on selective basis
Certification Performance
Year 2002 2002 2003 Event Pre-Certification (EI) Certification (E) Pre-Certification (EI) Measures 100 100 100 MAJOR NC 1 0 0 Minor NC 2 2 0
2003
2004 2005 2005
Certification (E)
Certification (E) Pre-certification (EI) Re-Certification (E)
100
100 100 100
0
0 0 0
0
0 1 0
Event Pre-Certification (EI) Certification (E) Pre-Certification (EI) Certification (E) Certification (E) Pre-certification (EI) Re-Certification (E)
MAJOR NC 1 0 0 0 0 0 0
Minor NC 2 2 0 0 0 1 0
Objective:
Limits: No Major above 1Below 2%; No Minor above 3 Interpretation: Meeting accepted limits all the time Limitations May indicate things are better than they are if
inspector is not diligent Presentation: Linear time table Action plan: Maintain program, respond through OFI and Corrective Actions Exit plan: Compile with each inspection
Composite Indicators
Reflecting a single subject with a number of sub-components
When the finished value is greater than just the sum of the parts
3
4
5
Identify individual components Weight the components Define Limits Measure and Combine Monitor for trend
Weighting
+10 +5
-10
+10 +25 -100 +5 -10
105
0 0 1 1 30 90 5 80 30 16 0 0 0 0 1 1
VALUE Complaints Consults Contract Cancellations Contract Renewals New Contracts Negative Opinions
Positive Opinions
Survey
Limits
76
0 3 0 0 6 6 22 85 2004-2005 2005-2006 85 20 2 2 1 0 4 1
Complaints Consults
5 0 0 0 24 90 2002-2003
2003-2004 85 22 10
2 6
4 0 0 5
Contract Cancellations Contract Renewals New Contracts Negative Opinions Positive Opinions
Survey
Year
103.5
100
100 96.5
90
96.5
80
70
2002-2003 2003-2004 2004-2005 2005-2006
In Summary
Quality Quality Indicators are a required component of a quality management system. Quality Quality Indicators can be characterized and distinguished from Weak and Terrible Quality Indicators.
Watch out for the weak ones Avoid the terrible ones
Quality Quality Indicators provide the information and opportunity essential for POSITIVE action.
60 minutes
Relevant or Easy?
useful information.
Dont get tied to your indicators
David Hsia
Medicaqe Quality Impqovement Bad Apples oq Bad Systems? JAMA. 2003;289:354356.
Timed