Surgical audit

Dr Bashiru Aminu
Moderator Dr Abur
Outline
Introduction
• Improving the quality of surgical care is a public health imperative
• As many as 100,000 patients die every year undergoing surgery in the
United States
• an order of magnitude more experience serious complications
• Despite recent studies suggesting that surgery is becoming safer over time,
a large body of literature suggests that a large proportion of surgical
morbidity and mortality may be avoidable
• In its seminal 1999 report To Err is Human, the Institute of Medicine
estimated that between 44,000 and 98,000 Americans die every year as a
result of medical errors, at least half of whom are surgical patients
Introduction
• Medical audit is the systematic, critical analysis of the quality of
medical care, including the procedures used for diagnosis and
treatment, the use of resources, and the resulting outcome and
quality of life for the patient
• The patient has some expectations regarding your services
• The surgeon also has his own expectation for the patient
• The system itself has expectation regarding quality of services
provided
Introduction
• Different health systems have different incidences for potentially
avoidable deaths
• This may be an indication of quality of medical care
• The aim of surgical audit is to improve quality of care
• This is done via quality assurance which is a system by which
provision or performance is measured against expectation with the
declared intention of minimizing deficiency
Introduction
• Its content is primarily clinical and educational as opposed to
managerial and its focus is on the process and outcome of medical
care
• Clinical audit encompasses all activities of all heath professionals
while Medical audit refers to the assessment by peer review of
medical care provided by the medical profession
• Please note audit is only of value if the system is run with the
standard funding required
Introduction
• Whenever possible it should be quantified.
• Audit should lead to changes in the organisation and availability of
services, clinical policy and clinical practice with consequent
improvement in the quality of medical care as measured by
appropriate indicators
Introduction –what audit is not
• Audit measures how far you are from a standard and it is a cycle of
activities
• Review measures where you actually are at a given time and it is not a
complete cycle
• Audit is not computing, computing or calculation is a component of
audit
• Audit is not resource management because resource management
sees resource as a first objective
• Audit is not contracting, though value based purchase may demand
audit as a component
Introduction –what audit is not
Research
• Research is an investigation which
aims to increase the sum of
knowledge
• it usually involves an attempt to test a
hypothesis
• Research may involve allocating
patients randomly to different
treatment groups
• Research may involve administration
of a placebo
• Research may involve a completely
new treatment, while
Audit
• No increase in new knowledge
• No hypothesis
• audit never involves allocating
patients randomly to different
treatment groups.
• audit never involves a placebo
treatment
• audit never involves a
completely new treatment
Criteria for a successful audit
• Relevance to common clinical problems
• Objectivity
• Honesty
• Accurate standardized forms
• Complete medical records
• Confidentiality
• Educational not punishment
Principles of audit
• Identify the problem
• It should be an activity that can be changed
• Stick to common problems
• It should have effect on patient outcome
• It should have an agreed standard
• Objectives and goals must be agreed upon
Principles of audit
• Collect data and analyze it
• Check what is obtained as against the standard
• Refine above criteria in light of experience
• Formulate recommendation and follow up action
• There should be changes consequent upon these findings
Principles of audit
• Measurement of outcome only useful with complete record keeping
• criteria should look at new findings in science or the cycle fossilizes
practice
• Ideal basis should be outcome but in practice it is usually process
• Autopsy is the final investigation of considerable value in audit
• Remember-it is a cycle of activity
Classification- organization
• Audit coordinator
• Resources
• Training
• Appropriate organizational framework
Classification- organization-coordinator
• Co-ordinate various specialty groups involved in audit.
• Liaise between consultants and managers in respect of resources
required for audit
• Collate information from all sources of audit
• Make appropriate recommendations
• implementation of recommendations emerging from medical audit
Classification- organization-coordinator
• Monitor the action recommended.
• Monitor statistics, including performance indicators.
• Arrange interdisciplinary meetings.
• Promote and support education.
• Review the literature on medical audit and disseminate it.
• Give advice when requested on what to audit and how.
• Liaise closely with the department of postgraduate education, district
medical education committee and other bodies
Classification- type of audit
(a) retrospective internal audit within a specialty, hospital, general
practice or district community in which records are used to review
past events
(b) concurrent audit which is a continuous assessment of patient
management
• In both types of audit results are compared with agreed standards
which may be implicit or explicit, protocols or criteria.
• Retrospective internal audit is likely to be the most appropriate
approach for the introduction of audit, but these approaches to audit
are not mutually exclusive
Classification-location
• Diagnostic departmental audits- process and outcome
• Anesthesia –structure, process, outcome
Classification-indicators
• With growing recognition that the quality of surgical care varies
widely, good measures of performance are in high demand
• Patients and their families need accurate information to help them
choose the safest hospitals for surgery
• Employers and payers need reliable measures for their value-based
purchasing programs
• Motivated in part by these external pressures, clinical leaders need
better measures to guide their quality improvement efforts
Classification-indicators
• Structure
• Process
• Outcome
• Composite
• No quality measure is perfect
• In the past care givers are encouraged to participate but today
measures are put in place to help only good performers to thrive
Classification-indicators
• With the proliferation of value-based purchasing, which requires a
global assessment of quality, there has been a rapid growth in the use
of composite measures
• the National Quality Forum (NQF) has emerged as the leading
organization endorsing quality measures
• Other organizations create theirs specific for themselves
• example, the Center for Medicare and Medicaid Services (CMS)
Surgical Care Improvement Program (SCIP), which includes process
measures related to SSI, DVT etc
Classification-indicators
• Agency for Healthcare Research and Quality (AHRQ) has focused on
quality measures that can be used with administrative data
• For example, the AHRQ maintains and distributes state and national
administrative data as part of their Healthcare Cost and Utilization
Project (HCUP)
• Processes of care are generally not available in these datasets, so the
AHRQ measures focus mainly on structure (e.g.,hospital volume) and
outcomes (e.g., mortality rates)
Classification-Structure
• Healthcare structure refers to fixed attributes of the system in which
patients receive care
• Many structural measures describe hospital-level attributes, such as
the resources or staff coordination and organization (e.g., nurse-to-
patient ratios, hospital teaching status)
• Other structural measures reflect attributes of individual physicians
(e.g., subspecialty board certification, procedure volume).
Classification-structure
• First, they are strongly related to patient outcomes
• For example, with esophagectomy and pancreatic resection,
operative mortality rates at high volume hospitals are often 10%
lower, in absolute terms, than low-volume centers
• In some instances, structural measures such as procedure volume are
more predictive of subsequent hospital performance than any known
processes of care or even direct mortality measures
Classification-structure
• Perhaps the most important advantage of structural variables is the
ease with which they can be assessed.
• Many can be determined using readily available sources, such as
administrative billing data
• Although some structural measures require surveying hospitals or
providers, such data are much less expensive to collect than measures
requiring detailed patient-level information
Classification-structure
• Perhaps the greatest limitation of structural measures is that they are not readily
actionable
• For example, a small hospital cannot readily make itself a high-volume center
• Structural measures are also limited in their ability to discriminate the
performance of individual providers
• For example, in aggregate, high-volume hospitals have much lower mortality
rates than lower-volume centers for pancreatic resection
• However, some individual high-volume hospitals may have high mortality rates,
and some low-volume hospitals may have low mortality rates Although the true
performance of individual hospitals is difficult to confirm empirically (for sample
size reasons), this lack of discrimination is one reason structural measures are
often viewed as “unfair” by many providers.
Classification-process
• Process of care measures are the clinical details of care provided to
patients
• Although long the predominant quality indicators for medical care,
their popularity in surgery is growing rapidly.
• Perhaps the best example of the trend toward using process
measures is the CMS’s SCIP
• As previously mentioned, this quality measurement initiative focuses
exclusively on processes related to prevention of surgical site
infections, postoperative cardiac events, venous thromboembolism,
and respiratory complications.
Classification-process strength
• Since processes of care reflect the care actually delivered by physicians,
they have face validity and enjoy greater buy-in from providers.
• They are also directly actionable and provide good substrate for quality
improvement activities.
• Although risk adjustment may be important for outcomes, it is not required
for many process measures.
• For example, the appropriate prophylaxis against postoperative venous
thromboembolism is a widely used process measure.
• Since virtually all patients undergoing open abdominal surgery should be
offered some form of prophylaxis, there is little need to collect detailed
clinical data for risk adjustment
Classification-process limitation
• The biggest limitation of process measures is the lack of correlation
between processes of care and important outcomes
• First, most process measures currently used in surgery relate to
secondary outcomes.
• While none would dismiss the value of prophylactic antibiotics in
reducing risks of superficial wound infection, this process is not
related to the most important adverse events of major surgery,
including death.
• Second, process measures in surgery often relate to complications
that are very rare
Classification-process limitation
• For example, there is consensus that venous thromboembolism
prophylaxis is necessary and important.
• The SCIP measures, endorsed by the NQF, include the use of
appropriate venous thrombosis prophylaxis.
• However, pulmonary embolism is very uncommon, and improving
adherence to these processes will, therefore, not avert many deaths.
Outcome
• Outcome measures reflect the end result of care, from a clinical
perspective or as judged by the patient.
• Although mortality is by far the most commonly used measure in
surgery, other outcomes that could be used as quality indicators
include complications, hospital readmission, and a variety of patient-
centered measures of quality of life or satisfaction
Outcome strength
• There are at least two key advantages of outcome measures
• First, outcome measures have obvious face validity, and thus are
likely to get the greatest “buy-in” from hospitals and surgeons
• Surgeon enthusiasm for the ACS-NSQIP and the continued
dissemination of the program clearly underline this Point
• Second, the act of simply measuring outcomes may lead to better
performance – the so-called Hawthorne effect
Outcome limitation
• Hospital- or surgeon-specific outcome measures are severely constrained
by small sample sizes
• For the large majority of surgical procedures, very few hospitals (or
surgeons) have sufficient adverse events (numerators) and cases
(denominators) for meaningful, procedure-specific measures of morbidity
or mortality
• Although identifying poor-quality outliers is an important function of
outcome measurement, focusing on this goal alone significantly
underestimates problems with small sample sizes
• Discriminating among individual hospitals with intermediate levels of
performance is even more difficult.
• Another significant limitation of outcome assessment is the expense of
data collection
Composite
• Composite measures have two main advantages over individual
quality indicators
• First, pooling multiple measures overcomes the problem with small
sample sizes described earlier
• Second, this approach deals with the problem of multiple conflicting
measures, and simplifies quality measurement by providing a single,
summary measure of performance
Composite
• Composite measures have both technical and practical limitations.
Perhaps the biggest technical challenge is weighting the input
measures
• Most existing composite measures are created using equal weighting,
expert opinion, or the “all or none” approach. For example, the STS
cardiac surgery composite score calculates a domain-specific score for
morbidity, mortality, process of care, and perioperative medications,
and then places equal weight on each domain
• Unfortunately, these simplistic weighting schemes do not take into
account the simple fact that some measures are more important than
others
Composite
• Composite measures also have a practical limitation.
• By design, composite measures reflect global performance with a
procedure or specialty.
• It is hard to know exactly where improvement is needed with this
global assessment.
• Thus, it is important to deconstruct the composite measures into the
individual process and outcome measures.
Advantages of audit
• Reduces unnecessary investigations
• Reduces overall costs
• fall in the number of reoperations and a reduction in deaths
associated with a range of surgical operations.
• Reductions in wound infection rates have been linked to surgical
audit.
• These improvements were linked to reduction in length of stay
• Frees up scarce resources for use elsewhere
Disadvantage of audit
• Time spent can be used better elsewhere
• No regard to actual challenge of a surgeon
• Recommendations may not be utilized
• Perpetual underfunding by governments
• Continuing medical education may be better in results
Current trends
• Quality of care and efficiency is the current thrust in choosing the
right measure
• Selective referral is also a thrust in quality care
• Centers of Excellence (COE) programs, aim to direct surgical patients
to hospitals or surgeons with the best results-blue cross blue shield
• COE programs, pay for performance(payers and patients), and
outcome feedback(surgeons)
• Reliability adjustment can eliminate statistical noise
• Autopsy in clinical audit