Benchmarks of Fairness for Health Sector Reform in Developing Countries: Overview and Latin American Applications

Norman Daniels PIH, HSPH Ndaniels@hsph.harvard.edu Santiago, Chile, Jan 16, 2004

Historical Development of the Benchmarks

1993 Clinton Task Force 1996 Benchmarks of Fairness for Health Care Reform Oxford University Press. Pilot work in Pakistan, 1997 1999-2000 Adaptation: Pakistan, Thailand, Colombia, Mexico: Daniels, Bryant et al Bulletin of WHO, June 2000 2001-3 Demonstration Phase: Mexico, Portugal, Pakistan, Thailand; Vietnam Cameroon, Ecuador, Nicaragua, Guatemala, Chile, Yunnan (China), Sri Lanka, Bangladesh, Zambia

The Adapted Benchmarks

1. Intersectoral public health 2. Financial barriers to equitable access 3. Nonfinancial barriers to access 4. Comprehensiveness of benefits, tiering 5. Equitable financing 6.Efficacy,efficiency,quality of health care 7. Administrative efficiency 8. Democratic accountability, empowerment 9. Patient and provider autonomy

Connections to social justice

Equity
B1Intersectoral public health, B2-3 Access, B4Tiering, B5 Financing

Democratic Accountability
B8, B9Choice

Efficiency
B6 Clinical Efficacy and quality B7 Administrative efficiency

Structure of BMs
B1-9 Main Goals
Criteria -- Key aspects
Sub criteria-- main means or elements

Evidence Base + Evaluation

Indicators Scoring Rules

WHO Framework vs BM
WHO
Scope Objective Purpose Product Who uses Requires Cross national Current perform Motivate Index, ranks National pol mk Good info

BM
Nat, subnat Reform eval Deliberate Scores Various Info, tr. people

Problems
Overlap

Inform change?
Move to reforms

Subjectivity?
complementary

B1: Intersectoral Public Health

Degree to which reform increases per cent of population (differentiated) with: basic nutrition, adequate housing, clean water, air, worplace protection, education and health education (various types), public safety and violence reduction Info infrastructure for monitoring health status inequities Degree reform engages in active intersectoral effort

B2: financial barriers to access

Nonformal sector
Universal access to appropriate basic package Drugs Medical transport

Formal Sector Social/Private Insurance

Encourages expansion of prepayment Family coverage Drug, med transport Integrate various groups, uniform benefits

B3: Nonfinancial barriers to access

Reduction of geographical maldistribution of facilities, services, personnel, other Gender Cultural -- language, attitude to disease, uninformed reliance on traditional practitioners Discrimination -- race, religion, class, sexual orientation, disease

B6: Efficacy, efficiency and quality of health care

Primary health care focus
Population based, outreach, community participation, integration with system, incentives, appropriate resource allocation

Implementation of evidence based practice

Health policies, public health, therapeutic interventions

Measures to improve quality

Regular assessment, accreditation, training

B8: Democratic accountability and empowerment

Explicit public detailed procedures for evaluating services, full public reports Explicit deliberative procedures for resource allocation (accountability for reasonableness) Fair grievance procedures, legal, non-legal Global budgeting Privacy protection Enforcement of compliance with rules, laws Strengthening civil society (advocacy, debate)

Why is evidence base important?

Evidence base makes evaluation objective Making evaluation objective means:
Explicit interpretation of criteria Explicit rules for assessing whether criteria met and the degree to which alternatives meet them

Objectivity provides basis for policy deliberation

Gives points of disagreement a focus that requires reasons and evidence

Evidence Base: Components

Adapted Criteria--convert generic benchmarks into country-specific tool
Reflect purpose of application Reflect local conditions

Indicators
Outcomes Process revisability

Scoring rules
Connect indicators to scale of evaluation Specify in advance

Process of selecting indicators

Clarity about purpose Type of criterion determines type of indicator
Outcomes vs process indicator appropriate Standard vs invented for purpose Requires clarity about mechanisms of reform

Availability of information Consultation with experts Final selection in light of tentative scoring rules Further revision in light of field testing

Scoring Benchmarks Reform relative to status quo

-5

+5

Or use qualitative symbols, --- or +++

Scoring Rules: General Points

Map indicator results onto ordinal scale of reform outcomes Final selection of indicators should be done as scoring rules are developed, so refinements can be made Scoring rules should be adopted prior to data collection to increase objectivity, but may have to be revised in light of problems

Two approaches to evidence

Thailand: survey of Guatemala, various groups judging Cameroon: team based on discussion of evaluation based on evidence indicators, scoring rules Strengths: range of views, involvement of Strengths: clarity larger groups about evidence base for evaluation Weakness: vaguer basis for judgment? Weakness: trained team, narrow input

Guatemala, Ecuador: Stage 1: Theoretical adaptation

Conceptualizing public health
The set of actions implemented through a health care system which includes personal, collective, environmental and health promotion interventions. The delivery of services can be through public or private providers (with public funding) and its design and evaluation concerns providers, financers (public and private) and regulators.

Output:
Working document with specific version adapted to the context of Guatemala and Ecuador

Adapted benchmarks
Defined by Daniels et al (2000)

Benchmark I: Intersectorial Public Health Benchmark II: Financial barriers to equitable access Benchmark III: Non financial barriers to access Benchmark IV: Comprehensiveness of benefits and tiering Benchmark V: Equitable financing Benchmark VI: Efficacy, efficiency and quality of care Benchmark VII: Administrative efficiency Benchmark VIII: Democratic accountability and empowerment Benchmark IX: Patient and provider autonomy

Adaptation to Public Health Benchmark I: Intersectorial public health Benchmark II: Universal access to public health interventions Preventive services, Curative services Social protection against catastrophic illness Reduction of financial barriers Reduction non-financial barriers. Benchmark III: Equitable and sustainable financing Equity in health financing Sustainability in public financing Benchmark IV: Ensuring the delivery of effective public health services Technical quality (standard treatment guidelines) Efficiency (relation between inputs and outputs) User satisfaction Benchmark V: Accountability Social participation, community involvement in the evaluation and monitoring of inequities in health care delivery and resource allocation

Stage 2: Data collection and data analysis tools

Intervention level: Province/Department
Decentralization transferred policy-implementing responsibilities and resources to the sub-national level. Development of tools and field testing follows from the provincial to the municipal level.

Outputs:
Data collection: questionnaires (quantitative & qualitative) to assess criteria and indicators for each benchmark Data analysis: index to assess inequities, health expenditures analysis through proxies (drug consumption), excel database.

Stage 3: Field testing

Outputs:
Data collection tools for benchmarks I to V.

Examples of application
Starting with an analysis of inequities in the delivery of basic health care services and inequities in the distribution of basic resources.

INDEX OF PRIORITY FOR HEALTH SERVICE (IPSS)

IPSS= (Ciin-CDxin ) + Ciin (Ciap-CDxap )+ (Cips-CDxps ) Va Ciap Cips 3 IPSS= Index of priority for health services Ciin= Ideal coverage for immunization (100%) CDxin= Immunization coverage for district X Ciap= Ideal coverage for antenatal care (100%) CDxap= Antenatal coverage for district X Cipss=Ideal coverage for supervised deliveries (100%) CDxps=Coverage of supervised deliveries for district X Va= Sum of three values NOTES: The coefficient will go from 0.01 up to 0.99 The higher the value, the higher the priority for the delivery of basic services to the population

INDEX OF RESOURCES
IR = (GPDx X 0.4 ) + (MDx X 0.3)+ (FDa X 0.3) GPDa MDa FDx IR= Index of resources GPDx= per capita expenditure district x GPDa= District with the highest per capita expenditure MDx= Medical staff per population for district x MDa= District with the highest number of medical staff/pop FDa= District with the highest number of health facilities per population (district with the lowest number of inhabitants per health facility) FDx= health facility per population in district x

Indexes
DISTRICTS
SAN MIGUEL
CUBULCO GRANADOS

SAN JERONIMO
PURULHA EL CHOL RABINAL SALAMA

IPSS 0.51 0.47 0.38 0.36 0.33 0.33 0.28 0.15

IR 0.29 0.34 0.81 0.38 0.59 0.55 0.47 0.34

IPSS VERSUS IR
0.90

0.80

0.70

0.60

0.50 IPSS 0.40 IR

0.30

0.20

0.10

0.00 SAN MIGUEL CHICAJ CUBULCO GRANADOS SAN JERONIMO PURULHA EL CHOL RABINAL SALAMA

Examples of application
Benchmark II: Universal access to integrated public health services Definition of integrated public health: the delivery of services related to curative, preventive and health promotion, as well as services for both, transmittable and non-transmittable diseases and chronic diseases. An integrated effort should include some forms of protection against catastrophic diseases.

CRITERIA

INDICATORS

RESULTS

Access to the curative services % of population receiving the services N/A included in the basic package of at any of the three subsystems (public, services social security and private) with public funding

Access to preventive services % of population receiving the services N/A included in the basic package of at any of the three subsystems (public, services social security and private) with public funding
The provision of services aimed % health facilities at the district level at non-transmittable, chronic offering services for the following and degenerative diseases problems: diabetes, hypertension, cardiovascular diseases, screening cervical cancer 42% (5 facilities from a total of 12)

Actions implemented aimed to protect the individuals against the socio-economic consequences of catastrophic illnesses

% of health districts or municipalities 0%. This type that have a catastrophic disease fund of benefit for their population does not exist in the area

CRITERIA

INDICATORS

RESULTS
0% (interviews to health authorities 100% (focus groups with community members) 30% (see table & graph
for distribution)

Reduction of % health facilities in a given district in which financial barriers the population contributes with cash or in kind resources to the delivery of basic health care services (both curative and preventive) Reduction of non- % of health personnel (by category) that speak financial barriers the local indigenous language % of health staff (by category) who are women % of health facilities offering services in a schedule that is appropriate to the occupation and schedules of the local population (24 hours emergency; OPD services offered until late evening) % of first level health facilities that experienced shortage of basic resources during last year (equipment, drugs, medical staff)

59% (see table &graph

for distribution)

25% (3 out of 12 facilities)

(pending tabulation)

Instrument #1b: Human Resources (feed analysis of nonfinancial barriers and inequities in the distribution of health personnel)

PERSONNEL TOTAL WOMEN SPEAK LANGUAGE Doctors 12 2 0 Nurses 16 16 2 Auxiliary Nurses 17 17 2 Rural health technicians 9 0 3 Institutional facilitators 4 1 2 Community facilitators 12 4 12

PERSONNEL

Community facilitators

Instiutional facilitators

CATEGORY

Rural health technicians

SPEAK LANGUAGE WOMEN TOTAL

Auxiliary Nurses

Nurses

Doctors

10

12

14

16

18

NUMBER

Lessons learned
Benchmarks and their potential contribution to the analysis of inequities
Start by analyzing inequities in the delivery of basic health services and inequities in the distribution of basic resources From here the benchmarks can help to explain the factors that may be related to the observed inequities

Lessons learned
Difficulties of transferring concepts into practice
Identifying and assessing indicators for accountability, social participation, intersectorial work, etc.

Limitations related to health information systems

Existing system collects mainly traditional information (health service production) and has little flexibility to introduce new indicators (intersectorial work and others)

Lessons learned
Skills in research team
Actors at sub-national levels require skills development

Qualitative research
Potential users and data collectors have little experience & skills for qualitative research

Planning cycle
The benchmarks approach seems more useful as an approach that helps the planning cycle: evaluate existing situation-design interventions-implement-evaluate. Issues related to equity and social justice within the health system can be addressed in each of the stages of the planning cycle.

Ecuador
Team members:
12 people representing the following institutions: Universidad Nacional de Loja, PAHO-Ecuador, ALDES, Universidad de Cuenca, Fundacin Eugenio Espejo, Harvard School of Public Health (USA) Liverpool School of Tropical Medicine (UK)

Work carried out during the year 2003

5 workshops (two days per workshop) 9 work-meetings (one day or less) Outputs:
Adapted version to Ecuador of the generic matrix (Daniels et al 2000) with specific indicators for each benchmarks criteria Development of data collection instruments to assess indicators

Adaptation of generic matrix

Followed simmilar process to Guatemala Exchange of ideas and indicators between the Guatemalan team and the Ecuadorian team. Adaptation in Ecuador emphasize the assessment of recent health policies: national health system law, free MCH services law

Field application (Jan-April 2004)

Two provinces: Azuay y Canar 25 health facilities (11 MoH 7 social security; 7 NGOs; 1 local government. In addition, a household survey that will allow to investigate socio-economic inequalities and its relation with access to reproductive health and MCH services.

Expected use of findings (field application)

Inform local government health plans Inform advocacy groups in Azuay and Canar Field testing of the benchmarks approach as a tool that can aid the monitoring and evaluation of health policy implementation

APHA
Thailand Guatemala Cameroon

Later
Zambia--HIV/AIDS Yunnan, China-rural reform Ecuador, public health, comprehensive Vietnam-comprehensive reform Pakistan- community use Chile, Nicaragua, Sri Lanka, Nigeria (ACOSHED), Bangladesh

Plans for Benchmarks

Research Network for all sites, other efforts at monitoring reform Funding for country level projects using adapted benchmarks Coordination with WHO, regional organizations of WHO, World Bank, USAID