review
22th Jan 2016
Doc No.: KPI/bc02
Framework overview
Indicator proposals
Annex
Overview
Five MEC & Steering Committee meetings since the autumn provided strong guidance on Gender, RSSH,
Key populations, Human Rights, Domestic investments indicator development
Technical partners have been engaged in reviewing the technical soundness of indicator proposals
Preliminary feedback from the OIG audit of the current KPI framework has showed good level of
acceptance of framework design
This document and the accompanying narrative sets out the preliminary first draft of the proposed
framework for Board constituency input
Written feedback is requested from constituencies which will then be reviewed and summarized by the
Secretariat (see accompanying form)
Next Steps
Consultation meetings with constituencies will be held to resolve differing viewpoints on major issues
This input will be integrated into the second draft of the proposals for review by the Board Committees 8/9th
March 2016
After Board approval of Corporate KPI framework, Secretariat will fully develop the framework of
cascaded Operational KPIs
2
Committee
update
Technical
development
Constituency
review
Jan 22nd 2016
High level
framework design
shared with June
SIIC
Example KPI
development
thinking shared with
October SIIC
Ongoing consultation
with technical partners
No presentations
yet made to FOPC
(or AEC)
Areas of contention
will be discussed
during the planned
consultations with the
implementer and
donor groups
Consultation
meetings
Jan Feb 2016
One-on-one
meetings with
constituency groups
Implementer group
meeting (date TBC)
Civil society meetings
Jan 19th & Feb 11th
Donor group meeting
(date TBC)
Review &
approval
March 8th 9th
Review of second draft of
framework incorporating
constituency feedback by
Board Committees
April
Final revisions will be
incorporated into submission
to Board for approval
May - November
Final methodological
development, baseline &
target setting for Board
approval
Full development of
cascaded Operational KPIs
3
New framework design draws from best practice structures and guiding principles
Leveling
indicators
Thematic
reporting
Variance
analysis
Lesson learned
Current focus
2
Corporate
level
Strategic KPIs
Secretariat
level
Operational KPIs
Strategic KPIs
2017
2022
Operational KPIs
Strategic
Objectives
Activity specific
KPIs
Evolve the
funding model
4 Efficiency of
7 Access to
Global Fund
funding
investment
decisions
5 Health system 8 New Funding
strengthening
Model
transition
6 Alignment
with national
reporting
systems
Retain
Revise
Delegate to operational
Drop
Strategic targets
4
4
6
2
11 Grant expenses
forecast
Strategic enablers
Implementing
operational excellence
Enhance partnerships
15 Efficiency of grant
management
operations
14 Domestic
financing for
AIDS, TB &
Malaria
16 Quality of
management and
leadership
4
4
4
Retained
Revised
Addition
Strategic targets
Strategic KPIs
Strategic
Objectives
Strategic vision
Activity specific
KPIs
3 Alignment of
investment & need
10 Resource mobilization
4 Investment efficiency
7 Fund utilization
9 Human rights
11 Domestic investments
12 Availability of affordable
health technologies
9
Definition
Accountability
Contribution: Global Fund is one actor among many contributing to results achieved
Attribution: Global Fund can largely attribute results to its actions
Data availability
Coverage
Projections available
Is it possible to forecast this KPIs results, thus enabling the Secretariat to take action
to mitigate potential future underperformance
Frequency of reporting
Given the frequency of data collection, or the size cohort sampled, how often will
meaningful KPI results be produced? (eg. Annually, quarterly)
KPI Proposals
Accountability
Data
availability
Actionable &
reliable results
Coverage
Projections
available
Frequency
of reporting
Contribution
Structured
Full portfolio
Annually
Contribution
System
Full portfolio
Semi
annually
Attribution
System
Full portfolio
Quarterly
Investment efficiency
Contribution
Structured
Portfolio segment
Semi
annually
Contribution
Not yet
available
Portfolio segment
Annually
Mix
Not yet
available
Portfolio segment
Annually
Fund utilization
Mix
System
Mix
Mix
Quarterly
Contribution
Structured
Portfolio segment
Annually
Human rights
Mix
Not yet
available
Portfolio segment
Mix
Annually
10 Resource mobilization
Contribution
System
Full portfolio
Quarterly
11 Domestic investment
Contribution
Structured
Full portfolio
Annually
Mix
Structured
Mix
Mix
Annually 11
Accountability
Data
Challenges
Time lag to
ascertain
measurable
effect
Challenge
Proposed approach
12
Strategic targets
Impact
targets
Global Plans
HIV /
AIDS
TB
Malaria
Global Fund
leading targets
14
Strategic Targets
KPI 1
Mobilize Increased
Resources
Strategic Vision
Measure
a)
b)
Aim of indicator
15
Strategic Targets
KPI 1
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Comment
Contribution
Structured
Full portfolio
Annually
Data is reported with a one-year lag due to partner data collection schedules
16
Strategic Targets
KPI 2
Mobilize Increased
Resources
Strategic Vision
Measure
a)
b)
c)
d)
Aim of indicator
Measures extent to which the Strategic Objectives are
achieving the high level service delivery targets.
Services tracked as part of the measure will be
reviewed with technical partners to ensure alignment.
As projection methodology is strengthened and results
forecast is institutionalized, the indicator will drive
portfolio performance management in conjunction with
ITP project.
e)
f)
g)
# males circumcised
# of IRS services delivered
# of people who received HIV testing
& counseling and know their results
Data Management for Impact initiative & Project AIM will improve
data and systems for reporting internally and at country level
17
Strategic Targets
KPI 2
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of
reporting
Comment
Contribution
It is proposed that all grants will report national results, however, the Global Fund
share of national results will be determined based on a contributive model
System
Target setting will be informed by the replenishment result and allocation model
assumptions. KPI performance is affected by expected levels of service delivery
being signed into grants and intervention level grant performance.
Full portfolio
Countries representing 90% of investment will report semi annually with annual
reports from whole portfolio. Projections could fill gaps for semi-annual reporting
Semi
annual
18
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic targets
Aim
Build Resilient
& Sustainable
Systems for
Health
Maximize Impact
Against HIV, TB
and malaria
Mobilize
Increased
Resources
Promote and
Protect Human
Rights & Gender
Equality
Strategic Targets
KPI 3
Mobilize Increased
Resources
Strategic Vision
Further improve alignment of investments with
country need
Aim of indicator
The measure tracks the extent to which the Global
Fund is able to rebalance the grant portfolio to
effectively invest funds in the countries where need is
greatest.
Measure
Alignment between investment decisions and country "need"; with
need defined in terms of disease burden and ability to pay
Limitations & mitigation actions
Strategic Targets
KPI 3
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Attribution
System
Illustrates extent to which grant expenses are committed to countries with most
need, and not necessarily those with best absorption. Provides a control for KPI7
Fund utilization
Full portfolio
Quarterly
22
Strategic Targets
KPI 4
Mobilize Increased
Resources
Investment efficiency
Strategic Vision
Increase the efficiency of program design to
maximize impact of fund investments
Aim of indicator
Measure
Improvement in cost per life saved or infection averted from
supported programs
Limitations & mitigation actions
23
Strategic Targets
KPI 4
Mobilize Increased
Resources
Investment efficiency
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Structured
Building disease modeling into grant processes will increase data availability
Close country engagement during modelling process will increase scope for
taking action
Portfolio
segment
Semi
annually
Initial focus on high impact portfolios, but there is also a demand for modelling in
UMICs
24
Strategic Targets
KPI 5
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
Strategic Targets
KPI 5
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Not yet
available
Portfolio
segment
?
Annually
Countries will be selected from the 55 expected to have key population size
estimates by the end of 2016 (HIV only)
If baseline coverage data available, projections may be possible based on
availability of service delivery targets in Performance Frameworks focusing on
key population interventions
Each reporting period will include few new data points as countries may only
conduct surveys every 2-3 years
26
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Aim
Build Resilient
& Sustainable
Systems for
Health
Maximize Impact
Against HIV, TB
and malaria
Mobilize
Increased
Resources
Promote and
Protect Human
Rights & Gender
Equality
Strategic Targets
KPI 6
Mobilize Increased
Resources
Strategic Vision
Increase the share of countries with resilient and
sustainable national systems for health that meet
standards for use by Global Fund programs
Aim of indicator
Indicator should provide portfolio-level view of national health
system strength based on explicit risk, functional and quality
standards
Measure
Share of the portfolio that meet expected standards for:
a) Procurement and supply chain systems
b) Financial management systems
c) Data systems and analytical capacity
Limitations & mitigation measures
Strategic Targets
KPI 6
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of
reporting
Comment
Contribution
Not yet
available
Systems do not yet exist for some of this data. Health facility assessments & PQR
may provide some data
Portfolio
segment
Annually
Cohort of countries sampled may differ across the three focus areas
Likely no projections will be available for forecasting system improvements
Each reporting period may include few new data points for some elements, as
countries may only conduct health facility assessments every 2-3 years
30
Strategic Targets
KPI 7
Mobilize Increased
Resources
Fund utilization
Strategic Vision
Increase the rate at which countries effectively absorb
allocated funds
Aim of indicator
A resilient and sustainable system for health should be able to
effectively use the full allocation of funds to deliver services to
increase program impact. This will be measured in two ways:
a) Allocation utilization provides high level view on the
extent to which:
countries can use their allocation, and
the Secretariat can optimize portfolio level investments
b) Absorptive capacity measures whether programs can
spend the budgeted funds
measure will focus on the 20 ITP countries with strong
links to Supply Chain Investment work and other
absorption related initiatives
Measure
a) Allocation utilization
Grant Expense (actual + MTP) / Allocation
b) Absorptive capacity
Cum. expenditure / Cum. grant agreement budget
Strategic Targets
KPI 7
Mobilize Increased
Resources
Fund utilization
Indicator dimensions
Dimension
Accountability
Data availability
Comment
Mix
System
Sourced from existing systems for grant expense actuals and forecasts, as well
as Enhanced Financial Reporting systems to provide expenditure data
Coverage
Mix
Allocation Utilization will cover whole portfolio, while Absorptive Capacity will
cover only selected countries
Projection available
Mix
Allocation Utilization will be forecast as part of the Mid-Term Plan, but Absorptive
Capacity is based on country expenditures and cannot be forecast
Frequency of reporting
Quarterly
32
Strategic Targets
Mobilize Increased
Resources
33
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
PQR could be primary data source, but this would mean delays
in data reporting (sometimes up to one year)
May be challenging to track administrative lead time without
additional data request to country
Outcomes can be impacted by factors outside the procurers
control (e.g. changes in market conditions for API can impact
price or OTIF)
34
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Comment
Contribution
Not yet
available
Currently data systems only exist for countries participating in PPM. Price data,
OTIF and Lead Time could be sourced through PQR, though with up to one-year
lag
Portfolio
segment
Projected prices may be available, OTIF and Lead Time forecasts may not be
possible
Annual
Data points available at each reporting period may be limited given PQR and
OTIF data coverage
35
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Comment
Contribution
Not yet
available
Some data may be available through health facility assessments, but likely not
at the frequency required for this indicator to be useful
Portfolio
segment
Likely data will only be available from countries conducting health facility
assessments or similar surveys
Projection available
Frequency of reporting
tbc
37
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Mix
Not yet
available
Portfolio
segment
Semi
Annually?
Each reporting period will include few new data points as 15-20 countries may
only conduct assessments periodically (tbc)
39
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic Vision
Well functioning HMIS enables better decision making and
ultimately better programs
Measure
Percent of high impact countries with fully deployed (80%
of facilities reporting for combined set of indicators),
functional (good data quality per last assessment) HMIS
Aim of indicator
This indicator does not measure data use directly, rather the
completeness and functionality of the information systems needed for
facilities/districts to have access to the data and tools to use the data
This aligns well with the significant GF investments into deploying and
improving these systems, as well as with international priorities
This KPI options alignment with investments and priorities, and its
feasibility to measure and reflect change, make it a preferred option of
for this KPI. However, the data are still more likely to be available for
high impact and core countries than focus countries
Strengthening HMIS is a priority workstream of the Global
Collaborative from the Measurement Summit, and specific tasks are
planned around this
40
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Contribution
Not yet
available
Portfolio
segment
HFAs and HMIS desk review are proposed as primary data sources
Assessment of HMIS coverage and data quality should guide investment and
data system strengthening focus in the countries monitored
Data will be more prevalent in High Impact and core countries
Annually
41
Strategic Targets
Op-KPI
Strategic Vision
Countries are able to report on the minimum set of outcome
and impact indicators to enable country monitoring and
meet international commitments
Measure
The percent of high impact and priority countries that were able to
report on core set of HIV, TB, Malaria and RMNCH indicators
Aim of indicator
Mobilize Increased
Resources
42
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Contribution
Structured
Portfolio
segment
Annually
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Strategic Vision
Measure
Countries not only have data available, but also use those
data to improve planning, budgeting, allocation process
and quality improvement
Aim of indicator
To track the extent to which countries have the data systems
and the capacity for analysis and decision making to improve
the efficiency and effectiveness of service provision
Ethiopia, Rwanda and Senegal cited as examples to inform
development of this strategic objective
Indicator does not measure true use of data, but rather the countrys
use of data quality and data use tools
Periodic surveys would provide data on this KPI every two years, and
for only a portion of the portfolio
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Not yet
available
Portfolio
segment
Evaluations would be rolled out only in High Impact portfolio, for example
Annual
Build Resilient
& Sustainable
Systems for
Health
Maximize Impact
Against HIV, TB
and malaria
Mobilize
Increased
Resources
Promote and
Protect Human
Rights & Gender
Equality
Strategic Targets
KPI 8
Mobilize Increased
Resources
Strategic Vision
Reduce gender and age disparities in health
Aim of indicator
HIV infection rates among young women are twice as high
as among young men in some regions. The indicator will
track the extent to which an enhanced programmatic
focus on women and girls results in a reduction in new
infections in selected countries with large disparities in
incident infections
This objective is closely linked to other strategic objectives
focused on scale-up of programs supporting women and
girls; advancing sexual and reproductive health and
rights; support to womens, childrens, and adolescent
health; and removing barriers to access
Measure
HIV incidence in women aged 15-24
47
Strategic Targets
KPI 8
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Projection available
Frequency of reporting
Comment
Contribution
Structured
Sourced from UNAIDS, modelled data of new incident cases, reported with a one
year lag
Portfolio
segment
?
Annually
Strategic Targets
KPI 9
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
The Global Fund has a key niche as the major funder of interventions
aimed at removing legal barriers to access. There is strong backing
from partners for this work WHO, UNAIDS, OSF, Ford
49
Strategic Targets
KPI 9
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Coverage
Comment
Contribution
Not yet
available
Portfolio
segment
Projection available
Frequency of reporting
The global fund has a key niche role as the major funder of interventions aimed at
removing legal barriers to access, but partner support remains key
Considerable work will be required to build the necessary tracking systems
The country dialogue and Concept note development stages will be key, as will be
using the time before then to start building capacity and partnerships. After these
stages of the funding process, building human rights programs into grant design
will become increasingly difficult
Countries will be selected across different epidemic, burden, conducive
environment and barrier contexts
tbc
50
Strategic Targets
KPI 9
Mobilize Increased
Resources
Human rights: b) key populations and human rights in middle income countries
Strategic Vision
Increase programing for key populations and human
rights in middle income countries
Measure
Percentage of country allocation invested in programs targeting
key populations and human rights barriers to access in middle
income countries, for:
i) Generalised epidemics
ii) Concentrated epidemics
Aim of indicator
Strategic Targets
KPI 9
Mobilize Increased
Resources
Human rights: b) key populations and human rights in middle income countries
Indicator dimensions
Dimension
Comment
Accountability
Attribution
Data availability
Structured
Portfolio
segment
A KPI will not be required if funding policy is sufficiently specific and rigorously
enforced
Middle income countries
A projection should be possible when new grants, subject to these stipulations,
come into force
Quarterly
52
Strategic Targets
KPI 9
Mobilize Increased
Resources
Strategic Vision
Upper middle income countries in transition take over
programing for key populations and human rights
Measure
Percentage of funding for programs targeting key populations and
human rights barriers to access from domestic (public & private)
sources
Aim of indicator
53
Strategic Targets
KPI 9
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Portfolio
segment
Annually
There will be a lag of 24-30 months after the start of the replenishment period
54
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Aim
Note the planned in-depth country-level evaluations of human right barriers to access proposed for 2016, 2019 & 2022 will
provide an additional source of information to understand the performance of the Global Fund in this area
55
Build Resilient
& Sustainable
Systems for
Health
Maximize Impact
Against HIV, TB
and malaria
Mobilize
Increased
Resources
Promote and
Protect Human
Rights & Gender
Equality
56
Strategic Targets
KPI 10
Mobilize Increased
Resources
Resource mobilization
Strategic Vision
Increase the financial resources available to the
Global Fund for investment in programs to tackle
the three diseases
Measure
a) Actual pledges as a percentage of the replenishment target
b) Pledge conversion rate. Actual 5th replenishment
contributions as a percentage of forecast contributions
Aim of Indicator
57
Strategic Targets
KPI 10
Mobilize Increased
Resources
Resource mobilization
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
System
Full portfolio
Quarterly
58
Strategic Targets
KPI 11
Mobilize Increased
Resources
Domestic investments
Strategic Vision
Measure
Aim of indicator
Strategic Targets
KPI 11
Mobilize Increased
Resources
Domestic investments
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Structured
No system exists currently to track this data, but this will be addressed by Project
AIM
Policy levers may need to be enhanced
Full portfolio
Annually
60
Strategic Targets
KPI 12
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
Ensure that supply is available from multiple qualityassured manufacturers, reducing risk of supply disruption
61
Strategic Targets
KPI 12
Mobilize Increased
Resources
Indicator dimensions
Dimension
Accountability
Data availability
Actionable & reliable
results
Coverage
Projection available
Frequency of reporting
Comment
Contribution
Structured
Where risks are identified, a number of actions can be taken, such as: proactive
outreach to potential new suppliers, targeting ERP, reserving volumes for new
entrants in PPM tenders, employing mechanisms to incorporate new entrants in
existing PPM tenders
Full portfolio
As a global indicator, all countries will be included; however, only select products
will be tracked
Annually
62
Strategic Targets
KPI 12
Mobilize Increased
Resources
Strategic Vision
Measure
Aim of indicator
Strategic Targets
KPI 12
Mobilize Increased
Resources
Indicator dimensions
Dimension
Comment
Accountability
Attribution
Data availability
Structured
Coverage
Projection available
Frequency of reporting
Underlying price data is captured via current systems, but requires manual
analysis. Forecast volumes are not captured via current systems
Target setting will be closely linked to anticipated volumes by product from PPM,
e-marketplace and other channels
Portfolio
segment
Countries that access PPM framework agreement prices as part of PPM, emarketplace or through other channels
This should be feasible with development of data capture system for LoHPs
Annually
Strategic Targets
Op-KPI
Mobilize Increased
Resources
Aim
Annex
66
End November:
Nov-March:
21 December:
11 January:
22 January:
Jan-Feb:
29 February:
Feb-March:
End-March:
Early April:
May-August:
Autumn:
Attract funding
Invest efficiently
10: Resource
mobilization
3: Alignment of
investment with need
11: Domestic
investment
4: Investment efficiency
Bridge
investments to
services
6: Resilient &
sustainable systems
for health
Address key
challenges
5: Key population
service coverage
1: Performance against
impact targets
8: Human rights
2: Performance against
service delivery targets
7: Fund utilization
12: Availability of
affordable health
technologies
Deliver impact
This alignment, complemented by Operational indicators, should provide early warning system for
tracking performance and taking management action
68
Activity
specific
KPIs
Operational
KPIs
Strategic targets
Mobilize increased
resources
Rationalize, standardize and align routine reporting to MEC, the Committees and the Board
69
Aim of measure
Main audience
Reporting
Corporate KPIs
Board;
Committees;
MEC
Compiled 1-4
times per year
Operational KPIs
MEC;
Secretariat
Compiled 4 times
per year
Thematic reporting
Board;
Committees;
MEC
Compiled
biannually
Management
information
Secretariat;
MEC as
necessary
Regular
monitoring at
divisional level
Current
Focus
Corporate KPIs are only the leading edge of structured routine reporting
70
Corporate KPIs
7 Fund utilization
a) Allocation utilization
b) Absorptive capacity
Human rights
Operational KPIs
Management
Information &
Thematic
Reporting
Financial oversight
RSSH report?
Gender report?
Standardized thematic reporting under each Strategic Objective should provide consistent, quantitative & qualitative
information complementary to KPI Framework Major reporting gaps currently exist
71
Corporate
level
2017
2022
Strategic KPIs
72
Global Funds Pyramidal KPI approach is aligned with other development organizations
Global Fund
Development Banks
GAVI
Operational
Management
Information
Strategic
ActivitySpecific
73