Request for Proposal: 2037-01

A Data Quality Audit of the Vitamin A Supplementation and assessment of the

functionality of the Monitoring and Evaluation system in Indonesia

Issued by Nutrition International “NI” (formerly known as the Micronutrient Initiative)

Deadline for receipt of proposals at the NI:

Thursday, January 9th, 2020

17:00 West Indonesia Standard Time

Page 1 of 27
NutritionIntl.org
RFP No: 2037-01

Contents
1. REQUEST FOR PROPOSAL NOTICE.............................................................................................3
2. INTRODUCTION TO THE RFP ........................................................................................................3
3. GENERAL INSTRUCTIONS AND CONSIDERATIONS ...................................................................4
4. Conflict of Interest ............................................................................................................................4
5. General Disclosures .........................................................................................................................4
6. SUBMISSION OF PROPOSALS ......................................................................................................5
7. RECEIPT, EVALUATION AND HANDLING OF PROPOSALS .........................................................5
8. SELECTION CRITERIA ...................................................................................................................6
9. GUIDELINES FOR PREPARING PROPOSALS ..............................................................................7
ANNEXURE A ..………………………………………………………………………………………11
ANNEXURE B .................................................................................................................................... 21
TEMPLATE OF BUDGET ................................................................................................................... 21

NutritionIntl.org
Page 2 of 27
RFP No: 2037-01

1. REQUEST FOR PROPOSAL NOTICE

1.1 Request for Proposals

Nutrition International (NI) formerly known as the Micronutrient Initiative (MI), a renowned
International Development organization based out in Ottawa, Canada, has a commitment to
eradicate global ‘hidden hunger’ by implementing interventions that focus on women and children
in developing countries. NI invites proposals from competent Consultant or Agencies to conduct “A
Data Quality Audit of the Vitamin A Supplementation and assessment of the functionality of the
Monitoring and Evaluation system in Indonesia”. The submission deadline is Thursday, January
9th, 2020 at 17:00 West Indonesia Standard Time.

2. INTRODUCTION TO THE RFP

2.1 Nutrition International (NI, formerly known as Micronutrient Initiative) has collaborated with
the Ministry of Health (MoH) in supporting the Vitamin A Supplementation (VAS) program
in Indonesia since 2006. Since 2010, NI has been supporting 12 low-performing districts1
in six identified high mortality provinces. The program was focused on supply chain
management, capacity building of the health managers and workers on standardization of
target population estimations, micro-planning, needs assessment, monitoring, and
supervision and supply chain management. Field extender has been placed in the 12
districts to provide hands on technical support to the district Governments. At the national
level, MoH, UNICEF and NI have worked together in assisting local government in
implementing the VAS program.
Since April 2013 to September 2016, NI scaled up its technical and financial support on
VAS program in 6 selected provinces2. Then from October 2016 to March 2019, NI has
scaled up the support to national level with focus to (i) Strengthen government commitment
on VAS program and integrate with other maternal and child health and nutrition programs;
(ii) Provide technical support to improve the quality of VACs to align it with the revised
national specifications which are aligned to the global recommendations, (iii) Support MoH
for improved recording and reporting through simplification and streamlining the QRS and
(iv) Support a Large country Lot Quality Assurance Sampling Study (LC-LQAS) or Post-
Event Coverage Surveys (PECS) for VAS programs.
Further, from April 2019 to March 2024, NI will continue its support on VAS program at
national level to increase the coverage of VAS program and focus to strengthen
government commitment on the VAS program to prioritize budget for VAS program
implementation; Support MoH to improved implementation of the VAS supply chain
management and better monitor the QA VACs through expanding training on the QA/QC
system to additional provinces to better monitor the quality of VACs; Strengthen VAS
program management implementation through provide technical and financial support for
MoH to conduct advocacy workshop to introduce “the Replication Package” which has
been implemented in six NI supported provinces to at least 5 additional non-NI provinces;
Strengthen the BCI strategy and materials on VAS; and Support MoH to review the
implementation of electronic recording and reporting integrated nutrition system (E-

1
Serdang, Berdagai, Binjai, Indragiri Hilir, Dumai, Musi Banyu Asin, Muara Enim, Lebak, Tangerang, Purwakarta,
Tasikmalaya, Bima and Sumbawa
2
North Sumatera, Riau, South Sumatera, Banten, West Java and West Nusa Tenggara
NutritionIntl.org
Page 3 of 27
RFP No: 2037-01

PPGBM) to identify any gaps with the intention of then supporting MoH to improve the
quality of data used to manage the VAS program.

2.2 This Request for Proposals (RFP) and particularly the Guidelines for Preparing Proposals
that follow, are designed to help Respondents to produce proposals that are acceptable to
the NI, and to ensure that all proposals are given equal consideration. It is essential,
therefore, that respondents provide complete information, in the attached formats and on
the terms specified.

3. GENERAL INSTRUCTIONS AND CONSIDERATIONS

3.1. These instructions should be read in conjunction with information contained in the
enclosed Terms of Reference (TOR), and in any accompanying documents within this
package.

This Request for Proposals (RFP) is to support NI to conduct an “Assessment of the

acceptance among health providers and caregivers and the effectiveness of the BCI
materials to increase caregiver awareness and knowledge about vitamin A
supplementation and the August 2018 round of VAS in four provinces of Banten, Riau,
West Java and West Nusa Tenggara of Indonesia”.
3.2. NI is not bound to accept the lowest priced, or any, proposal. NI reserves the right to
request any (or all) Respondent(s) to meet with the NI to clarify their proposal(s) without
commitment, and to publish on its website answers to any questions raised by any
Respondent (without identifying that Respondent).

3.3. Respondents are responsible for all costs associated with the proposal preparation and
will not receive any reimbursement from the NI.

4. Conflict of Interest

4.1. Respondents must disclose in their proposal details of any circumstances, including
personal, financial and business activities that will, or might, give rise to a conflict of
interest. This disclosure must extend to all personnel proposed to undertake the work.

4.2. Where Respondents identify any potential conflicts, they must state how they intend to
avoid any impact arising from such conflicts. NI reserves the right to reject any proposals
which, in the NI’s opinion, give rise, or could potentially give rise to, a conflict of interest.

4.3. With respect to this condition, please be advised that the organizations that may fall within
the scope of this evaluation will include those below, with which any association must be
disclosed:
a. Nutrition International (NI)
b. The Donor who is the primary funding source for the procurement
5. General Disclosures
5.1. Respondents must disclose:

NutritionIntl.org
Page 4 of 27
RFP No: 2037-01

5.1.1. If they are or have been the subject of any proceedings or other arrangements
relating to bankruptcy, insolvency or the financial standing of the Respondent
including but not limited to the appointment of any officer such as a receiver in
relation to the Respondent personal or business matters or an arrangement with
creditors or of any other similar proceedings.

5.1.2. If they have been convicted of, or are the subject of any proceedings, relating to:
a. Criminal offence or other offence, a serious offence involving the activities of
a criminal organization or found by any regulator or professional body to have
committed professional misconduct.
b. Corruption including the offer or receipt of any inducement of any kind in
relation to obtaining any contract, with NI, or any other contracting body or
authority
c. Failure to fulfil any obligations in any jurisdiction relating to the payment of
taxes.

6. SUBMISSION OF PROPOSALS
6.1 The Technical (only includes detail of meeting arrangement, team composition and timeline)
and financial proposal along with all requisite documentation must be received in English by
NI no later than Thursday, January 9th, 2020.

6.2 The technical and financial proposal in two separate files shall be put into a covering email
specifically indicating with subject line: “A Data Quality Audit of the Vitamin A
Supplementation and assessment of the functionality of the Monitoring and Evaluation
system in Indonesia” need to be sent to email: proposalsindonesia@nutritionintl.org

6.3 For any clarification required, please write an email on following email address:
proposalsindonesia@nutritionintl.org

6.4 Only email bids will be accepted. Only those short-listed will receive an acknowledgment
and will be called for a personal interaction, at their own cost. The interaction will be held
at the NI office in Jakarta, Indonesia.

6.5 Late proposals will not be accepted in any circumstances. Proposals received after the
due date and time will not be considered.

7. RECEIPT, EVALUATION AND HANDLING OF PROPOSALS

7.1. Once a proposal is received before the due date and time, NI will:

7.1.1. Log the receipt of the proposals and record the business information
7.1.2. Review all proposals and disqualify any non-responsive ones (that fail to meet the
terms set out in these instructions), and retain the business details on file with a
note indicating disqualification
7.1.3. Evaluate all responsive proposals objectively in line with the criteria specified
below
7.1.4. Inform Respondents within 15 business days of the evaluation decision being
made.

NutritionIntl.org
Page 5 of 27
RFP No: 2037-01

7.2. NI reserves the right:

7.2.1. To accept or reject any and all proposals, and/or to annul the RFP process prior
to award, without thereby incurring any liability to the affected Respondents or any
obligation to inform the affected Respondents of the grounds for NI's actions prior
to contract award, and

7.2.2. To negotiate - with Respondent(s) invited to negotiate - the proposed technical

approach, and the proposed price based on the Respondent’s proposals.

7.2.3. Amend this RFP at any time.

8. SELECTION CRITERIA
8.1. 60% of the evaluation weighting will be for the technical and 40% for the financial
proposal. For detail, please see table 1.

8.2. The Evaluation Team may, in its sole discretion, establish a short-list of Respondents
based on the Technical Scores of the Respondents (the “Short-listed Respondents”) for
the purpose of conducting interviews. If NI short-lists the Respondents, it will short-list
the Respondents with the highest scores.

8.3. Only the Short-listed Respondents will be interviewed. The number of Respondents
short-listed for an interview is in the sole discretion of NI.

8.4. Interviews of Short-listed Respondents will be carried out by the Evaluation Team or a
sub-group of the Evaluation Team. The Evaluation Team will score each Short-listed
Respondent based on the quality of the Respondent’s interview (the “Interview Score”).

8.5. The successful Respondent will be expected to enter into a contract with the NI
for the duration of the Work. In the event of a Contract award, all the terms and
conditions of the RFP, including the Respondent’s response, will normally form part
of the Contract.

NutritionIntl.org
Page 6 of 27
RFP No: 2037-01

"A Data Quality Audit of the Vitamin A Supplementation and assessment of the
functionality of the Monitoring and Evaluation system in Indonesia"
Request for Proposal No: 0143-01
Proposals
Assessment Category: Presentation and discussion Weights
Agency-1 Agency-2 Agency-3
Capacity of the firm (A) 50% 0,00 0,00 0,00
Adequate Professionals / Staff / Resources: The
survey team consists of: MCH expert with more than
5 years of experience in conducting maternal and
newborn health research and programs, Statistician 20%
with more than 5 years of experience in data
management and expert in the in qualitative data
collection, transcription, coding and interpretation.
Recognition and acceptance of firm among
30%
stakeholders
Firm's ability to complete preparatory work
30%
(recruitment, training and finalization of tools)
Firm's flexible approach in taking feedback 20%

Realistic Plan and ability to accomplish the task (B) 50% 0,00 0,00 0,00
Firm's clarity about methodology to be adopted to
20%
undertake the study
Firm's plan for completing the study 25%
Firm's ability to complete the task as per NI
25%
expectation mentioned in the TOR
Firm's realistic timelines to accomplish the entire
30%
work
Total Score - Technical Proposal (A+B) 100% 0,00 0,00 0,00
Overall weightage of Technical Proposal – 60%
Assessment Category: Financial
Presented reasonable estimate of fees 25%
Takes into consideration all potential field expenses
30%
(i.e. no obvious omissions)
Presented realistic estimate for field expenses 25%
Reasonable estimate for completing the meeting, and
20%
report writing expenses
Total Score - Financial Proposal 100% 0,00 0,00 0,00
Overall weightage of Financial Proposal – 40%

Total Weighted Score (Technical + Financial) 0,00 0,00 0,00

9. GUIDELINES FOR PREPARING PROPOSALS

9.1 Language: Proposals must be submitted in English.

NutritionIntl.org
Page 7 of 27
RFP No: 2037-01

9.2 Structure : Proposals must be set out in four main parts:

Part 1: Covering Letter and Declaration
Part 2: Executive Summary
Part 3: General and Technical Proposal
Part 4: Financial Proposal

Part 1: Covering Letter and Declaration

Proposals must be accompanied by a covering letter on company-headed paper showing the full
registered and trading name(s), trading and registered office address and business number of the
Respondent. The letter must be signed by a person of suitable authority to commit the Respondent
to a binding contract. It must quote the RFP number and title, and include the following declarations:

a. We have examined the information provided in your Request for Proposals (RFP) and
offer to undertake the work described in accordance with requirements as set out in the
RFP. This proposal is valid for acceptance for 6 months and we confirm that this
proposal will remain binding upon us and may be accepted by you at any time before
this expiry date.

b. We accept that any contract that may result will comprise the contract documents issued
with the RFP and be based upon the documents submitted as part of our proposal.

c. Our proposal (Technical and Financial) has been arrived at independently and without
consultation, communication, agreement or understanding (for the purpose of restricting
competition) with any other Respondent to or recipient of this RFP from the NI.

d. All statements and responses to this RFP are true and accurate.

e. We understand the obligations regarding Disclosure as described in the RFP Guidelines

and have included any necessary declarations.

f. We confirm that all personnel named in the proposal will be available to undertake the
services.

g. We agree to bear all costs incurred by us in connection with the preparation and
submission of this proposal and to bear any further pre-contract costs.

h. I confirm that I have the authority of [insert name of NGO/company/agency] to submit

this proposal and to clarify any details on its behalf.

Part 2: Executive Summary

A brief overview of the General and Technical proposal that summarizes how the Respondent will
use their competencies in the area to achieve the outputs/deliverables. Financial information should
not be included here; but the summary may indicate the level of effort proposed.

NutritionIntl.org
Page 8 of 27
RFP No: 2037-01

Part 3:

General and Technical Proposal

The General and Technical section should be structured as follows:

▪ Section 1: Your understanding of the TOR provided with this RFP as Annexure A. You may
also propose qualifications to the TOR that you consider may enhance the value of the outcome
to the NI.

▪ Section 2: Technical Response: a concise description of the methodology and approach that
are proposed for the delivery of the TOR and an implementation plan in the form of a work
breakdown analysis. This should describe the activities to be undertaken, the
deliverables/outputs and the milestone and completion dates (grouped by phase where
appropriate). The dependency of any activities and associated results on earlier results needs
to be clearly indicated.

▪ Section 3: Personnel Profile: names, designation and Curricula Vitae (CV) of personnel
assigned to work on the Project. CVs must not exceed 3 pages, but must include:
o a brief summary of the professional competencies of the individual relevant to the
Scope of Work/ToR
o a chronological list of relevant professional experience starting with the most recent
and showing key achievement /responsibilities
o brief details of qualifications educational / technical /professional / other
o language competencies other than English (if required to undertake the ToR)

• Section 4: Personnel Inputs: include name of personnel, and person days with reference
to activity to be undertaken. Do not include any reference to fees. This will constitute a
confirmation that all personnel will be available to provide the required services for the
duration of the contract

• Section 5: Company Information: proof of incorporation for registered incorporated

entities, proof of registration for registered entities.

• Section 6: Previous experience: documentation demonstrating the Respondent’s

experience in the proposed area of work. This should include contact details for key clients
who may be contacted in respect of the Respondent’s relevant prior work.

Part 4: Financial Proposal

• The Financial proposal must contain the expected budget for accomplishing the complete
work with detailed breakup. All amounts quoted must be in IDR. The Respondent
should provide a detailed budget, based on the format attached as Annexure-B.

• Fees should be inclusive of all VAT, Taxes, Insurance and standard training
management/overhead cost.

• Please note that no fees are payable for travel days.

NutritionIntl.org
Page 9 of 27
RFP No: 2037-01

ANNEXURE – A

Terms of reference

A Data Quality Audit of the Vitamin A Supplementation and assessment of the functionality
of the Monitoring and Evaluation system in Indonesia

Introduction
Nutrition International (NI, formerly known as Micronutrient Initiative) has collaborated with the
Ministry of Health (MoH) in supporting the Vitamin A Supplementation (VAS) program in Indonesia
since 2006. Since 2010, NI has been supporting 12 low-performing districts3 in six identified high
mortality provinces. The program was focused on supply chain management, capacity building of
the health managers and workers on standardization of target population estimations, micro-
planning, needs assessment, monitoring, and supervision and supply chain management. Field
extender has been placed in the 12 districts to provide hands on technical support to the district
Governments. At the national level, MoH, UNICEF and NI have worked together in assisting local
government in implementing the VAS program.
Since April 2013 to September 2016, NI scaled up its technical and financial support on VAS
program in 6 selected provinces4 with the focus on improving the following: (i). health facility staff
and frontline workers identify and reach the unreached children (who do not come to posyandu) to
ensure that all children are provided with the vitamin A capsule twice a year; (ii). Health service
providers effectively plan, implement, monitor and supervise the on-going VAS program; (iii). District
and sub-district level health officials use the modified MIS, improve recording and reporting and use
the data for corrective actions and decision making; (iv). Ministry of Health, GoI quality assure the
vitamin A capsule supply and streamline distribution mechanisms for uninterrupted supply from
central, province and district health office to posyandu/clinics; (v). Province level health officials
scale up and replicate the VAS program strategy to the remaining districts based on learning from
the MI supported districts and; (vi). national level officials integrate best practices which
demonstrate the VAS program strategy is effective and integrate the same in the VAS national
guidelines.
Then from October 2016 to March 2019, NI has scaled up the support to national level with focus
to (i) Strengthen government commitment on VAS program and integrate with other maternal and
child health and nutrition programs; (ii) Provide technical support to improve the quality of VACs to
align it with the revised national specifications which are aligned to the global recommendations,
(iii) Support MoH for improved recording and reporting through simplification and streamlining the
QRS and (iv) Support a Large country Lot Quality Assurance Sampling Study (LC-LQAS) or Post-
Event Coverage Surveys (PECS) for VAS programs.
Further, from April 2019 to March 2024, NI will continue its support on VAS program at national
level to increase the coverage of VAS program and focus to strengthen government commitment
on the VAS program to prioritize budget for VAS program implementation; Support MoH to improved
implementation of the VAS supply chain management and better monitor the QA VACs through
expanding training on the QA/QC system to additional provinces to better monitor the quality of

3 Serdang, Berdagai, Binjai, Indragiri Hilir, Dumai, Musi Banyu Asin, Muara Enim, Lebak, Tangerang,
Purwakarta, Tasikmalaya, Bima and Sumbawa
4 North Sumatera, Riau, South Sumatera, Banten, West Java and West Nusa Tenggara

NutritionIntl.org
Page 10 of 27
RFP No: 2037-01

VACs; Strengthen VAS program management implementation through provide technical and
financial support for MoH to conduct advocacy workshop to introduce “the Replication Package”
which has been implemented in six NI supported provinces to at least 5 additional non-NI provinces;
Strengthen the BCI strategy and materials on VAS; and Support MoH to review the implementation
of electronic recording and reporting integrated nutrition system (E-PPGBM) to identify any gaps
with the intention of then supporting MoH to improve the quality of data used to manage the VAS
program.

The rationale of this study

MoH is currently implementing a computerized recording and reporting system (e-PPGBM) for the
national nutrition program. This e-PPGBM system is set up to capture weight, height, nutritional
status, as well as VAS coverage (among other nutrition interventions), of all children U-5 by name,
and by address. It is a large undertaking and is challenging for the MoH to collect the data from all
34 provinces and report it in a timely and complete manner. To date, the total number of children
U-5 that has been inputted in the e-PPGBM is around 56% of the total target. Considering that the
completeness, timeliness and compliance to data management and adherence to SoPs on VAS
program is one of the challenges at all levels (national, province, district and village level), NI would
like to support MoH to conduct a Data Quality Audit (DQA) of the e-PPGBM information system with
a focus on the VAS data. This data quality audit aims to help the MoH to identify where the gaps
are occurring, and generate the detailed evidence needed to advocate to the PHO and DHOs to
improve the recording and reporting system for the VAS and nutrition program at all levels.
Study Objectives
The objective of the study is to perform a data quality audit to assess the functionality of the M&E
system and data reporting mechanism at different levels viz., posyandu, puskesmas, DHO and PHO
and Central or National level. Specifically, the assessment will help identify possible gaps, errors
or other data quality issues, if any, in reporting at each level during the data entry and roll-up process
and inform process improvements. In addition, it will include a validation of the target population
denominators of eligible children for vitamin A supplementation.

Background of the administrative structure of Indonesia: Indonesia with a population of 264 million,
is an archipelago consisting of approximately 16,056 islands. Based on the Regulation of the
Minister of Home Affairs Indonesia is administratively divided into 34 provinces, 514
districts/municipalities (comprising 416 districts and 98 municipalities), 7,094 sub-districts, 8,412
administrative villages (Indonesian: kelurahan) and 74,093 villages. As of December 2013, there
were 9,754 puskesmas in the country, which provide basic and additional health programs. A
puskesmas covers a number of Desa. A Desa is divided into several Dusuns and a Kelurahan is
divided into several community groups or Rukun Warga (RW) and one Rukun Tetangga (RT) is
divided into several neighbourhood groups or RT based on population, reach and terrain.

Health system in Indonesia: Following decentralization reform in 1999, health services were
decentralized to provincial and district governments which are under the Ministry of Home Affairs.
This change resulted in decentralization of responsibility for planning and managing service delivery
from the Ministry of Health to local governments. The central Ministry of Health (MoH) retained and
continued to operate some tertiary and specialist hospitals, but has shifted much of its function to
regulation, ensuring the availability of resources, including personnel, and to taking an increasingly
lead role in the supervision of social insurance schemes. There are also several other central
government ministries and agencies involved in the health sector; for instance, the Ministry of Home
Affairs, the Social Security Managing Agency, and National Board of Population and Family
NutritionIntl.org
Page 11 of 27
RFP No: 2037-01

Planning. The health service at the local level is divided between provincial level and
district/municipality level. Provincial governments own the provincial hospitals and organize the
health services through the provincial health offices (PHOs). The PHOs play a coordinating role for
health issues within the provincial/region and across districts. The district/municipality governments
own district/municipality hospitals and organize health services through district/municipality health
offices (DHOs). DHOs also operate health services provided through the primary health centres
(puskesmas) and their networks. However, the relationship between MoH, PHO and DHO is not a
hierarchical one.

Vitamin A is delivered twice a year, during the months of February and August, at the Posyandu
level. The vitamin A data reporting flowchart in the country is presented as follows:

Ministry of
Health, National
Provincial level
Health Office The provincial data is
rolled up from the 34
District Health (PHO) provinces.
Office (DHO) Provincial health office,
where data is rolled up
Puskesmas • District office, where from the district level
data is rolled up from
• Primary health care the Puskesmas level
centres, where data is
Posyandu rolled up from the
• the posyandu is a community Posyandu
initiative to engage in health
promotion and preventive
activities. The backbone of
posyandu are the health
cadres, community volunteers
who are trained by village
midwives and/or other
puskesmas staff to provide
assistance in promoting healthy
living behaviour, mother and
child health, and nutrition to the
community.

Study Methodology

For this study, it is suggested to audit the data for the following aspects by validating both the
denominators and numerators and process and time of data roll up:
1. Audit the data on population of children in the age group of 6 to 11 months and 12 to 59
months to verify with the targets at the Posyandu level and above.
2. Audit the data on vitamin A for those children in the age group of 6 to 11 months and 12 to
59 months to assess the quality of the reporting at the Posyandu level and above. This will
be done by comparing the data recorded in the source documents against the summaries
submitted to the next level This will be validated from the registers in the second phase,
when the data quality audit at the facility level is being conducted, not from the field.

3. Audit of data rolled up from the Posyandu to the Puskesmas to the district (District Health
Office) and to the province (Provincial Health Office) to the national level.

Suggested sample size for data quality audit of the target population denominators of children under
five years of age:

NutritionIntl.org
Page 12 of 27
RFP No: 2037-01

Usually, population projections are estimated up to the province or district leveI based on the
previous decadal censuses, which is not available at the lowest administrative levels (village / Desa
/ Kelurahan level), thus the age proportions may vary from the province and the district level.

It is proposed to select three provinces, covering three regions for representation and applicability,
and it is proposed to select one district and one city randomly from each of the provinces from the
three regions. Select one Desa from each of the puskesmas in the district and one Kelurahan each
from each of the puskesmas in the city for audit of the data and to verify the age proportions and
the target group for vitamin A supplementation. It is also proposed to do a census of the population
of different age groups in two Desa / two Kelurahan (including the posyandu) catchment area, each
in the three provinces to arrive at the actual population of 0 to 59 months, 6 to 11 month, 12 to 59
months within the total population that includes other age groups, and check how many of them
received vitamin A in this round, while verifying the registers during the facility audit. The age groups
for which data are to be collected are 0 to 5m, 6 to 11m, 12 to 23m, 24 to 59m 5 among other
population age groups as the census was conducted nine years before in 2010 and the age
proportions might have changed by now. It is proposed to verify the due list or register of children
expected to be supplemented with vitamin A, present with the Cadre at the posyandu with the list
thus generated in the census to validate the targets and coverage of children to be supplemented
with vitamin A at the Desa / Kelurahan and Posyandu and inform the Cadre during the round
(February 2020), if any infant or children is left out or missed. The registers at the puskesmas level,
DHO and PHO and central level also needs to be reviewed and data to be collected and compiled
along with the e-PPGBM data. A deviation score could be computed between the two sets of data;
one from the cadre register and the other from the census. A factor of left out and a factor of missed
out could be computed to assist in future VAS programming.

Nutrition International Indonesia has already developed a quality reporting self-assessment tool,
which would be used in addition to the DQA tool from measure evaluation for the purpose of data
quality audit. The tool is a follows:

Quality Reporting Self-assessment (QRS) Tool developed by Nutrition International, Indonesia

Country Office

Accuracy & QRS of

Vitamin A data_SubditGizi_August_2017.xlsx

DQA tool

Routine Data
Quality Assessment Tool_Jan 2010.xls

Figure 1 : Vitamin A supplementation data quality audit and census at different levels in 12 Desa
and 12 Kelurahans

5The population of the other age groups could also be collected like; 60 months to 72 months, 6 to 9 years, 10 to 14
years, 15 to 19 years, 20 to 49 years, 50 to 59 years, 60+ years by sex. The pregnancy status of the women in the age
group of 15 to 49 years could also be ascertained. This could be useful to validate this population data with the census
age wise classification, and proportions of the total population.
NutritionIntl.org
Page 13 of 27
RFP No: 2037-01

Vitamin A data quality audit and assessment of the functionality of the monitoring and
evaluation system in Indonesia

National / Central level

Eastern Central Western

Region Region Region

Province 1 Province 2 Province 3

District City District City District City

2 Puskesmas 2 Puskesmas 2 Puskesmas 2 Puskesmas 2 Puskesmas 2 Puskesmas

2 Desas 2 Kelurahans 2 Desas 2 Kelurahans 2 Desas 2 Kelurahans

All Posyandu All Posyandu All Posyandu

All Posyandu in the 2 All Posyandu in All Posyandu in the 2
in the 2 in the 2 in the 2
Desas the 2 Desas Desas
Kelurahans Kelurahans Kelurahans

Data Collection
Census of the target group of children 6 to 11 months and 12 to 59 months of age in the Desa
covering all the Posyandu in the 6 Desa and 6 Kelurahan.
DQA process:
During the DQA, multiple response technique will be used, including;
• Physical counts-This will involve counting of the cases in the source and summary
documents.
• Cross checking -This will include cross checking of the primary records against other data
source.
• Observation/spot checks - Entails verification of the actual delivery of services or
commodities to the target population
• Self-reporting – This will involve one on one interviews with staff on various aspects of
reporting

The DQA tool from MEASURE evaluation will be adapted and modified for this DQA exercise. Using
the DQA tool the following areas will be reviewed:
Data verification
a) Documentation review – This will entail review of availability and completeness of all
indicator source documents for the reporting period under DQA
NutritionIntl.org
Page 14 of 27
RFP No: 2037-01

b) Recounting reported data- The team will recount entries from source documents (registers,
tally sheets), compare the verified numbers to the site reported numbers (monthly
summaries aggregate) and identify plausible causes of the discrepancies (if any).
c) Validate the primary data source against a secondary data source for the same reporting
period
d) Cross-check reported data with secondary data sources: Cross-checks will be performed by
examining secondary data sources (registers, inventory records)

Systems Assessment (Suggested questions are included in Annexure 2)

a) M&E Structure, Functions and Capabilities - DQA will review if all staff understand their role
in reporting and whether capacity building has been provided.
b) Indicator Definitions and Reporting Guidelines - The team will check availability/access of
written guidelines to each reporting level on report structure, reporting timelines, reporting
formats, and report submission flow.
c) Data Collection and Reporting Forms and Tools - Check availability of source documents
and reporting forms.
d) Data Management Processes - This will include review of data quality controls in place, data
storage and data handling process.
e) Links with National Reporting System - Review if the nationally adopted tools and reporting
systems are being used for reporting.

Recommendations for the Posyandu

Based on the findings of the systems’ review and data verification at the service site, the DQA team
will identify challenges/weaknesses to data quality and provide recommended strengthening
measures with clear timelines. Initial recommendations will be provided on site based on preliminary
findings. Detailed recommendations will be provided upon in-depth analysis of the data.
The MEASURE evaluation RDQA tool will automatically generate summary graphs of assessment
data, the following summary indicators could be calculated:
❑ Strength of the data management and reporting System based on a review of the
programme’s data collection and reporting system
❑ Accuracy of reported data through the calculation of verification factors generated
from the recounting exercise performed at each level of the reporting system
❑ Availability, Completeness and Timeliness of reports through percentages
calculated at the province level

NutritionIntl.org
Page 15 of 27
RFP No: 2037-01

Proposed tentative timelines

The data quality audit is proposed to be done prior to (census), during and following up to two-three
months of the February 2020 vitamin A supplementation 1st round, once the data is rolled up to the
national level from the Posyandu. On an average, the Posyandu reports to Puskesmas one month
after distribution at posyandu and sweeping or by March/April for the 1st round and September or
October for the 2nd round in August). It might take some time for the data to be rolled up from the
Puskesmas to the district and to the Province to be finally compiled at the national level. Therefore,
this activity is proposed to be conducted from January to September 2020.

Deliverables
The following deliverables are to be submitted in hard copy and electronic form by the agency to
the Nutrition International, as the implementation progresses:
• IRB clearance certificate, including NI’s access to the raw de-identified dataset, and
training certificates for all lead team members
• Study tools in both English and Bahasa versions
• Final data audit tools or questionnaires/MIS tools with ODK/CAPI tools and results of
tool testing, for approval by NI prior to commencing training
• Roll-out plan for training and data collection
• Training modules for the field team including interviewers and supervisors and their
training report
• List of Posyandu, Desa, Kelurahan, District, City, DHO and PHO sampling and data
collection plan, data analysis plan, indicator tabulation plan, weekly reporting system
and data quality audit plan.
• Final data analysis plan and draft report outline with dummy tables, for approval by NI
• Final reports both hard and soft copies
o Province-specific reports (3 colour hard copies per province)
o Province-specific factsheet/Infographics- 2 pages- 3 colour hard copies per
province
o Combined Report (3 colour hard copies)
o Combined Factsheet/ Infographics- 4 pages- 3 colour hard copies per province
• Cleaned and labelled datasets in excel and SPSS format and its syntax for outputs and also
generated variables
• Power Point presentation (Province-specific and combined)

References
1. Data Quality Review: Module 1 : Framework and Metrics
2. https://apps.who.int/iris/bitstream/handle/10665/259224/9789241512725-
eng.pdf?sequence=1&isAllowed=y
3. Data Quality Review: Module 2: Desk Review of data quality
4. https://apps.who.int/iris/bitstream/handle/10665/259225/9789241512732-
eng.pdf?sequence=1&isAllowed=y
5. Data Quality Review: Module 3: Data verification and system assessment
6. https://apps.who.int/iris/bitstream/handle/10665/259226/9789241512749-
eng.pdf?sequence=1&isAllowed=y

NutritionIntl.org
Page 16 of 27
RFP No: 2037-01

7. Data quality review toolkit and the spreadsheet for RQDA from measure evaluation
(https://www.measureevaluation.org/resources/tools/data-quality/RDQA%20Multi-
Indicator%202015.xls/at_download/file)
8. Multiple indicator RQDA tool :https://www.measureevaluation.org/resources/tools/health-
information-systems/data-quality-assurance-
tools/Routine%20Data%20Quality%20Assessment%20Tool_Jan%202010.xls
9. Single indicator RQDA tool :https://www.measureevaluation.org/our-work/data-
quality/data-quality-review-tool.

NutritionIntl.org
Page 17 of 27
 RFP No: 2037-01

Annexure 1
Table 3 : Example of denominator validation : Estimated number of infants / children under five years based on the
projected population per 100 and 1,000 population (WPP, 2017)
Country Children (0 Children (6 to Infants (6 Children Children Children (6 Infants (6 Children (12
to 59 59 months) to 11 (12 to 59 (0 to 59 to 59 to 11 to 59
months) per per 100 months) months) months) months)/ months) months) per
100 per 100 per 100 per 1,000 per 1,000 per 1,000 1,000
Indonesia 9.4 8.4 1.0 7.5 94 84 10 75
Source: Computations based on United Nations, Department of Economic and Social Affairs, Population Division (2017).
World Population Prospects: The 2017 Revision, Key Findings and Advance Tables. Working Paper No. ESA/P/WP/248
Under the assumption that the proportion of population at each month is uniformly distributed within 0 to 59 months age
group, which may or may not be applicable everywhere. The national level population proportions might vary by province,
district and Desas/ Kelurahan.

Table 4: Estimated numerator validation of population of 6 to 11 months and 12 to 59 months as an example

Desa/ Kelurahan Assumed total Total population of Total population of Total population of Total population of
population of a infants 6 to 11 infants 6 to 11 infants / children 12 infants / children
Desa/ Kelurahan months enumerated months receiving to 59 months 12 to 59 months
in the census of the vitamin A in the enumerated in the receiving vitamin
Desa/ Kelurahan, round (February census of the Desa/ A in the round
2019 2020) @ 100% Kelurahan 2019 (February 2020)
@ 100%
1 1,000 10 10 75 75
2 2,000 20 20 150 150
3 5,000 50 50 375 375
4 10,000 100 100 750 750
5 15,000 150 150 1,125 1,125

Table 5 : Tabulation format for the census of the Desa/ Kelurahan: The agency needs to report the total number of
eligible children 6 to 11 months and 12 to 59 months enumerated in census in the catchment area and mention the
number of children covered by vitamin A supplementation in the February 2020 round.
Desa/ Dusun RW RT Total population of Total population of Total population of Total population
Kelurahan infants 6 to 11 months infants 6 to 11 months infants / children 12 to of infants /
enumerated in the receiving vitamin A in 59 months enumerated children 12 to 59
census of the Desa, the round (February in the census of the months receiving
2019 2020) Desa 2019 from the vitamin A in the
register round (February
2020) from the
register
1
.
12

Table 6: Data quality audit at all the levels: The agency needs to document the coverage at each level to see, if there is
any discrepancy, while rolling up of the data from the lower to the higher level.
Province District/ Sub-district Desa/ Posyandu Total Total Total Total Total
City (Kecamatan) Kelurahan population population population of population population of
of infants infants 6 to of infants / infants /
6 to 11 11 months children children 12
months receiving 12 to 59 to 59 months
with the vitamin A in months receiving
cadre the round with the vitamin A in
(February cadre the round
NutritionIntl.org
Page 18 of 27
 RFP No: 2037-01

2020) from (February

the register 2020) from
the register

1
.
24

Table 7 : Time taken to collate and roll up the reports: The agency needs to document the time duration it takes to roll
up the report from the Posyandu to the Puskesmas and from the Pusekesmas to DHO, PHO and MoH at the central
level
Sl. No. Posyandu Desa/ Kelurahan Puskesmas DHO PHO MoH
1 February 2020 February 2020
.
.

Table 8: Age distribution of the population of the Desa in broad age groups: The agency needs to provide data of
population from the census enumeration for the following age groups, which can be validated with the census to see, if
there is any change.
Desa/ 0 to 5 6 to 11 12 to 23 24 to 59 60 to 72 6 to 9 yeas 10 to 14 15 to 19 20 to 49 50 to 59 60+ years Total
Kelur months months months months months years years years years
ahan

1 M F T M F T M F T M F T M F T M F T M F T M F T M F T M F T M F T M F T
.
.
12

Annexure 2

Table 9: Assessing the functionality of Monitoring and evaluation system

FUNCTIONAL AREAS SUMMARY QUESTIONS
I M&E Structures, Functions 1 Are key M&E and data-management staff identified with clearly assigned
and Capabilities responsibilities?

2 Have the majority of key M&E and data-management staff received the required
training?

II Indicator Definitions and 3 Are there operational indicator definitions meeting relevant standards that are
reporting Guidelines systematically followed by all service points?

4 Has the program/project clearly documented (in writing) what is reported to who,
and how and when reporting is required?

III Data Collection and 5 Are there standard data-collection and reporting forms that are systematically
Reporting forms and Tools used?

6 Is data recorded with sufficient precision/detail to measure relevant indicator?

7 Are data maintained in accordance with international or national confidentiality

guidelines?

NutritionIntl.org
Page 19 of 27
RFP No: 2037-01

8 Are source documents kept and made available in accordance with a written
policy?

IV Data Management processes 9 Does clear documentation of collection, aggregation and manipulation steps exist?

10 Are challenges of data quality identified and are there mechanisms in place for
addressing them?

11 Are there clearly defined and followed procedures to identify and reconcile
discrepancies in reports?

12 Are there clearly defined and followed procedures to periodically verify source
data?

V Links with National Reporting 13 Does the data collection and reporting system of the program/project link to the
System National Reporting System?

NutritionIntl.org
Page 20 of 27
RFP No: 2037-01

ANNEXURE B

TEMPLATE OF BUDGET

A Data Quality Audit of the Vitamin A Supplementation and assessment of the functionality of the
Monitoring and Evaluation system in Indonesia

Cost per Total

No Description Quantity Unit Remarks
Unit Cost
A Meeting with MoH
Consultation meeting with MoH
(before the field assessment
and after assessment)
Please list the
Honorarium for PI and CoPIs
member of
B. (PI/team leader, reseacher, co-
reseacher
PI, etc)
team
Project Investigator/Team person-
B.1.
leader days -
person-
B.2 etc.
days -
please add
Administration for Ethical
C package the detail cost
Approval
for the permit

Tools Development and

D
Training
Tools Development and
D.1
Finalization

-
Training for field technical for
D.2
field coordinator -
D.2.1 Local transport person
-
D.2.2 Training materials (survey kit) package
-
D.2.3 Meeting Package person
-
E. DATA COLLECTION
E.1 WESTERN REGION
E.1.1 Transport to province
-

NutritionIntl.org
Page 21 of 27
RFP No: 2037-01

person-
Airfare Jkt - province1 return
trip -
Rent a car (local transport) car-days
-
Transport & accomodation at
E.1.2.
district
Rent a car (local transport) car-days
-
person-
Accomodation researchers
days -
person-
Accomodation field coordinators
days -
Lumpsum researchers (PI, Co- person-
PI, etc) days -
person-
Lumpsum field coordinators
days -
E.1.3 Data Collection District 1
Training of enumerators
person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
-
Census district 1
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -
persons-
Fee for enumerator
days -
persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
E.1.4 Data Collection City 1
Training of enumerators
NutritionIntl.org
Page 22 of 27
RFP No: 2037-01

person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
-
Census district 1
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -
persons-
Fee for enumerator
days -
persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
E.2 CENTRAL REGION
E.2.1. Transport to province
person-
Airfare Jkt - province1 return
trip -
Rent a car (local transport) car-days
-
Transport & accomodation at
E.2.2.
district
Rent a car (local transport) car-days
-
person-
Accomodation researchers
days -
person-
Accomodation field coordinators
days -
Lumpsum researchers (PI, Co- person-
PI, etc) days
person-
Lumpsum field coordinators
days
E.2.3. Data Collection District 1

NutritionIntl.org
Page 23 of 27
RFP No: 2037-01

Training of enumerators
-
person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
Census district 1
-
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -
persons-
Fee for enumerator
days -
persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
E.2.4. Data Collection City 1
Training of enumerators
person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
-
Census district 1
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -
persons-
Fee for enumerator
days -

NutritionIntl.org
Page 24 of 27
RFP No: 2037-01

persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
E.3. EASTERN REGION
E.3.1. Transport to province
person-
Airfare Jkt - province1 return
trip -
Rent a car (local transport) car-days
-
Transport & accomodation at
E.3.2.
district
Rent a car (local transport) car-days
-
person-
Accomodation researchers
days -
person-
Accomodation field coordinators
days -
Lumpsum researchers (PI, Co- person-
PI, etc) days
person-
Lumpsum field coordinators
days
E.3.3. Data Collection District 1
Training of enumerators
-
person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
Census district 1
-
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -

NutritionIntl.org
Page 25 of 27
RFP No: 2037-01

persons-
Fee for enumerator
days -
persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
E.3.4 Data Collection City 1
Training of enumerators
person-
Meeting package
days -
person-
Local transport
days -
Field test or try out package
-
Census district 1
person-
Meal
days -
person-
Acommodation for enumerator
days -
Acommodation for field person-
coordinator days -
persons-
Fee for enumerator
days -
persons-
Fee for supervisor
days -
Local Transportation for person-
enumerator days -
Local Transportation for person-
supervisor days -
Souvenir (Contact materials) for
package
respondents -
Allowance for local
persons
counterparts -
Tablet rental tablets
-
F. OFFICE SUPPLIES
Printing and photocopy
(i.e.questionnaire, letter, etc)

NutritionIntl.org
Page 26 of 27
RFP No: 2037-01

Paper, marker, etc (please add

other supplies needed)
INTERNET &
G
COMMUNICATION

DATA MANAGEMENT,
H
ANALYSIS, REPORT
Data cleaning meeting person
-
Analysis & report writing
person
meeting -
Report printing 5 package
-
TOTAL
-
Overhead cost (XX%)
-
TOTAL BUDGET
-

NutritionIntl.org
Page 27 of 27