Designreport Idn GP Cbog Feb18

Evaluation Design Report
MCC Indonesia Green Prosperity Project

Grant Facility Community-Based Off Grid Renewable Energy Grant
Portfolio
August 2017 -
This report was prepared independently by Social Impact, Inc. at the request of MCC.
EVALUATION DESIGN REPORT
MCC Indonesia Green Prosperity Project

Grant Facility Community-Based Off Grid Renewable Energy
Grant Portfolio
v.2
Submitted: August 2017
Submitted to:
Millennium Challenge Corporation
875 Fifteenth Street, NW
Washington, DC 20005-2221
Contracting Officer’s Representative: Vivian Agbegha
Contract MCC-13-BPA-0017, Task Order MCC-17-CL-0005
Submitted by:
Social Impact, Inc.
2300 Clarendon Blvd., Suite 1000
Arlington, VA 22201
703.465.1884
www.socialimpact.com
DISCLAIMER
The authors’ views expressed in this publication do not necessarily reflect the views of the Millennium
Challenge Corporation (MCC) or the United States Government.
EDR: MCC Indonesia GP Grant Facility Window 3A – Renewable Energy, v.1
CONTENTS

1 Introduction & Background .............................................................................................. 9
1.1 Country context ................................................................................................................................ 9
1.2 Objectives of this report .................................................................................................................... 9
2 Overview of Compact & Interventions .......................................................................... 11

2.1 Overview of the Project and Implementation Plan .......................................................................... 11
2.1.1 Original Project Description .................................................................................................... 12
2.1.2 Project Participants and Geographic Coverage ...................................................................... 20
2.1.3 Implementation to Date and Planned Outputs ........................................................................ 26
2.2 Theory of Change ........................................................................................................................... 36
2.3 Cost Benefit Analysis & Beneficiary Analysis ................................................................................. 39
2.4 Literature Review ........................................................................................................................... 40
2.4.1 Summary of Existing Evidence ............................................................................................... 40
2.4.2 Gaps in Literature ................................................................................................................... 42
2.4.3 Policy Relevance of the Evaluation ......................................................................................... 43
3 Evaluation design ........................................................................................................... 45

3.1 Evaluation Questions ..................................................................................................................... 45
3.2 Evaluation Design Overview........................................................................................................... 46
3.2.1 Portfolio Evaluation Design ..................................................................................................... 46
3.2.2 Optional Portfolio Evaluation Add-Ons.................................................................................... 48
3.3 Quantitative Approach .................................................................................................................... 49
3.3.1 Methodology ........................................................................................................................... 49
3.3.2 Timeframe of Exposure........................................................................................................... 49
3.3.3 Study Sample ......................................................................................................................... 52
3.3.4 Primary Data Collection .......................................................................................................... 56
3.3.5 Secondary Data ...................................................................................................................... 60
3.3.6 Analysis Plan .......................................................................................................................... 60
3.4 Qualitative Approach ...................................................................................................................... 63

3.4.1 Methodology ........................................................................................................................... 63
3.4.2 Timeframe of Exposure........................................................................................................... 64
3.4.3 Study Sample ......................................................................................................................... 65
3.4.4 Primary Data Collection .......................................................................................................... 66
3.4.5 Analysis Plan .......................................................................................................................... 69
3.5 Challenges & Limitations ................................................................................................................ 70
3.5.1 Limitations of Interpretations of the Results ............................................................................ 70
3.5.2 Risks to the Study Design ....................................................................................................... 70
4 Administrative ................................................................................................................. 72

4.1 Summary of IRB Requirements and Clearances ............................................................................ 72
4.2 Data Protection............................................................................................................................... 72
4.3 Preparing Data for Public Use ........................................................................................................ 72
4.4 Dissemination Plan ......................................................................................................................... 72
4.5 Evaluation Team Roles and Responsibilities.................................................................................. 73
4.6 Evaluation Timeline and Reporting Schedule ................................................................................. 74
5 References ....................................................................................................................... 75
6 Annexes ........................................................................................................................... 80

6.1 Annex 1: Stakeholder Comments and Evaluator Responses ......................................................... 80
6.1.1 MCA-I Comments and Evaluator Responses ......................................................................... 80
6.1.2 MCC Evaluation Management Committee Comments and Evaluator Responses .................. 95
6.2 Annex 2: Evaluation Budget ........................................................................................................ 103
6.3 Annex 3: Instruments ................................................................................................................... 104
6.3.1 Household Questionnaire ..................................................................................................... 104
6.3.2 SPV Leadership KII Protocol ................................................................................................ 136
6.3.3 Village Official KII Protocol .................................................................................................... 138
6.3.4 Regency (sub-district) Official KII Protocol ............................................................................ 146
6.3.5 Grantee KII Protocol ............................................................................................................. 151
SOCIALIMPACT.COM ii
6.3.6 Community Beneficiary FGD Guide ...................................................................................... 153

6.3.7 Enterprise KII Guide.............................................................................................................. 156
SOCIALIMPACT.COM iii
TABLES & FIGURES
TABLES
Table 1: CBOG RE Grants Signed (as of August 16, 2017) .......................................................................... 14
Table 2: W3A Akuo Energy Solar/Micro-Hydro, Berau Summary of Physical Outputs .................................. 26
Table 3: W3A Charta Putra Biomass, Siberut Island Summary of Physical Outputs ..................................... 29
Table 4: W3A Anekatek Solar, East Sumba Summary of Physical Outputs .................................................. 30
Table 5: W3A Sky Energy Solar, Karampuang Island Summary of Physical Outputs ................................... 33
Table 6: ERR for each of the Window 3A Grants .......................................................................................... 39
Table 7: Evaluation Design Overview ............................................................................................................ 47
Table 8: Possible Add-On Lines of Inquiry .................................................................................................... 49
Table 8: Power Calculation Summary Statistics percent ............................................................................... 53
Table 9: Summary of instruments, respondents, and estimated respondent numbers per treatment unit
(Qualitative Questionnaire) ............................................................................................................................ 68
Table 10: Categorization of threats to identification of impacts, and mitigation strategy ............................... 71
Table 11: Evaluation Team ........................................................................................................................... 73
Table 12: Evaluation Timeline and Reporting Schedule ................................................................................ 74
FIGURES
Figure 1: Map of Target Villages for W3A Akuo Energy Solar/Micro-Hydro, Berau ....................................... 20
Figure 2: Map of Target Villages for W3A Charta Putra Biomass, Siberut Island .......................................... 21
Figure 3: Map of Target Sub-Villages for W3A Anekatek Solar, East Sumba ............................................... 22
Figure 4: Map of Target Villages for W3A Puriver Solar, Tomia Island ......................................................... 23
Figure 5: Map of Targeted Village for W3A Sky Energy Solar, Karampuang Island ...................................... 23
Figure 6: W3A Akuo Energy Solar/Micro-Hydro, Berau SPV Organigram ..................................................... 27
Figure 7: W3A Akuo Energy Solar/Micro-Hydro, Berau SPV Profit Utilization ............................................... 28
Figure 8: W3A-56-58 SPV Organization and Management ........................................................................... 30
Figure 9: W3A Anekatek Solar, East Sumba SPV Organigram ..................................................................... 32
SOCIALIMPACT.COM iv
Figure 10: Log Frame for W3A Anekatek Solar, East Sumba ....................................................................... 38
Figure 11: Relationship between cluster size and MDES .............................................................................. 53
ACRONYMS
ADB Asian Development Bank
AEI Akuo Energy Indonesia
APDS Asosiasi Periau Danau Sentarum
ARI Acute respiratory infection

BA-SBT Bentang Alam Sumba Bagian Tengah
BI Bahasa Indonesian
BPS Badan Pusat Statistik
BUMD Badan Usaha Milik Daerah
BUMDes Badan Usaha Milik Desa
CBA Cost benefit analysis
CBOG Community-Based Off Grid
CEM Coarsened exact matching
CEO Chief Executive Officer
CIA Conditional Independence Assumption
CIFOR Center for International Forestry Research
CoK Center of Knowledge
CPI Charta Putra Indonesia
CUKK Koperasi Kredit Keling Kuman
DFS Detailed Feasibility Study
DiD Difference in differences
DRB Disclosure Review Board
EDR Evaluation Design Report
SOCIALIMPACT.COM v
EPC Engineering, Procurement and Construction
EQ Evaluation Question
ERR Economic Rate of Return
ESMP Environmental and Social Management Plan
EVI Electric Vine Industries
FEED Front End Engineering Design
FGD Focus Group Discussion
GEF Global Environment Facility
GHG Greenhouse gas
GIS Geographic Information System
GOI Government of Indonesia
GP Green Prosperity
GSK Green Sumba Consortium
GW Gigawatt
ha Hectare
IBEKA Inisiatif Bisnis dan Ekonomi Kerakyatan
ICC Intra-Cluster Correlation
IR Indonesian Rupiah
IE Impact Evaluation
IEA International Energy Agency
IIEE Indonesian Institute for Energy Economics
IRB Institutional Review Board
ISEAS Yusof Ishak Institute
KEN Kebijakan Energi Nasional/ National Energy Policy
KII Key Informant Interview
KKI Warsi Komunitas Konservasi Indonesia WARSI
KSM Kelompok Swakelola Masyarakat
SOCIALIMPACT.COM vi
kWp Kilowatt peak value
LATIN Yayasan Lembaga Alam Tropika Indonesia
LED Light Emitting Diode
LLPSLH Lingkungan Hidup
M&E Monitoring and Evaluation
MCA-I Millennium Challenge Account- Indonesia
MDES Minimum detectable effect size
MHP Micro Hydro Power
MKS PT Mikro Kisi Sumba
MW Megawatt
NRE New and Renewable Energy
NRM Natural Resource Management
O&M Operations and Maintenance
OECD Organization for Economic Cooperation and Development
PE Performance Evaluation
PEKA Yayasan Peduli Konservasi Alam
PLTMH Micro Hydro Power Plants
PLN Perusahaan Listrik Negara
PLTS penerangan lampu tenaga surya rumah tangga
PLUP Participatory Land Use Planning
PMAP 1 Participatory Mapping and Planning
PODES Village Potential Statistics
PSGIP Project Social and Gender Integration Plan
PSM Propensity Score Matching
PV Photovoltaic
RE Renewable Energy
REDD Reducing Emissions from Deforestation and Forest Degradation
SOCIALIMPACT.COM vii
RESCO Renewable Energy Service Center
RT Sub-village administrative unit, also known as kampungs
RUPTL Rencana Usaha Penyediaan Tenaga Listrik
SI Social Impact
SHS Solar Home System
SPV Special Purpose Vehicle
SWP Solar Water Pump
TA Technical Assistance
TAPP Technical Assistance and Preparation Project
TV Television
UN United Nations
UNEP United Nations Environment Program
VBS Village Boundary Setting
WtP Willingness to Pay
WWF World Wildlife Fund
YLBHL Yayasan Lembaga Bantuan Hukum Lingkungan Jambi
SOCIALIMPACT.COM viii
1 INTRODUCTION & BACKGROUND

1.1 Country context
Indonesia, which has the largest economy in Southeast Asia, has experienced steady growth averaging
between 5-6 percent since the Asian financial crisis of 1997-1999.1 Nonetheless, as an archipelago nation
stretching over 5,000 kilometers across Oceania, Indonesia is vulnerable to the increased occurrence of
extreme weather events, flooding due to sea-level rise, and water-borne illnesses that are likely to accompany
the climate change that is already being observed across the country.2 For this reason, it is a stated objective
of the Government of Indonesia (GOI) to achieve a reduction of greenhouse gas (GHG) emissions “in a way
that is consistent with pro-growth, pro-poor, and pro-job development objectives.”3 As one way of achieving
these parallel objectives, Indonesia’s National Energy Policy (Kebijakan Energi Nasional, or KEN) set a target
of increasing the country’s usage of new and renewable energy (NRE) from 4 percent of all energy usage in
2011 to 23 percent by 2025 and 31 percent by 2050.4 Indeed, the Asian Development Bank (ADB) indicates
that renewable sources of electricity offer many “positive cobenefits” in addition to reduced GHG emissions
including rural revitalization, jobs and employment, economic development, and avoided environmental costs
of fuel extraction and transport.5
Although Indonesia has rapidly electrified a large proportion of its population, 16 percent of households still
lacked access to electricity as of 2014.6 Compared to the 84 percent with access to electricity, these
households are more frequently found in remote islands or rural villages where the feasibility and cost of
electrification through traditional means is prohibitive. As a result, households in these villages typically resort
to “costly and polluting”7 diesel-fired power generation for intermittent electricity throughout the day.8 For some
of these communities, off-grid, renewable resources (such as solar, biomass, or micro-hydro systems)
represent a more feasible path to electrification than traditional, fossil-fuel based power grids.
1.2 Objectives of this report

This report has four primary objectives. The first is to present the process for identification of the two focal
projects within the portfolio of community-based off grid (CBOG) renewable energy (RE) grants of the
Millennium Challenge Account-Indonesia (MCA-I) Green Prosperity (GP) Project’s Grant Facility for pre/post
evaluation. The second is to communicate the purpose and guiding research questions behind the evaluation
of those two projects. The third is to define the quantitative and qualitative methods Social Impact (SI) has
chosen to respond to these evaluation questions, along with the limitations of these methods. The final
objective is to outline SI’s administrative approach to executing the evaluation, including the evaluation team
structure and schedule.
1
https://www.adb.org/sites/default/files/publication/178039/ino-paper-09-2015.pdf pg. 6.
2
https://www.adb.org/sites/default/files/publication/215986/adbi-wp622.pdf pg. 2
3
https://www.illegal-logging.info/sites/files/chlogging/uploads/IndonesiasiaranpdfGreenPaperFinal.pdf pg. 20
4
http://prokum.esdm.go.id/pp/2014/PPpercent20Nomorpercent2079percent202014.pdf pg. 8
5
https://www.adb.org/sites/default/files/publication/217001/ewp-502.pdf pg. 7-8
6
W3A Anekatek Solar, East Sumba DFS pg. 1-3.
7
In addition to the increased household-level cost of this energy source relative to renewable sources, the Asian Development Bank estimates that about $0.50 of
every $1.00 expended on conventional electricity leaves the local economy, whereas every dollar invested in renewable electricity can produce $1.40 in gross
economic gain due to the local and labor-intensive nature of the capital required.
8
W3A-80 DFS pg. 1-2.
SOCIALIMPACT.COM 9
Since these two projects are not fully representative of the entire CBOG RE grant portfolio, this report will also
aim to suggest ways in which the scope of the evaluation could be broadened in future data collection periods
to more representatively comment on the portfolios achievements and lessons learned for future programming
as a secondary objective.
SOCIALIMPACT.COM 10
2 OVERVIEW OF COMPACT & INTERVENTIONS

2.1 Overview of the Project and Implementation Plan
To combat environmental degradation and alleviate rural poverty, The Millennium Challenge Corporation
(MCC) entered a five-year, $600 million Compact with the Government of Indonesia (GOI) in April 2013,
establishing MCA-I, which aims to reduce poverty through economic growth. The GP Project, the flagship
project of the Indonesia MCC Compact with a budget of $332 million, is designed to support the GOI’s
commitment to a more sustainable, less carbon-intensive future by promoting environmentally sustainable,
low carbon economic growth. The main objective of the project is to work with local communities to create
economic opportunities that alleviate poverty and improve management of Indonesia’s natural capital. The
project will provide a combination of technical assistance and grants to help communities improve land
management practices and design and implement economic activities that enhance livelihoods and protect
critical ecosystem services that people rely on for income and wellbeing. It is anticipated that activities under
the GP project will complement the GOI’s efforts to reduce emissions from deforestation and environmental
degradation. More broadly, the project is also expected to help foster greater, greener, and smarter outside
investment in Indonesia by improving the basis by which land use decisions are made and by creating
incentives for increased deployment of cleaner technologies.
The Green Prosperity project as a whole is comprised of four discrete activities, detailed below:
1. The Participatory Land Use Planning (PLUP) activity is meant 1) to ensure that projects funded by
the GP Finance Facility are designed based on accurate and appropriate spatial and land use data,
and adhere to and reinforce existing national laws, regulations and plans; and 2) to strengthen the
capacity of local communities and district-level institutions to manage their own land and resources.
This is accomplished through participatory village boundary setting (VBS), updating and integrating
land and other natural resource use plans, and enhancing district and provincial spatial plans. The first
PLUP contract, called Participatory Mapping and Planning 1 (PMAP 1), was awarded to Abt
Associates to implement PLUP Tasks 1 through 4 in the four starter districts. Seven additional PMAPs
with varying levels of implementation of the four PMAP 1 tasks were also originally planned, although
one of these—PMAP 5—has since been cancelled. As of August 2017 PMAPs 1-4 are complete and
PMAPs 6-8 are underway. Overall, PMAP contracts will include implementation in up to 45 districts
throughout Indonesia.
2. The GP Facility provides grant financing to mobilize greater private sector investment and community
participation in RE and sustainable land use practices. The GP Facility investments are intended to
enhance sustainable economic growth and social conditions while also reducing Indonesia’s carbon
footprint. The GP Facility targets investments in commercial and community-based renewable energy
projects less than 10 megawatts (MW) in size, sustainable natural resource management, and
community-based projects to promote improved forest and land use practices. These investments will
support a number of objectives that promote productive use of energy and protect renewable
resources from which energy can be derived. Grants will be funded through three schemes, or “funding
windows”: Partnership Grants (Window 1), Community-based Natural Resource Management Grants
(Window 2), and RE Grants (Window 3).
SOCIALIMPACT.COM 11
3. The Technical Assistance and Oversight activity is designed to provide assistance and oversight
for eligible districts, project sponsors and community groups to identify and develop potential
investments in sustainable low-carbon economic growth. This activity will also institute a
comprehensive set of procedures to track and evaluate the progress of the projects it funds and the
effectiveness of the GP Project activities implemented to facilitate the success of those projects.
Technical Assistance will include performing or reviewing detailed feasibility studies, engineering
designs, as well as requirements on environmental, social and economic benefit, monitoring and
evaluation to meet GOI permitting and international performance standards.
4. The Green Knowledge activity supports and enhances the results of GP projects by facilitating the
collection, application and dissemination of knowledge relevant to low carbon development within and
beyond GP districts. The activity will provide capacity building for local and provincial stakeholders,
develop and improve centers of excellence in science and technology related to low carbon, and broad
networks for information exchange, knowledge generation, and sharing.
2.1.1 Original Project Description

At the outset of the GP Facility, grants were meant to be issued through three separate funding windows, each
with different mechanisms for selecting grantees. At the time of this report, MCC prefers to consider
aggregations of GP Facility grants by thematic area (e.g. CBOG RE, palm oil, cocoa, etc.) rather than by the
funding window through which they were granted. We will introduce the CBOG RE portfolio in the context of
how each grant was funded before proceeding to characterize each grant by the method through which it aims
to promote the usage of CBOG RE in Indonesia. By introducing the grants in this way, we also hope to facilitate
referencing them throughout the report.
Window 1 of the GP Facility aimed to co-fund grants leveraging private resources to accomplish an array of
larger GP objectives including “improving land governance, resource management, and renewable
development to improve people’s access to clean energy.”9 Ultimately two grants co-financed through this
window included renewable energy components, although only one (implemented by a Hivosled consortium)
maintained this component.
Window 2 of the Facility sought to issue grants for small-scale, community-based natural resource
management (CBNRM) projects that “promote community-based initiatives in forestry, agriculture and off-grid
renewable energy, enhanced management of watersheds and forests to improve the sustainability of
renewable energy and/or agriculture investments, and support rural livelihoods and economic development.”10
Although it is uncommon that these grants focus entirely on CBOG RE components, many (18 of 49) include
some kind of CBOG RE component in their programming.
Finally, Window 3 of the GP Facility funded grants focusing almost entirely on the promotion of RE. These
grants were divided into two funding schemes: Community-based RE grants (Window 3A, or W3A) and
9
http://www.mca-indonesia.go.id/en/project/green-prosperity/grant/green-prosperity-partnership-grant
10
http://www.mca-indonesia.go.id/en/project/green-prosperity/grant/community-based-nrm-cbnrm-grants
SOCIALIMPACT.COM 12
Commercial-scale RE Grants (Window 3B).11 The former funding scheme provides grants for “project
preparation, construction, initial Operations & Maintenance (O&M), and training for suitable small RE projects
that will benefit local communities. These grants will help communities receive reliable and adequate supplies
of electricity and benefit from revenue streams derived from energy production.”12 The projects financed by
these grants are defined by new or expanded electricity generation from a community-based facility utilizing
off-grid micro-hydro, solar, biomass, and/or wind energy systems.
By July of 2015, 21Technical Assistance & Project Preparation (TAPP) Grant Agreements had been issued
to organizations working with various communities across Indonesia to implement the Window 3A projects
described above. Seven of these were granted TAPP extensions. The stated purpose of these grants was to
strengthen “Implementer project preparation on par with MCA-Indonesia standards in order to support high
quality, evidenced-based project preparation.” Under each grant, implementers were to produce four key
deliverables13:
1. A Detailed Feasibility Study (DFS) and Front-End Engineering Design (FEED);
2. Specific studies to bridge design gaps identified in any existing feasibility study;
3. Capacity-building, staff training, and supervision services necessary for successful project preparation
for implementation; and
4. Incremental work related to complying with MCC Environmental Guidelines and MCC Gender Policy
and landscape-lifescape analysis.
On the basis of the deliverables produced under these seven extended TAPP grant agreements, MCA-I
funded the implementation of six additional implementation grant agreements.
Table 1, below, includes a high-level summary of all 26 grants that comprise the CBOG RE Portfolio as of July
31, 2017. Although grant numbers and project titles are included in this table, these numbers and titles are
not referred to consistently across project documentation. Henceforth in this report, to avoid confusion, we will
refer to grants using the following convention: “[W(indow)#] [Grantee] [Technology], [Location].” For example,
the first grant in the solar category below would be referred to as “W2 Yayasan Javlec Solar, Berau” and the
first grant in the biomass category below would be referred to as “W3A Charta Putra Biomass, Siberut Island.”
It was not possible to review documentation for all the grants in the CBOG RE portfolio prior to writing this
report; nor will it be possible to include all of them in the scope of a pre/post evaluation meant to characterize
the portfolio’s achievements and lessons learned. In the sections that follow, we give a more detailed overview
of the targeted participants and implementation to date of ten grants whose project documentation SI was able
to access as of July 31, 2017. This overview is meant to serve as a resource for our justification of which
grants to select for pre/post evaluation as well as for suggestions of additional grants that could be included
in a recommended ex post portion of the portfolio evaluation to broaden the representativeness of evaluation
findings to a larger proportion of the overall portfolio.
11
As none of the Window 3B grants include CBOG RE components, they are outside the scope of this evaluation and will not be included in this report.
12
http://www.mca-indonesia.go.id/en/project/green-prosperity/green-prosperity-facility
13
W3A-80 TAPP Agreement pg. 7
SOCIALIMPACT.COM 13
Table 1: CBOG RE Grants Signed (as of August 16, 2017)14

Grant Planned Technical Total Project
Project
Window Year / Grantee Project Title Project Location Capacity Assistance Value with Description of RE works
Grant Value
Number (KW) Grant Value Co-financing
SOLAR
Solar PV & small-scale ice

2016 Yayasan Javlec Developing Eco- Berau, East cube processing unit for
2 100 N/A $1,187,822 N/A
Grant 044 Indonesia friendly Businesses Kalimantan fisherman; Mangrove
Information Center
Utilization of Natural
Solar PV (Sumber Agung)
Resources and
Yayasan Peduli and seaweed/ fish cake
2016 Sustainable Berau, East
2 Konservasi Alam 320 N/A $870,469 N/A processing unit; Solar PV
Grant 039 Renewable Energy for Kalimantan
(PEKA) (Giring Giring) and cocofiber
Community Welfare
processing unit
Improvement
Solar PV (APDS); Solar PV

(APMB); Solar PV (APMP);
Natural Resources Kapuas Hulu, Solar PV (APNL); Solar PV
2016
2 Yayasan Dian Tama Management of Peat West 6 N/A $1,848,953 N/A (APBS); Honey Production
Grant 037
Swamp Forest Kalimantan Houses (Central and each
location); Ecotourism
(Selimbau)
New renewable energy

2016 PT Cahaya Inti Malinau, North Solar PV (Metut); Solar PV
2 development utilizing 101 N/A $1,764,363 N/A
Grant 047 Trimanunggal Kalimantan (Long Berang)
solar power
14
Note: multiple grants are undergoing amendments in 2017 that may change the scope of the renewable energy work
SOCIALIMPACT.COM 14

Project
Grant Value
Lembaga Kajian dan

Pengembangan
Sumber Daya Improvement of poor Solok Selatan &
Solar Home System (SHS)
2016 Manusia – Pengurus household income Tanjug Jabung
2 86 N/A $1,241,250 N/A (Rawasari); SHS (Sungai
Grant 035 Besar Nahdlatul through green Timur, West
Rambut); SHS (Bukik Bulek)
Ulama business practices Sumatra
(LAKPESDAM –
PBNU)
2016 Subur Makmur DAS Sumba Timur &

2 Bumi Manira [pump] N/A $827,943 N/A Solar water pump
Grant 071 Kadahang Sumba Tengah
Enhancing Community
Irrigation; Small Retention
2016 Burung/ Konsorsium Livelihood and
2 Sumba [pump] N/A $1,813,475 N/A Basin; Rainwater reservoir;
Grant 024 Sumba Hijau Conserving
Deep Wells
Environment
Building a productive
2016 and Sustainable Social
2 Kemitraan Sumba Timur 7.8 N/A $1,370,264 N/A Solar PV
Grant 032 Forestry
Entrepreneurship
Nursery house (capacity

Cacao commodity
2016 Sumba Barat 10.000 seeds); Barsha pump
2 YKP Donders development and food 4 N/A $1,203,938 N/A
Grant 029 Daya 10 unit; Solar Water Pump
crop plantation
(SWP)
2017 Anekatek
3A Sumba 492 $498,350 $9,200,000 $10,091,279 Solar PV
W3A-59 Consortium
2017 Wakatobi,
3A Puriver Consortium 800 $648,302 $7,857,472 $8,833,169 Solar PV
W3A-68 South Sulawesi
SOCIALIMPACT.COM 15

Project
Grant Value
2017 Sky Energy Mamuju, West

3A 598 $561,523 $5,786,266 $6,588,883 Solar PV
W3A-80 Consortium Sulawesi
BIOMASS
Siberut Island,
2017
PT Charta Putra Mentawai Bamboo &/or biomass power
3A W3A - 700 $973,288 $11,946,181 $13,417,229
Indonesia Island, West plant
56/7/8
Sumatera
Biogas/ Biodigester (Muaro

Yayasan Lembaga
Optimizing land use to Pijoan); Communal Cow
2016 Bantuan Hukum Jambi, Central [1 HH bio-
2 support food and N/A $411,498 N/A Cattle (Muaro Pijoan);
Grant 056 Lingkungan Jambi Sumatra digester]
energy souvereignity Rehabilitation Irigation (S.
(YLBHL)
Duren)
Solok Selatan,
Yayasan Lembaga Supporting community Biogas/Biodigester (Lubuk
2016 Sub District [7 HH bio-
2 Alam Tropika based forest N/A $1,378,080 N/A Gadang); Ecotourism (Solok
Grant 054 Sangir, West digesters]
Indonesia (LATIN) management Selatan)
Sumatra
Hydro
Dusun Tuo 150 KW micro-

2015 Riau, and Jambi hydropower plant (RE
1 WWF Indonesia 150 N/A $5,500,000 $10,000,000
Grant/014 Sumatra Barat component of grant:
$1,125,872)
Development of Micro hydro (Rantau Malam);

2016 Lembaga Penelitian Community-based Sintang, West Microhydro (Jelundung);
2 154 N/A $1,063,038 N/A
Grant 060 dan Pengembangan Sustainable Agriculture Kalimantan Irigation & Small Bridge
Sumberdaya dan Model (Jelundung); Farmer Hut
(Jelundung); Rubber
SOCIALIMPACT.COM 16

Project
Grant Value
Lingkungan Hidup Production Unit (Rantau

(LPPSLH) Malam); Tempoyak
(Fermented Durian)
Production House (Rantau
Malam)
Utilization of Small
Hydropower
Microhydro (Tepuse);
Renewable Energy for
Microhydro (Suwan); Cocoa
2016 Households Mahakam Ulu,
2 Yayasan Pena Bulu 64 N/A $1,454,393 N/A Production House (Long
Grant 048 Electrification and East Kalimantan
Apari); Cocoa Production
Improvement of
House (Long Pahangai)
Community Cacao
Business
Microhydro (Pulakek Koto

Komunitas Birah); Biodigester/ Biogas;
Konservasi Improvement of Animal Watching Shelter
Solok Selatan,
2016 Indonesia WARSI community’s Welfare (Solok Selatan); Composting
2 Pesisir Selatan, 120 N/A $866,097 N/A
Grant 061 (KKI Warsi) - through inclusive Production Unit (Solok
West Sumatra
initiative Sumatera livelihood Selatan & Pesisir Selatan);
Barat Rice Milling (Solok Selatan &
Pesisir Selatan)
SOCIALIMPACT.COM 17

Project
Grant Value
Kerinci,
Komunitas Strengthening green
Merangin,
Konservasi development practices
2016 Muaro Jambi, Microhydro (Beringin Tinggi);
2 Indonesia WARSI to improve the 200 N/A $1,016,817 N/A
Grant 062 Tanjung Jabung Microhydro (Rantau Kermas)
(KKI Warsi) - environment’s carrying
Timur, Central
initiative Jambi capacity
Sumatra
Economic
Solok Selatan,
Indonesian Institute improvement through
2016 Sub District
2 for Energy Renewable energy- 50 N/A $1,378,980 N/A Microhydro (Wonorejo)
Grant 063 Towoti,West
Economics (IIEE) based Center of
Sumatra
Knowledge (CoK)
Pro Poor for

community based RE
Micro Hydro (Kutta); Micro
development,
2016 Hydro (Kalilang); Micro Hydro
2 IBEKA watershed Sumba Timur 160 N/A $1,923,000 N/A
Grant 066 (Kamanjara); Knowledge
management,
Center Facilities
ecotourism, and
sustainable agriculture
Bayan and
2017 Lombok Utara Hijau Santong, North
3A 1,320 $930,315 $7,375,360 $10,845,768 Mini hydro
W3A-04 Consortium Lombok, West
Nusa Tenggara
Combination
SOCIALIMPACT.COM 18

Project
Grant Value
3,200 home bio-digesters; 55

2015 school or kiosk solar charging
1 HIVOS - PG Lombok, Sumba 50 N/A $4,700,000 $9,400,000
Grant/018 stations (RE component of
grant: $727,782)
Microhydro (Lebuk Lantang);

Microhydro (Lanjau);
Sub District:
Microhydro (Sungai Buluh);
Benua Tengah
SHS(Benua Tengah); Pipe
2016 Koperasi Kredit (CU) District: Kapuas
2 150 N/A $1,489,100 N/A Water Supply (Benua); Pipe
Grant 046 Keling Kumang Hulu
Water Supply (Riam Batu);
Province: West
Homestay (Sunagi Utik);
Kalimantan
Ecotourism (Lebuk Lantang)
& Solar home system
2017 PT Akuo Energy Berau, East

3A 1,243 $921,673 $9,796,525 $10,705,875 Solar PV/mini hydro
W3A-33 Indonesia Kalimantan
SOCIALIMPACT.COM 19
2.1.2 Project Participants and Geographic Coverage

2.1.2.1 W3A Akuo Energy Solar/Micro-Hydro, Berau
The Off-Grid Power Plants for three Villages in Berau Regency-East Kalimantan Project (W3A Akuo Energy
Solar/Micro-Hydro, Berau) targets three villages in the Berau Regency of the East Kalimantan Province: Teluk
Sumbang, Long Beliu, and Merabu. All the households in Teluk Sumbang and Merabu (comprising 167 and
73 households, respectively) will be connected to the new and/or upgraded power systems. In Long Beliu,
223 out of 251 total households will be connected to the new power system. In all cases, the grantee plans to
attempt to connect all households where a connection would be practical and feasible based on distance from
the grid and socioeconomic conditions. In the case of the non-connected households in Long Beliu, all these
pertain to a sub-village administrative unit (or “RT”) that is wealthier than other parts of the village, located
directly along the main village road. The target households pertain to a different RT seven kilometers away
from the road.
At the village level, site selection occurred on the basis of government priority lists of villages with low or no
rates of electrification. The Mining and Energy Agency in Berau (Dinas Pertambangan dan Energi
(Distamben)) collected applications from villages to receive grant assistance and presented a list of ten
suitable villages to PT. Akuo Energy for potential inclusion in the project. PT Akuo Energy initially selected
four of these villages in which they would conduct DFS, and ultimately dropped one (Balikukup) when the DFS
found that the most suitable location for a solar PV micro-grid was prone to erosion and potentially
unsustainable. Figure 1 displays the final three villages selected for grant assistance, along with the originally
considered fourth village.
Figure 1: Map of Target Villages for W3A Akuo Energy Solar/Micro-Hydro, Berau
SOCIALIMPACT.COM 20
2.1.2.2 W3A Charta Putra Biomass, Siberut Island

The Siberut Aggregated Biomass Gasification Power Plant Project (W3A Charta Putra Biomass, Siberut
Island) targets three villages on Siberut Island in the Mentawai Islands regency of West Sumatra, including
Madobag and Matotonan in South Siberut and Saliguma in Central Siberut. The project is targeting all
households in the South Siberut villages for connection to the new biomass off-grid system, and all but thirty
to forty households in Saliguma. The non-targeted households in Saliguma are not being considered for
collection because of their distance from the rest of the households of the village, although the project still
hopes to improve their access to electrification via rechargeable batteries or some other means.
The project implementer, PT Charta Putra Indonesia (also known as Clean Power Indonesia, or CPI) selected
the Mentawai Islands regency for the project because it has the lowest rate of electrification in Western
Indonesia. Siberut Island is the largest of the Mentawai Island chain, and CPI claims that the three villages
targeted by the project were selected because, together, they “represent the whole island.” The selected
villages include culturally and ecologically critical portions of the island (Madobag and Matotonan), as well as
a new, coastal village which is the poorest in the regency (Saliguma). CPI would ideally like to replicate the
Biomass-based micro-grids across the remaining seventeen villages of Siberut Island, although this activity
would not be funded under the grant issued by MCA-I. The three villages targeted by the MCA-I grant can be
found in Figure 2.
Figure 2: Map of Target Villages for W3A Charta Putra Biomass, Siberut Island
SOCIALIMPACT.COM 21
2.1.2.3 W3A Anekatek Solar, East Sumba

The Solar PV Distributed System in East Sumba Project (W3A Anekatek Solar, East Sumba) targets 909
households in the East Sumba Regency for electrification via connection to eleven, sub-village (or kampung)
level solar PV micro-grid systems. These eleven systems are distributed across five villages: Tawui, Lailunggi,
Praimadita, Tandula Jangga, and Praiwitu. The 909 households targeted include all the households in the
eleven kampungs targeted across the five villages.
The East Sumba regency was targeted by this project based on previous studies executed under an ADB
Technical Assistance grant (TA 8287) held by Castlerock Consulting, a service provider on cross-cutting
deliverables on the W3A grant.15 Within this regency, the implementer selected villages (desa) based on
criteria that included mobile network access and proximity to a Perusahaan Listrik Negara (PLN)16 station.
Finally, targeted kampungs were selected within these villages based largely on population density, as
measured by a GPS roof-tagging exercise. Aside from population density, it is the implementer’s belief that
there are no categorical differences between selected and non-selected kampungs. Figure 3, below, displays
the kampung targeted by the project in the larger context of East Sumba.
Figure 3: Map of Target Sub-Villages for W3A Anekatek Solar, East Sumba
15 The purpose of this TA was to “support the GoI’s Sumba Iconic Island Initiative,” which aims to electrify 95% of households on the island of Sumba via 100%
renewable means by 2025. The referenced Network Planner exercise was part of a “comprehensive least-cost electrification planning exercise” for Sumba, wherein
the most cost-effective and technically appropriate means for achieving a 100% electrification ratio were laid out (ADB 2014).
16
Indonesian state-owned company tasked with supplying the electricity needs of the Indonesian people.
SOCIALIMPACT.COM 22
2.1.2.4 W3A Puriver Solar, Tomia Island

The Solar Photovoltaic Electricity for Tomia Island: A Green Prosperity Model Project (W3A Puriver Solar,
Tomia Island) targets all 987 households in the Kahianga, Wawotimu, Kulati, Dete, and Lamanggau villages
of Tomia Island, one of the Wakatobi Islands in Southeast Sulawesi. These five villages were selected
because they are excluded from the PLN’s Electrical Power Provision Business Plan for 2015 – 2024 and it
would not be economically or environmentally feasible to integrate them into existing power grids on the
mainland of Sulawesi or other surrounding islands.17
Figure 4: Map of Target Villages for W3A Puriver Solar, Tomia Island
2.1.2.5 W3A Sky Energy Solar, Karampuang Island

The Solar Photovoltaic Electricity for Karampuang Island Project (W3A Sky Energy Solar, Karampuang Island)
targets all 784 households in Karampuang Village, which covers all of Karampuang Island in the Mamuju
regency of West Sulawesi. Although the criteria by which this island was selected for this grant are uncertain,
the project’s DFS indicates that demand for electricity on the island far outstrips the baseline supply provided
by ten community diesel generators and supplemental household generators. This project site is unique
compared to its surroundings, as is depicted in Figure 5, since it is a lone island off the coast of West Sulawesi.
Figure 5: Map of Targeted Village for W3A Sky Energy Solar, Karampuang Island
17
Detailed Feasibility Study, pg. 16
SOCIALIMPACT.COM 23
2.1.2.6 W2 Green Sumba Solar, Central Sumba

This Window 2 grant is implemented by the Green Sumba Consortium (GSK)18 in 79 villages in 13 sub-districts
in the Central Sumba Landscape (Bentang Alam Sumba Bagian Tengah (BA-SBT)). The area covers 260,000
hectares (ha) in three districts including Central Sumba, West Sumba, and East Sumba (70 percent, 18
percent, and 12 percent, respectively, from the area of BA-SBT), and an estimated 90,000 citizens. The project
aims to strengthen natural resource management (NRM) to increase prosperity, leading to climate resilience
as well as contributing to climate change mitigation and to the preservation of the natural ecosystem in BA-
SBT.
18
The grant agreement is between MCA-I and Perhimpunan Pelestarian Burung Liar Indonesia (Burung Indonesia), the lead institution in the Consortium. The
Consortium, in addition to Burung Indonesia, includes Lembaga Peduli Sejahtera dan Lestari Sumba (Pelita Sumba), Yayasan Bahtera, Yayasan Wahana
Komunikasi Wanita, Forum Perempuan Sumba (FOREMBA), and Forum Jaringan Manupeu Tanadaru (JAMATADA).
SOCIALIMPACT.COM 24
2.1.2.7 W2 IBEKA Micro-Hydro, East Sumba

This Window 2 grant is implemented by Inisiatif Bisnis dan Ekonomi Kerakyatan (IBEKA) Foundation (as the
lead institution) together with consortium partners.19 The project targets two sub-districts in East Sumba
(Kahaungu Eti and Pandawai), and 756 households within seven villages. These households are expected to
benefit from the development of four Micro Hydro Power (MHP) facilities that will have a total capacity 186
kW.
2.1.2.8 W2 Yayasan Dian Tama Pontianak Solar, Kapuas Hulu
This Window 2 grant is implemented by Yayasan Dian Tama Pontianak (Lead Consortium institution) together
with consortium partners.20 They implement the project in the district of Kapuas Hulu in West Kalimantan,21 in
the Kapuas watershed and Leboyan-labian sub-watershed. This 86,000-ha area includes peat swamp forest
and dry lowland forest, which are particularly vulnerable to forest fires and other land use changes. The project
plans to reach 18 villages in seven sub-districts in Kapuas Hulu (namely Selimbau, Jongkong, Batang Lupar,
Suhaid, Badau, Bunut Hilir, and Embaloh Hilir), and 1,014 households.
2.1.2.9 W2 CUKK Micro-Hydro/Solar, West Kalimantan
This Window 2 grant is implemented by Koperasi Kredit Keling Kuman (as the Consortium leader (CUKK),
together with consortium partners.22 CUKK has 62 branch offices, four of which are located in the project
area – namely Kapuas Hulu and Sintang District in West Kalimantan. The project will work in six villages
in the former district (two sub-districts), and seven villages in the latter district (one sub-district).
Beneficiaries include 789 households (or 3,190 individuals) in Kapuas Hulu and 444 households (or
1,776 individuals) in Sintang.
2.1.2.10 W1 Hivos Solar/Biogas, Sumba/Sulawesi
This grant operates in nine districts spread across three provinces: East Nusa Tenggara, West Nusa
Tenggara, and South Sulawesi. The specific districts targeted include East Sumba, West Sumba, Central
Sumba, Southwest Sumba, North Lombok, East Lombok, Central Lombok, North Luwu, and East Luwu.
Although most of the grant’s physical outputs (such as solar lanterns or solar PV units) target schools, kiosks,
and agro-processing mills on the island of Sumba, the grant will also aim to install household biogas digesters
across all three provinces. Besides physical RE outputs, the grant targets government, private sector, and
civil society stakeholders with community engagement programming. In total, the grant estimates that it will
have 61,500 direct beneficiaries. These direct beneficiaries are mostly comprised of rural households with
school-aged children, with emphasis placed on households where program outputs might promote livelihood
security, reduce economic constraints, or promote economic opportunities. 23
19
The Consortium, in addition to IBEKA, includes Koperasi Serba Usaha (KSU) Kamanggih, Koperasi Jasa Peduli Kasih Kamanggih, PT.RENERCONSYS, PT.
Caruban Inti Technology, and CV Insan Bangun Utama.
20
Consortium partners include Yayasan Dunia Lingkungan Hidup Indonesia / World Wildlife Fund for Nature Indonesia, Perkumpulan Kahan, Koperasi Asosiasi
Periau Danau Sentarum (APDS), Lembaga Pengkajian dan Studi Arus lnformasi Regional (LPS- AIR), Yayasan Riak Bumi, and Komunitas Pariwisata Kapuas
Hulu (KOMPAKH).
21
APL area.
22
The consortium includes Koperasi Produsen K77 and Aliansi Masyarakat Adat Nusantara Kalimantan Barat (AMAN Kalbar).
23
Grant agreement, Attachment B
SOCIALIMPACT.COM 25
2.1.3 Implementation to Date and Planned Outputs

Each of the Window 3A grants has six general categories of outputs in common (as enumerated in their grant
agreements): (i) an Engineering, Procurement and Construction (EPC) contract with an MCA-I approved
contractor; (ii) physical infrastructure including off-grid Power Plants, electricity distribution lines, house
installation lines, protection devices, and meters; (iii) Mandatory operational permits and licenses for electricity
generation and distribution; (iv) fully implemented and monitored environmental and social performance
(ESMP) and project social and gender integration plans (PSGIP); (v) community members with adequate
technical, managerial, and entrepreneurial capacity to sustainably operate a SPV responsible for overseeing
the off-grid system and supporting productive activities for the electricity’s use; and (vi) project management,
monitoring, and reporting. As of the submission of this report, all five grants have initiated construction on key
physical outputs as well as community capacity building.
Although the grants have similar outputs in the abstract, they differ in the nature and amount of physical
infrastructure in each grant, the timeline for the infrastructure’s activation, and the capacity building
requirements for the establishment of an SPV in each community. These specific differences in planned
outputs are presented in the following sub-sections, with the exception of W3A Puriver Solar, Tomia Island for
which complete data was not available in time for preparation of this report.
2.1.3.1 W3A Akuo Energy Solar/Micro-Hydro, Berau
W3A Akuo Energy Indonesia (AEI) Solar/Micro-Hydro, Berau started construction of all facilities and
necessary complementary outputs for all three villages in July of 2017. The facility in Merabu is scheduled for
commissioning in December of 2017, while the remaining facilities will be commissioned in March of 2018. A
summary of the main physical outputs from this project and their corresponding power capacities can be found
in Table 2.
Table 2: W3A Akuo Energy Solar/Micro-Hydro, Berau Summary of Physical Outputs
Number of Capacity Household
Location Technology
facilities (kWp) connections
Teluk Solar PV, Micro‐ 414 (solar),
2 138
Sumbang hydro 30 (hydro)
Long Beliu Solar PV 1 518 165
Merabu Solar PV 1 311 97

TOTAL 4 1,273 kWp 400
While the facilities are under construction, AEI will work with the local communities to form SPVs that will be
responsible for the facilities’ long-term operation. According to the grant’s SPV Business Plan (revised May 8,
2017), these village-level SPVs will be dually owned by the implementer and a village-owned enterprise
(Badan Usaha Milik Desa, or BUMDes). During construction, AEI will have a majority share in the SPV,
whereas after construction shares will be split 75% to 25% in favor of the BUMDes. Each SPV will be shaped
according to the organigram in Figure 6.
SOCIALIMPACT.COM 26
Figure 6: W3A Akuo Energy Solar/Micro-Hydro, Berau SPV Organigram

Operationally, the technician is responsible for day-to-day O&M of the plant. The finance and administration
staff is responsible for book-keeping and documentation as well as managing the SPV’s voucher-based sales
system and financial reporting. The safeguard compliance staff is responsible for coordinating community
development and compliance with environmental, social, and gender safeguard procedures. Routine
preventative maintenance and intermediate troubleshooting will be contracted out to a local O&M company,
while system control and advanced and inverter troubleshooting will be handled by AEI.
Although AEI will have a 25% share in the SPV, all SPV dividends will belong to the BUMDes. These dividends
will be utilized according to the procedure outlined in Figure 7.Specifically, the 10% of gross profits reserved
for community benefits each year will target electricity usage effectiveness awareness and economic activities
by women’s groups.
SOCIALIMPACT.COM 27
Figure 7: W3A Akuo Energy Solar/Micro-Hydro, Berau SPV Profit Utilization

2.1.3.2 W3A Charta Putra Biomass, Siberut Island
The Siberut Aggregated Biomass Gasification Power Plant Project is due to commission all seven of its
biomass gasifier facilities in March of 2018. These facilities will be split among three villages with the capacity
indicated in Table 3. As of July 2017, these facilities are under construction.
SOCIALIMPACT.COM 28
Table 3: W3A Charta Putra Biomass, Siberut Island Summary of Physical Outputs
Location Technology Number of facilities Capacity (kW) Household connections
Madobag Biomass 3 300 537
Matotonan Biomass 2 150 270
Saliguma Biomass 2 200 397

TOTAL 7 650 kW 1,204
As this grant is the only one to implement biomass-based micro-grids, it has a unique economic model and
community engagement mechanism relative to the other Window 3A grants. The project will construct an
SPV24 co-owned and operated by local villagers (as represented by three Village-Level Enterprises, or VLEs),
regency government representatives (as represented by a Badan Usaha Milik Daerah, or BUMD), and the
project implementer (CPI).
The VLEs will harvest and supply bamboo as feedstock for the grids, at first from indigenous sources before
ultimately harvesting from a new bamboo plantation. These VLEs are the majority owners of the SPV and
primary beneficiaries of the project. The BUMD is responsible for guaranteeing the financial viability of the
power plants, monitoring electricity demand from local industries and businesses, and encouraging productive
uses of the electricity through government programming or subsidy. CPI is responsible for the project
implementation, including appropriate vocational training of local villagers as both bamboo farmers and power-
plant managers and operators.
Representatives from each of these three groups will be involved in two separate teams: an SPV Project
Management team that will dissolve after the project has been fully implemented, and an O&M Team that will
persist through the lifetime of the power plants. Each team will manage a contractor related to its role in
implementation. In the case of the O&M team, the O&M contractor will be appointed for five years with an
option for an additional five-year extension. See Figure 8, below.
24
The SPV approach described here is based on the DFS, which is the most updated SPV plan available to SI as of July 2017. SI acknowledges based on MCA-I
comments that this approach has been updated since this time.
SOCIALIMPACT.COM 29
Figure 8: W3A-56-58 SPV Organization and Management25

2.1.3.3 W3A Anekatek Solar, East Sumba
Construction of Solar PV facilities and complementary structures for W3A Anekatek Solar, East Sumba is due
to commence in August of 2017. At this time, the facilities in Tawui Riyang, Tawui Northeast, Tawui North,
and Tawui West are due to be commissioned by November 10, 2017. The remaining facilities will be completed
between November 28,
2017 and January 31, 2018. Table 4 summarizes the capacity and expected household connections of each
of these facilities.
Table 4: W3A Anekatek Solar, East Sumba Summary of Physical Outputs

Tawui Riyang Solar PV 1 9 18
Tawui West Solar PV 1 12 28
Tawui Northeast Solar PV 1 7.5 17
Tawui North Solar PV 1 12 27
25
As pictured on pg. 26 of W3A 56-58 DFS.
SOCIALIMPACT.COM 30

Tawui South Solar PV 1 99 209
Lailunggi Solar PV 1 103.5 216
Rehi Jara Solar PV 1 16.5 32
Tanah Rong Solar PV 1 24 44
Tandula Jangga Solar PV 1 75 136
Praiwitu North Solar PV 1 103.5 136
Praiwitu South Solar PV 1 30 46
TOTAL 11 492 kW 909
Compared to the W3A grant in Berau, which will set up an SPV in each village in which it operates, W3A
Anekatek Solar, East Sumba has established one SPV, “PT Mikro Kisi Sumba (MKS),” to cover all eleven
treatment areas spread across five villages.26 The implementer’s sister company, Electric Vine Industries
(EVI), will have 100% ownership of the SPV during the construction phase, after which ownership will be split
51% to 49% in favor of the communities. The communities will be represented by a secondary cooperative
comprised of members of five primary cooperatives representing each village in which the project will operate.
27
Operationally, the SPV will issue a contract to “PT LVI” for O&M of the facilities and management of
administration and finance. Where other grant’s SPVs typically aim to complete finance and administration in-
house, PT MKS is paying for these to be completed externally since the contractor has key experience and
software to implement a mobile phone-based, pre-paid “smart metering” system that aims to increase project
sustainability by matching payment cycles with end-user’s income cycles. Users of the micro-grids will lose
access to power once they have used their pre-paid credit. Custodians employed by the SPV will be
responsible solely for O&M tasks related to cleaning arrays and clearing vegetation and debris from the roots
and distribution. Besides the custodians, the only other operational SPV staff will be community, social, and
environmental officers responsible for overseeing the implementation of ESMP and PSGIP along with liaising
between cooperative members and technical and managerial SPV staff—including registering customer
complaints. See Figure 9 for an overview of the SPV’s structure following the end of the construction phase.
26
The information presented in this section is based off of the grant’s SPV Business Plan, dated July 5, 2017 which is the most updated plan available to SI as of
July 2017. SI acknowledges based on MCA-I comments that this approach may have been updated since this time.
27
In all villages but Praiwitu, these cooperatives will be established from scratch. Since Praiwitu is the only village with an
existing cooperative, this cooperative will be assessed for suitability as an SPV before a cooperative is established from scratch.
SOCIALIMPACT.COM 31
Figure 9: W3A Anekatek Solar, East Sumba SPV Organigram28

In order to increase sustainability of the micro-grids, members of the primary cooperatives (households in
each village) will receive capacity building over the course of construction on various themes, including:
community development, social inclusion, and gender awareness; SPV management and sustainability;
operation and maintenance, and renewable energy. These trainings are meant to increase community
members’ awareness of and engagement with the benefits of the RE technology as well as their ability to
successfully manage it after the project has ended.
Once the SPV is generating revenue and funds have been set aside for a maintenance reserve, dividends
remaining after O&M and contractor costs will be allocated for activities. These may include capital for new
businesses in the villages, incentives for members that do not have sufficient income to pay electricity tariffs,
or capital for cooperative members.
2.1.3.4 W3A Sky Energy Solar, Karampuang Island
According to the schedule laid out in the grant agreement, procurement and construction of all Solar PV
facilities on Karampuang Island was underway by February of 2017. The four plants will be commissioned
between September and December of 2017. Their combined capacity, as laid out in Table 5, will be sufficient
to connect all 784 households on the Island.
28
W3A Anekatek Solar, East Sumba SPV Business Plan (dated July 5, 2017); Exhibit 2
SOCIALIMPACT.COM 32
Table 5: W3A Sky Energy Solar, Karampuang Island Summary of Physical Outputs
Location Technology Number of Capacity Household
facilities (kW) connections
Karampuang Solar PV 4 599 784
Island
TOTAL 4 599 kW 784
The structure of the SPV29 for W3A Sky Energy Solar, Karampuang Island is centered on a Village-Owned
Enterprise (Badan Usaha Milik Desa, or BUMDes) headed by the village chief as CEO (Chief Executive
Officer) and supported by a Secretary, O&M Coordinator, and Treasurer. In addition to these technical roles
within the BUMDes, there will be two BUMDes representatives for each of the four sub-villages responsible
for maintaining relationships between the villagers and the BUMDes. Besides these central roles within the
BUMDes, the SPV will also include “Shareholders” responsible for stepping in to address major problems in
the SPV and an O&M Contractor responsible for major O&M problems that cannot be resolved by BUMDes
O&M staff. Chapter 11 of the W3A-80 DFS clearly maps out the roles and responsibilities of each of these
parties across several business processes, including procurement, routine O&M, major O&M, and voucher
sales.
According to the schedule found in the grant agreement, public consultation, technical training, and managerial
training of the SPV is due to take place between March and September of 2017. MCA-I will complete handover
to the SPV in January of 2018.
2.1.3.5 W2 Green Sumba Solar, Central Sumba
This project is expected to result in renewable energy (RE), forest management, and sustainable agriculture
benefits. The project has three high-level outcomes as follows:
 Outcome A: Strengthened livelihoods of people in BA-SBT through natural resources management
and capacity building of village level organizations
 Outcome B: Strengthened practice of land management to increase forest cover and strengthened
practice of utilizing renewable energy
 Outcome C: The mainstreaming of the development of productive and sustainable BA-SBT
management
Specifically related to RE, Outcome B includes an output titled ‘Increased households which utilize renewable
energy’. The consortium expects activities focused on promoting household solar power lighting (penerangan
lampu tenaga surya rumah tangga (PLTS)) to help provide electricity to 13 villages covering around 283
households in the project implementation area.
The project importantly includes a community-based approach to the promotion of RE sources toward the goal
of sustainability post-implementation. In order to improve the livelihoods of local communities, the project not
only focuses on improving agriculture and animal husbandry, but also technical capacity, social investment,
and social organization. At the village level, the project develops community groups that discuss access to
29
The SPV approach described here is based on the DFS, which is the most updated SPV plan available to SI as of August 2017. SI acknowledges based on MCA-
I comments that this approach has been updated since this time.
SOCIALIMPACT.COM 33
natural resources, park boundaries, and monitoring of resource use. These groups promote village-level
agreements and regulations to better manage their lands. The project encourages participation in the
government-established musrenbang and village development planning process, so that they play a key role
in achieving a productive and sustainable landscape.
The grant began in July 206 and will conclude activities within 18 months of its start date in December 2017.).
It is currently completing work in quarter 5 of the grant agreement and is on track with most planned activities.
2.1.3.6 W2 IBEKA Micro-Hydro, East Sumba
The W2 IBEKA Micro-Hydro project aims to achieve the following three outcomes:
 Outcome 1: Strengthened communities and cooperatives as Local Economic Development Centers
 Outcome 2: Operationalization of four micro hydro power plants (PLTMH) to supply seven villages
 Outcome 3: Increased agricultural productivity
Specifically related to Outcome 2, the project not only constructs four MHP facilities but also trains operators
for each location. This outcome’s success is closely linked to Outcome 1, which includes outputs related to
the development of community cooperatives with the purpose of operating and maintaining the MHP facilities.
In Outcome 1, the project involves the community through orientations and regular meetings, in addition to
actual participation in the construction of the facilities. The project plans to involve the community in facility
location selection, construction, and maintenance. The project also regularly meets with farmer and women’s
groups to collect feedback and data on needs and impacts of MHP development. In this way, the community
takes on a leadership and maintenance role related to the RE source introduced through this Window 2 grant.
The grant will conclude activities within 17 months of its start date in December 2017. (start date of August
2016). In quarter 1 and 2, the project was facing significant challenges with receiving required licenses for
MHP development (leading to construction delays). This reportedly affected community buy-in and
participation, and has led to the need for re-socialization activities and adjusted community approaches.
2.1.3.7 W2 Yayasan Dian Tama Pontianak Solar, Kapuas Hulu
The goal of this project is to increase productivity and value added of community products through the use of
renewable energy, management of peat forests to increase people’s incomes, management of peat swamp
forests, and reduction of dependence on fossil fuel in and around conservation areas in Kapuas Hulu. The
project has two expected outcomes, as follows:
 Outcome 1: Increased productivity, product added value, product standardization and marketing
networks in three ecotourism management groups and five solar energy sub-centers of honey (39
groups of fish and processed products farmers, 5 groups of fishermen women) without the use of
fossil fuel.
 Increased management of peat land ecosystems, aquaculture ecosystems, ecotourism destinations
and habitat of bees through sustainable use of land.
These outcomes are further specified by seven specific outputs, including forest fire mitigation/management,
ecotourism development, a market study, RE (solar energy) sub-center development, and information sharing
within the community about renewable energy.
SOCIALIMPACT.COM 34
The RE component of this grant involves the development of five solar energy processing houses for honey
and fish. The production houses (and the processes) will reduce their use of fossil fuels and reduce public
spending on fossil fuels for production purposes by using 250 W solar panels. The project plans to conduct
capacity building activities regarding production house (and solar panel) maintenance and operation by
December 2017.
The grant will conclude activities within 18 months of its start date, in November 2017 (start date of June
2016). The houses were expected to be completed in December 2016 (with solar panels completed by June
2017). Issues noted in the quarter 3 project report included availability of funding, which has slowed
implementation.
2.1.3.8 W2 CUKK Micro-Hydro/Solar, West Kalimantan
The goal of this project is to reduce poverty and improve people's quality of life through fair and sustainable
environmental management efforts for sustainable economic growth. The project will conduct empowerment,
cultivation, productivity and RE activities/trainings. The project has four high-level outcomes, as follows:
 Outcome 1: Decreasing the dependency on fossil fuel by providing renewable energy.
 Outcome 2: Improving Saran and Embaloh Hulu territories governance participatorily (sic) and
sustainably.
 Outcome 3: Changing community behaviors on maintaining natural resources and increasing
productivity.
 Outcome 4: Optimizing catchment area functioning.
Particularly related to Outcome 1 and the RE component of this grant, various targeted villages at the time of
project launch relied on diesel-fueled power plants. This resulted in high diesel prices and air pollution. The
power was only provided for 3 hours at a time, and, resultingly, residents had to resort to kerosene fuel to light
their homes. To address this, the project is procuring RE sources through development of Micro Hydro Power
Plants (PLTMH)30 and a Solar Power Plant (PLTS)31. The project will also develop a governance system to
maintain these facilities, and encourage community participation in the
development/construction/maintenance process.
The plants will range in capacity from 21 – 74 KW, reaching 151 KW to 273 households.
The grant will conclude activities within 19 months of its start date, in December 2017 (start date of June
2016). At inception, the project implementer already identified challenges will accessing parts to maintain
PLTMH and PLTS in West Kalimantan. In their third quarterly report, the implementer reported completing
participatory mapping workshops in seven villages. The project had also already received letters of
recommendation regarding the development of the solar plant. The project planned to socialize and conduct
focus group discussions regarding PLTS and PLTMH in March 2017.
30
To be developed in Lebuk Lantang (servicing 500 households, 2 churches, 1 homestay and 1 town hall), Lanjau (servicing 90 households, 1 village office, 1
village hall, 1 primary school and 1 church) and Sungai Buluh (in some grant documents, this is listed as Rawa Bangun – 60 households, 1 village office, 1 village
hall, 1 primary school, and street lighting).
31
To be developed in Benua Tengah (servicing 60 households, 10 street lights, 1 church and 1 health clinic).
SOCIALIMPACT.COM 35
2.1.3.9 W1 Hivos Solar/Biogas, Sumba/Sulawesi

Hivos’ grant aims to improve rural livelihoods through utilization of renewable energy across two dimensions:
increased access to and application of RE technology and improved human capacity and social cohesion with
respect to RE technology. It aims to accomplish the first dimension by installing 50 solar-powered agro-
processing mills, 25 school-based solar PV systems, and 20 solar remote charging stations in Sumba, while
installing 3,200 household biogas digesters across the East Nusa Tenggara, West Nusa Tenggara, and South
Sulawesi provinces. The grant will also rent 6,000 solar lanterns in the same areas where school-level solar
PV units are installed. In total, these technologies will have a combined capacity of 9,152 kW spread across
an estimated 61,500 direct beneficiaries.32
To complement these physical installations and accompanying rental technologies (including solar lanterns
accompanying the school-based solar PV systems and charging kiosks), the Hivos consortium will conduct
capacity building on business-, technical-, and gender-related themes to prepare communities for the
utilization of the new technology.
In order to sustain these outputs, the consortium will simultaneously create a market for the off-grid
technologies using a renewable energy service center (RESCO) approach coupled with stakeholder
engagement and community outreach to maintain a commitment to participatory and gender-sensitive
development of RE systems in government, civil society, and the private sector. The Waingapu- and
Waitabula-based RESCOs constitute a different approach to community engagement and ownership than the
SPV approaches utilized by the Window 3A grants. Specifically, they will collect monthly fees from customers
(mostly local cooperatives or user groups and kiosk owners) to fund maintenance fees, repayment of
equipment funded by the grant, and an operating margin. The repaid equipment portion of these fees will fund
replacement of RE system components when they fail, and the operating margin will cover operating expenses
such as salaries and rent. In turn, the RESCOs will ensure delivery of the RE service and provide monthly
maintenance and system repairs.
As of March2017, procurement of the non-biodigester physical installations was ongoing. In turn, 1,453 of the
targeted 3,200 biogas digesters had already been constructed. Training of RESCO staff was also ongoing,
specifically on themes related to system installation, reporting, site safety, and community engagement.33 All
program activities are scheduled to be completed by March of 2018.
2.2 Theory of Change34

All five of the Window 3A grants operate on a nearly identical theory of change, which can be summarized as:
if communities with low access to electrification in remote areas of Indonesia are provided with renewable-
energy based micro-grids and capacity building in the proper operation and management of these micro-grids,
then (i) the communities will have an increased awareness of RE and sustainable natural resource
management; (ii) households in these communities will have reliable and sustainable access to electricity; and
(iii) community cooperatives will have the capacity to operate and manage the micro-grids. Supposing these
32
Grant agreement, Attachment B
33
Q5 Quarterly Report.
34
As section 3.2 will explain, the evaluation at this time will focus on two Window 3A grants. As such, we provide an overview of the theory of change for Window
3A grants only in this section.
SOCIALIMPACT.COM 36
outcomes are realized and the communities derive sustainable benefits as SPVs continue to provide adequate
O&M services, household income will be increased and GHG emissions decreased due to the improved
access to and utilization of electricity generated from RE sources. In addition to the three outcomes mentioned
above, all but W3A-56-58 additionally posit that increased economic opportunities will result from productive
uses of the increased supply of electricity. By way of example, the log frame for W3A Anekatek Solar, East
Sumba depicts the logical progression of this theory of change from the status quo through to final impacts in
Figure 10.35
Although the DFS or M&E (Monitoring and Evaluation) Plans for the Window 3A grants typically include some
characterization of the theory of change above, they rarely include the underlying assumptions or detailed
intermediate steps required for the ultimate goals to be realized. We provide a bit more detail from the literature
here to highlight key measurement areas for the evaluation.
In theory, electrification is expected to positively affect households and service provision. First, it improves
incomes via a decrease in energy expenditures, an extension of working hours, the use of productive motive
power, and eventually better income opportunities and new and more efficient businesses. Second, it yields
better education via extended study hours, improved access to knowledge and information, and improved
school services. Third, it leads to improved health from a decrease in polluting lighting sources (kerosene)
and improved health services by electrified health facilities. Lastly, it yields positive effects via electrification
on security, community participation and (gender) attitudes via improved connectivity and media access (see
Lenz et al., 2017).
These theorized impacts are contingent upon a handful of key assumptions:
1. Households are open to using the new technology. While this is generally not a problematic
assumption, it could be violated if there is mistrust between the community and the implementer or a
lack of optimism in the community that the new technology will be sustainable.
2. Beneficiary communities will have adequate access to regional and national markets to allow village
enterprises to count on more than local demand. Without this, there may be little incentive to expand
or create new businesses. This assumption is likely to be tested more often in agricultural communities
that cannot count on the same export base as enterprises in communities that rely on fishing or eco-
tourism.
3. For education outcomes to materialize, schools must be up and running and students must have
access to study materials in order to allow households to use electricity in a beneficial way with regards
to education.
4. Finally, this theory of change assumes that all program components are fit for purpose. The physical
infrastructure and training of community members must be suitable for achieving the purposes set out
below. If it is not, the construction of solar arrays may not result in a sustainable source of usable
electricity that meets the energy demands of uses that contribute to the above stated goals. For
example, if energy supply in practice is only sufficient to power small household appliances or lights,
then new economic opportunities may not be available. Similarly, without sufficient training and
35
Our presentation of program logic in this section is representative of the benefit streams outlined in project M&E plans. There are frequently additional outcomes
associated with increased electricity access, including improved gender equality through changes in time use due to time-saving appliances and improved security
due to lighting. Our evaluation will aim to capture such outcomes of similar programming, even if they are not included in project M&E documents.
SOCIALIMPACT.COM 37
resources, communities might not have the capacity to conduct adequate O&M procedures to ensure
the sustainability of physical outputs.
SI will work to monitor the veracity of these assumptions, where appropriate, using our existing instruments.
As an example, we may monitor the assumption about access to markets by asking enterprises where their
customers generally come from alongside questions about their revenue and future prospects. Additionally,
we may ask community members about their interactions with grantees and their past direct or indirect
experiences with similar programming to monitor their openness to the new technology.
Figure 10: Log Frame for W3A Anekatek Solar, East Sumba
SOCIALIMPACT.COM 38
2.3 Cost Benefit Analysis & Beneficiary Analysis36

The largest and most consistent economic benefit considered by MCC cost benefit analyses (CBAs) for the
Window 3A grants is derived from the increased access to electricity from the newly established power
systems. This benefit mirrors outcome 2 in the grants’ logical frameworks. The economic benefit of this
outcome is quantified as the increased consumer surplus of the increased access to electricity (as measured
through a Willingness to Pay (WtP) methodology). Another benefit stream that appears consistently in all the
economic rate of return (ERR) calculations is a resource cost savings benefit, measured by the decrease in
consumer expenditure on electricity from the new RE sources compared to status quo sources like kerosene
or diesel generators. This substitution is not explicitly linked to any of the four grant outcomes, although it is
an implicit mechanism for the increased household income and decreased GHG emissions cited as the overall
objective and impact of the grants.
Inconsistently, individual CBAs consider additional benefits to the increased consumer surplus and resource
cost savings. In the case of W3A Akuo Energy Solar/Micro-Hydro, Berau, these additional streams include
increased income for honey and boat production and additional resource cost savings on ice for storage of
caught fish. These benefits are linked to outcome 4 of the grant logic, which involves productive uses of the
increased electricity supply. Although these benefits are between ten and fifty times the magnitude of the
standard resource cost saving benefit from substitution of the source of electricity, they still pale in comparison
to the increased consumer surplus benefit. After adding these to the WtP benefit, the overall 20-year ERR
only increases from 24.5 percent to 25.0 percent.
In the case of W3A-80, additional benefit streams include cost savings of public facilities and increased income
from quality improvements in various local micro-industries resulting from training funded by the grant. It is not
clear that the first of these fits into the grant’s logical framework, which is mostly concerned with household-
level outcomes and impacts, although it stands to reason that public facilities would make the same
substitution a household would make in the face of the increased, renewable supply. The second is linked
with outcome 4, as was the case for W3A Akuo Energy Solar/Micro-Hydro, Berau. Similar to W3A Akuo Energy
Solar/Micro-Hydro, Berau, these additional benefit streams are marginal compared to the more significant WtP
benefit.
Table 6: ERR for each of the Window 3A Grants
Grant 20-year ERR (standard benefits) 20-year ERR (total)
W3A Akuo Energy Solar, Berau 24.50 percent 25.03 percent
W3A Charta Putra Biomass, Siberut Island 26.57 percent 26.57 percent
W3A Anekatek Solar, East Sumba 19.45 percent 19.45 percent
W3A Puriver Solar, Tomia Island 8.77 percent Not available
W3A Sky Energy Solar, Karampuang Island 34.10 percent 34.90 percent
36
This section only describes the ERRs to which SI had access as of July 31, 2017
SOCIALIMPACT.COM 39
2.4 Literature Review

2.4.1 Summary of Existing Evidence
Micro-grids play a crucial role in efforts to provide universal access to electricity by 2030 around the world, as
proclaimed by the United Nations (UN) initiative Sustainable Energy for All (SE4All) and the Sustainable
Development Goal 7. The International Energy Agency (IEA) estimates that 42 percent of the additional
electricity generation capacity to reach universal access can most economically be achieved through micro-
grids (IEA 201037).
The academic literature is inconclusive about the impacts of rural electrification on rural development, and
there are only few rigorous studies to provide compelling evidence. For example, in India, Bangladesh, and
Vietnam respectively, Van de Walle et al. (2015)38, Khandker, Barnes, and Samad (2012)39, and Khandker,
Barnes, and Samad (2013)40 find evidence for positive effects on job market indicators, household income,
and educational performance as a result of electrification. Parikh et al. (2015)41 find positive effects in particular
for women from infrastructure provision, including electricity, in Indian slums on literacy, income and health.
Grimm, Sparrow and Tasciotti (2015)42 and Peters and Vance (2011)43 show that electrification contributes
substantially to the fertility decline in Indonesia and Côte d’Ivoire respectively. In addition, some positive
evidence on firm productivity comes from India, Kenya, Nicaragua, and South Africa (Rud, 201244; Gibson
and Olivia, 201045; Kirubi et al., 200946; Grogan and Sadanand, 201347).
There is, however, a set of more sobering findings. While research indicates that lighting is a high priority for
people and is in fact used also for purposes considered to be beneficial from a development perspective,
impacts on productive activities, however, are often much less pronounced than expected (Bernard, 20148;
Peters, Vance and Harsdorff, 201149; Neelsen and Peters, 201150; Grimm, Hartwig and Lay, 201351; Banerjee
37
Birol, F. (2010). World energy outlook 2010. International Energy Agency, 1(3).
38
van de Walle, D., Ravallion, M., Mendiratta, V., & and Koolwal, G. (2015). Long-term impacts of household electrification in rural India. World Bank Economic
Review, forthcoming.
39
Khandker, S. R., Barnes, D.F. & Samad, H.A. (2012). The Welfare Impacts of Rural Electrification in Bangladesh. The Energy Journal, 33(1), 187.
40
Khandker, S. R., Barnes, D.F. & Samad, H.A. (2012). The Welfare Impacts of Rural Electrification in Bangladesh. The Energy Journal, 33(1), 187.
41
Parikh, P., Fu, K., Parikh, H., McRobie, A., & George, G. (2015). Infrastructure Provision, Gender, and Poverty in Indian Slums. World Development, 66, 468-
486.
42
Grimm, M., Sparrow, R., & Tasciotti, L. (2015). Does electrification spur the fertility transition? Evidence from Indonesia. Demography, forthcoming.
43
Peters, J., & Vance, C. (2011). Rural Electrification and Fertility – Evidence from Côte d’Ivoire. Journal of Development Studies, 47 (5), 753-766.
44
Rud, J.P. (2012). Electricity provision and industrial development: Evidence from India. Journal of Development Economics, 97(2), 352–67.
45
Gibson, J., & Olivia, S. (2010). The effect of infrastructure access and quality on non-farm enterprises in rural Indonesia. World Development, 38(5), 717-726
46
Kirubi, C., Jacobson, A., Kammen, D. M., & Mills, A. (2009). Community-based electric micro-grids can contribute to rural development: evidence from Kenya.
World Development, 37(7), 1208-1221.
47
Grogan, L. & Sadanand, A. (2013). Rural Electrification and Employment in Poor Countries: Evidence from Nicaragua. World Development, 43(0), 252–265.
48
Bernard, T. (2012). Impact Analysis of Rural Electrification Projects in Sub-Saharan Africa. World Bank Research Observer, 27(1), 33–51.
49
Peters, Jörg, Colin Vance, and Marek Harsdorff. 2011. “Grid Extension in Rural Benin: Micro-Manufacturers and the Electrification Trap.” World Development,
39(5): 773–83.
50
Neelsen, Sven and Jörg Peters. 2011. “Electricity usage in micro-enterprises — Evidence from Lake Victoria, Uganda.” Energy for Sustainable Development,
15(1): 21–31.
51
Grimm, M., Hartwig, R. & Lay, J. (2013). Electricity Access and the Performance of Micro and Small Enterprises: Evidence from West Africa. European Journal of
Development Research, 25, 815-829.
SOCIALIMPACT.COM 40
et al., 201152; Lenz et al., 201753; Peters et al., 201354; Peters and Sievert 201555; Oakley et al., 200756; Obeng
and Every, 201057). A recent large-scale evaluation of a rural electrification program in Tanzania58, for
example, finds reductions in some traditional energy source uses and positive effects on land prices and
lighting usage as proxies for well-being. However, there are no impacts on non-agricultural employment or
firm creation. The reason is often that in most rural areas electricity is not the only bottleneck that impedes
business development. In the absence of roads and market access, electricity can only be used for productive
purposes that serve the local demand, which is often small. Moreover, households and enterprises in rural
areas typically have a very low ability to pay. As a result, typical household electricity demand is very low (see
for example D’Agostino et al. 201659; Grimm and Peters 201660; Bensch et al. 201661). Electricity in rural areas
is often only used for lighting, charging mobile phones and operating radios and sometimes TV (television)-
sets (see for example IEG 200862, Lenz et al., 201763).
The impacts of electrification on GHG emissions and the environment depends on the source of electricity
that is supplied and the initial energy sources that are being replaced. Currently, RE sources make up between
15 percent and 20 percent of the world’s total energy demand. In the case of solar PV and micro-hydro plant
installation, the energy provided is from non-depletable fuels solely and consumption does not emit GHG
(Akella et al., 2009)64. The more these new systems replace initial reliance on oil, coal, and natural gas, the
better the environmental impacts of the intervention. One example is dry-cell batteries and light emitting diode
(LED) lamps, which have replaced kerosene in many parts of the developing world (see Bensch, Peters and
Sievert 201765). Electrification can hence help to reduce e-waste in rural areas. Furthermore, high emission
reductions can in particular be expected when rural households replace diesel-driven machinery use or
biomass-based cooking and heating by electric appliances. Biomass use for cooking and heating is a major
cause of climate-relevant emissions (for example Shindell et al., 201266; Ramanathan & Carmichael 200867;
52
Banerjee, S. G., A. Singh, and Samad, H. (2011). Power and people : the benefits of renewable energy in Nepal. Washington D.C., World Bank.
53
Lenz, L., A. Munyehirwe, J. Peters und M. Sievert. 2017. Does Large Scale Infrastructure Investment Alleviate Poverty? Impacts of Rwanda's Electricity Access
Roll-Out Program. World Development 89 (17): 88-110.
54
Peters, J., M. Sievert and C. Vance (2013), Firm Performance and Electricity Usage in Small Manufacturing and Service Firms in Ghana. In: Deutsche
Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH (ed.), Productive Use of Energy – PRODUSE ‐ Measuring Impacts of Electrification on Small
and Micro‐Enterprises in Sub‐Saharan Africa. Eschborn: Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH. 75‐94
55
Peters, J., & Sievert, M. (2015). The provision of electricity to rural communities through Micro-Hydro Power in rural Indonesia: Micro Hydro Power pilot programme
within the national programme for community development (PNPM) supported by the Netherlands through energising development (No. 88). RWI Materialien
56
Oakley, D., P. Harris, et al. (2007). Modern energy ‐ Impact on micro‐enterprise. A report produced by the Department for International Development. R8145.
DFID. AEA Energy and Environment. March 2007.
57
Obeng, G. Y. and H. D. Evers (2010). Impacts of public solar PV electrification on rural microenterprises: The case of Ghana. Energy for Sustainable Development
14(3): 223‐231.
58
Chaplin, D., Mamun, A., Protik, A., Schurrer, J., Vohra, D., Bos, K., ... & Cook, T. Grid Electricity Expansion in Tanzania by MCC: Findings from a Rigorous Impact
Evaluation, Final Report (No. 144768f69008442e96369195ed29da85). Mathematica Policy Research.
59
D'Agostino, A.L., Lund, P.D. and Urpelainen, J., 2016. The business of distributed solar power: a comparative case study of centralized charging stations and
solar microgrids. Wiley Interdisciplinary Reviews: Energy and Environment.
60
Grimm, M., & Peters, J. (2016). Solar off-grid markets in Africa. Recent dynamics and the role of branded products. Field Actions Science Reports. The journal
of field actions, (Special Issue 15), 160-163.
61
Bensch, G., Grimm, M., Huppertz, M., Langbein, J., & Peters, J. (2016). Are promotion programs needed to establish off-grid solar energy markets? Evidence
from rural Burkina Faso (No. 653). Ruhr Economic Papers.
62
Independent Evaluation Group (IEG). 2008. The Welfare Impacts of Rural Electrification – An IEG Impact Evaluation. Independent Evaluation Group, World Bank.
63
Lenz, L., A. Munyehirwe, J. Peters und M. Sievert. 2017. Does Large Scale Infrastructure Investment Alleviate Poverty? Impacts of Rwanda's Electricity Access Roll-
Out Program. World Development 89 (17): 88-110.
64
Akella, A.K. 2009. Social, economical and environmental impacts of renewable energy systems. Renewable Energy 34: 390–396
65
Bensch, G., J. Peters und M. Sievert (2017), The lighting transition in rural Africa — From kerosene to battery-powered LED and the emerging disposal problem.
Energy for Sustainable Development 39 : 13-20.
66
Shindell, D., Kuylenstierna, J. C., Vignati, E., van Dingenen, R., Amann, M., Klimont, Z., ... & Schwartz, J. (2012). Simultaneously mitigating near-term climate
change and improving human health and food security. Science, 335(6065), 183-189.
67
Ramanathan, V., & Carmichael, G. (2008). Global and regional climate changes due to black carbon. Nature geoscience, 1(4), 221.
SOCIALIMPACT.COM 41
Bailis et al. 20168). While typically electricity is rarely used for cooking in developing countries, in Asia the use
of electric rice cookers is very common.
There are very few rigorous studies on the sustainability of micro-grid programs, partly because only few
examples of sustainably working micro-grid programs exist that have matured beyond the installation of just
a model micro-grid. There are a few potential reasons for low sustainability. First, institutional and political
challenges often impede cost-covering electricity consumption tariffs that would make investments into micro-
grids attractive. In most countries, rural electricity tariffs - even for the national grid - are not cost recovering
(see Trimble et al. 201669), but highly subsidized by governments or in the best-case cross-subsidized by
urban consumers. Accordingly, typically regulatory bodies or the incumbent utility will not readily approve
higher tariffs that are needed to make micro-grids cost covering (Peters and Sievert, 201570). In addition,
payment enforcement may be hampered by low ability to pay (D’Agostino et al. 201671) and irregular, seasonal
income flows that are typical among agriculture-reliant populations. Furthermore, there may be a low
willingness-to-pay, as the costs of renewable energies (solar, hydro, wind) are not directly visible for the
population given its local generation (as compared to, for example, the case of generators).
Mini-grids can be operated by public-private partnerships or by communities. For micro-grids operated by the
community, the two key challenges are tariff setting and payment enforcement (Peters and Sievert 201572).
Incentives and obstacles to enforce payment rigorously are different for a community member than for
outsiders working for a commercial operator. Most importantly, social entanglements may complicate rigorous
enforcement. In theory, the same mechanism can also work the other way around, where social cohesion
might lead people to feel more obliged to pay their contributions. Lastly, payment for operational staff may
seem dispensable in rural subsistence communities where paid labor is rather an exception than the rule.
This, again, may lead to too low tariffs and bad payment discipline.
2.4.2 Gaps in Literature

This evaluation can provide evidence on three gaps in the literature. In particular, two design features of the
Window 3A projects are highly interesting from a global learning point of view.
First, as outlined above, despite high costs attached to electrification, there is generally no consensus on the
impacts of electrification on rural development, and less so for the case of micro-grids. Given that micro-grids
play an important role in the SE4ALL goal of universal electricity access, evidence is highly required.
Second, a comparison of different micro-grid management or financing systems does not exist in the literature.
The only examination has been done in Indonesia for non-private micro-grids run by the community and fully
subsidized by the government (see Peters and Sievert 201573). Evidence on the impacts of the management
68
Bailis R., Drigo R., Ghilardi A. and O. Masera (2015). The carbon footprint of traditional woodfuels. National Climate Change 5:266–72
69
Trimble, Christopher Philip; Kojima, Masami; Perez Arroyo, Ines; Mohammadzadeh, Farah. 2016. “Financial viability of electricity sectors in Sub-Saharan Africa :
quasi-fiscal deficits and hidden costs”. World Bank Policy Research Working Paper.
70
Peters, J., & Sievert, M. (2015). The provision of electricity to rural communities through Micro-Hydro Power in rural Indonesia: Micro Hydro Power pilot
programme within the national programme for community development (PNPM) supported by the Netherlands through energising development (No. 88). RWI
Materialien
71
D'Agostino, A.L., Lund, P.D. and Urpelainen, J., 2016. The business of distributed solar power: a comparative case study of centralized charging stations and
solar microgrids. Wiley Interdisciplinary Reviews: Energy and Environment.
72
73
SOCIALIMPACT.COM 42
system on the sustainability of micro-hydro plants is not available and, more concretely, there is no
understanding of the dynamics that may hamper or foster payment enforcement among local customers and
O&M practices among the local community operators.
Third, there is no study that assesses the impact of providing electricity access paired with productive use
promotion. The exception is one study on microfinance and electricity (Khandker and Koolwal (201074). Given
high impact expectations from electrification and productive use aspirations, but often limited income effects
in practice, learning on combined interventions is highly relevant. The trainings on productive use, as provided
by the Window 3A projects, in conjunction with electricity provision therefore serve as a unique opportunity to
fill this gap.
2.4.3 Policy Relevance of the Evaluation

The electrification rate in Indonesia has been increasing at a steady pace, expanding from approximately 43
percent in 1995 to 84 percent in 201575. There are, however, great disparities in electricity access across
regions, ranging between 36.4 percent in Papua and 100 percent in Jakarta. Generally, electrification is
disproportionately provided in the centers of Java and Bali, while the eastern provinces are characterized by
the lowest electrification rates. In 2012, the provinces East Kalimantan (W3A Akuo Energy Solar/Micro-Hydro,
Berau) and East Nusa Tenggara (W3A Anekatek Solar, East Sumba) had electrification rates of 64 and 44
percent respectively, lagging behind the average electrification rate of 75 percent of that year.
The country has an installed electricity generating capacity of 51.92 GW (gigawatts), of which the vast majority
is generated from fossil fuels (83.2 percent), with coal being the predominant type of fossil fuel. 11 percent of
the capacity is generated by hydroelectric plants. The remaining 5.8 percent comes from other renewable
sources.76 The country produces high levels of GHG emissions. The use of fossil fuels, in particular in the
power sector and transportation, is expected to more than double the country’s energy-related CO2 emissions
in the coming 25 years, rising to more than 800 million tons by 2035.77
The Gol political agenda pursues as major objectives the increase in electricity access, an expansion of RE
use and green growth. The country was one of the first to ratify the United Nations Framework Convention on
Climate Change and to adopt the Kyoto Protocol78. The National Energy policy (KEN) aims at increasing the
country’s usage of new and RE from 4 percent of all energy usage in 2011 to 23 percent by 2025 and 31
percent by 205079. Simultaneously, the 2015-2019 National Medium Development Plan sets the goal of
reaching an electrification rate of 96.6 percent by the end of 2019 with a particular focus on disadvantaged
communities and remote, undeveloped regions.80 In an attempt of bringing together these multiple goals, the
GOI and the state electricity company Perusahaan Listrik Negara (PLN) have launched several rural
74
Khandker, S.R., Koolwal, G.B. (2010) How Infrastructure and Financial Institutions Affect Rural Income and Poverty: Evidence from Bangladesh. Journal of
Development Studies, Vol. 46 (6), p.1109–1137
75
https://www.cia.gov/library/publications/the-world-factbook/geos/id.html
76
https://www.adb.org/sites/default/files/publication/178039/ino-paper-09-2015.pdf pg.8
77
78
http://prokum.esdm.go.id/pp/2014/PP%20Nomor%2079%202014.pdf pg. 8
79
80
Current policies in the RE sector include the Ministerial Decree No.38/2016, which aims at expediting electricity access in remote Indonesia. However, the
Ministerial Decree No. 12/2017 by the Ministry of Energy and Mineral Resources regulates tariffs of electricity generated from RE, and Decree No. 4 and 5/2017 by
the Ministry of Industry set quality requirements for the content of solar PV modules. Both may hamper investments into RE (see
https://d2oc0ihd6a5bt.cloudfront.net/wp-content/uploads/sites/837/2017/06/ACEF-2017-Session-18-Info-sheet-02-06-2017.pdf)
SOCIALIMPACT.COM 43
electrification plans. Among them stands out the longer-term solar development plan Thousand Islands
Program, which aims at expanding the solar installed capacity to 620 MW (megawatts) by 202081.
However, the government faces several challenges in reaching the remaining 16 percent of its population that
lacks electricity access. This population group is the most costly and timely and technically more difficult to
serve, given the lower population density and ability to pay. Moreover, the mountainous topography of the
archipelagic nation represents a challenge for the expansion of electricity access. Electricity supply in the
provinces East Kalimantan (W3A Akuo Energy Solar/Micro-Hydro, Berau) and East Nusa Tenggara (W3A
Anekatek Solar, East Sumba) is particularly costly82.
Concerning off-grid electrification programs, the ADB83 summarizes the experience made by PLN and several
governmental agencies to be “mixed at best”. Private sector efforts are small in number and are described as
ad hoc. In addition, they seem to be hindered by project-specific regulatory requirements. Off-grid efforts by
line ministries and regional governments (Pemerintah Daerah) often only fund initial installation of plants, but
do not ensure financial and technical sustainability, resulting in high failure rates. PLN would be better placed
to assure sustainability, but has little experience with renewable technologies, is in a bad financial situation
and has a high workload in conventional grid extension.
As a result, many initial attempts of the Thousand Islands Programs have been delayed due to financing or
technical difficulties. The following problems have been encountered in the implementation of off-grid
electrification projects:
- Failure to assess full present and future electricity needs of the target population
- Poor design, materials and workmanship, compromising technical performance and sustainability
- Lack of financing mechanisms to trigger payment discipline among customers to finance O&M
- Lacking human resources to operate and maintain the plants
- Pricing that is inconsistent with ability to pay of the target population
- Limited scale-up opportunities due centralized focus on PLN and too little mobilization of local
governments, NGOs, the private sector, and community.
The Window 3A project approaches coincide largely with current and future (governmental) efforts of providing
electricity to the remaining unconnected 16 percent of the Indonesian population, which are characterized by
residence remoteness, low ability to pay, and limited productive activities. Thereby, the projects and the
evidence that Window 3A project create on sustainability and worthwhileness are relevant and timely. In
addition, the project design incorporates several features to tackle past challenges in sustainable off-grid
electricity provision outlined above. First, the community-based operation approaches (EQ 4: Special Purpose
Vehicles and the primary-secondary cooperative scheme) may serve as examples of how to trigger payment
discipline, thereby financing O&M and assuring sustainability of the plants. Second, the implementation of
income generating trainings (EQ 2) might represent a positive example of complementary activities to unlock
growth potentials of electrification interventions. Based on these experiences, learnings from this evaluation
may inform the design of a (still lacking) coordinated, sound policy instrument to foster sustainable off-grid
provision in rural areas. Third, this evaluation will provide evidence on electricity consumption patterns in the
81
https://www.adb.org/sites/default/files/publication/182314/achieving-electricity-access-ino.pdf pg.35
82
83
SOCIALIMPACT.COM 44
typical unconnected areas (EQ 1), which can improve assessment of present and future electricity needs of
the unconnected 16 percent of the population. Lastly, an assessment of off-grid electrification impacts on
households, GHG emissions (EQ 3) and the local economy can confirm or adjust theoretical impact
expectations, and provide evidence on potential bottlenecks to unlock them in practice.
3 EVALUATION DESIGN
3.1 Evaluation Questions
Taken as a whole, this evaluation aims, to the extent possible, to validate the program logic underlying the
portfolio of CBOG RE grants in the GP Grant Facility, doing so through a focused investigation of two specific
grants: W3A Akuo Energy Solar/Micro-Hydro, Berau and W3A Anekatek Solar, East Sumba. It will
simultaneously aim to measure impacts and compare and contrast how the grants operate, both in terms of
how similar programs operate in different contexts within Indonesia and in terms of how programs with different
approaches to electrification and community engagement operate.
At baseline, the evaluation will seek to characterize baseline conditions of outcomes of interest and important
contextual factors for program success through quantitative and qualitative means that will ultimately allow for
a rigorous validation of program logic and comparative study of approaches. Our baseline will contribute to
this effort by validating the logic that is underlying two typical Window 3A grants’ approaches to increasing
household income and reducing GHG emissions via the increased utilization of electricity generated from
renewable sources. The evaluation will be guided by four primary questions:
1.) How have energy consumption patterns changed among beneficiary households and businesses in
response to the provision of a renewable source of electricity?
a. What are the implications of these changes for household expenditures?
2.) Has the electricity provided through the RE infrastructure been used for economic purposes at the
community or household level?
a. Has the productive uses/profit-generating component of the grant been effective; and has it
helped the SPV be sustained?
3.) To what extent do any changes in energy consumption patterns favor reduced GHG emissions?
a. Are there any other ways in which the grants contribute to the objective of reducing or avoiding
GHG emissions?
4.) Has the Special Purpose Vehicle been an effective intervention to improve community buy-in and
sustainability of the infrastructure?
This evaluation will include multiple grants from the CBOG RE portfolio and, we will analyze and present
results both within and across the grants included in the evaluation to identify patterns or differences. However,
since the evaluation design and contextual factors will also vary across grants, we will note where attribution
SOCIALIMPACT.COM 45
of evaluation findings may be more heavily confounded (in a simple, treatment-only pre/post comparison vs.
a quasi-experimental, counterfactual-based comparison, for example).
It is possible that additional lines of inquiry to these evaluation questions may be pursued in future data
collection periods using ex post evaluation approaches of additional CBOG RE grants in the GP Facility.
Because the salient lines of inquiry for other grants may not become apparent until the grants are under way,
we will only suggest possibilities, as appropriate, in our Evaluation Design Overview. They will not be formally
included in the Evaluation Purpose or Questions at this time.
3.2 Evaluation Design Overview

As outlined in section 2.1.1, it is not possible due to financial, logistical, and technical constraints to include all
26 CBOG RE grants in the scope of a pre/post evaluation meant to characterize the portfolio’s achievements
and lessons learned. In order to be clear about which grants the approach outlined in this EDR applies to, this
evaluation design overview will be split into two sections: Portfolio Evaluation Design (3.2.1) and Optional
Portfolio Evaluation Add-Ons (3.2.2). All remaining sections of the EDR (including annexes) apply to the former
section, while the latter section is simply meant to serve as a point of departure for a discussion on how the
scope of the evaluation could be broadened—both in terms of lines of inquiry and in terms of grants
evaluated—if MCC desires to expand the representativeness of evaluation findings to the portfolio, as a whole.
Following these sections, we lay out the quantitative approach for an impact evaluation of W3A Anekatek
Solar, East Sumba as well as for the mini survey for W3A Akuo Energy Solar/Micro-Hydro, Berau. We then
discuss the approach to the qualitative components of the study.
3.2.1 Portfolio Evaluation Design

This portfolio evaluation will begin with two major components seeking to validate key quantitative and
qualitative tenets of the Window 3A program logic. The first component is an impact evaluation of grant W3A
Anekatek Solar, East Sumba, which will include a pre/post, large-scale quantitative exercise designed to
respond to evaluation questions 1-3 and a qualitative exercise to provide depth on evaluation questions 1-3
and respond fully to evaluation question 4. The quantitative exercise will be a rigorous, quasi-experimental
evaluation that collects primary data on outcomes of interest and important contextual factors in treatment as
well as comparison areas of East Sumba. The impact evaluation will use a matching methodology and
difference-in-differences analysis to construct a valid counterfactual. The qualitative exercise will cover both
treatment and control areas, focusing mainly on actors involved in or affected by the SPV approach to
community engagement and ownership. It will utilize key informant interviews and focus group discussions,
analyzed through rigorous coding and triangulated by the quantitative data.
The second component is a pre/post performance evaluation of W3A Akuo Energy Solar/Micro-Hydro, Berau
including a duplicate of the household survey utilized in East Sumba to provide non-experimental, quantitative
information in response to evaluation questions 1-3 and qualitative data collection to provide depth on these
and respond fully to evaluation question 4. The scope of this performance evaluation will focus on treatment
areas alone. As with the qualitative component of the W3A Anekatek Solar, East Sumba component, the PE
(performance evaluation) will utilize key informant interviews and focus group discussions, analyzed through
rigorous coding and triangulated by the quantitative data.
SOCIALIMPACT.COM 46
We propose to conduct two follow up data collection periods for each of these components—one occurring
one year after baseline data collection and another occurring three years after baseline data collection. The
justification for the first follow up is that it will allow for measurement of outcomes expected to manifest in the
short- and medium-term (such as increased energy consumption and decreased expenditure) without risking
contamination of the control group by electrification efforts conducted by other actors in East Sumba.
Meanwhile, the second follow up will capture longer-term outcomes and allow more time for challenges to
arise to project sustainability that may not be captured after only one year. It will be important to track progress
of other electrification efforts in the target areas prior to follow-up data collection. In the case of large scale
electrification of comparison areas, the design may need to be reconsidered.
A third component will be added to the evaluation during the two follow-up data collection activities: an ex-
post evaluation of the W1 Hivos Solar/Biogas, Sumba/Sulawesi grant. The evaluation of this grant, like the
one in Berau, will include quantitative (household survey) and qualitative (KII and FGD) elements that are not
qualified by a comparison group of non-treated households. Although it was not financially or logistically
feasible to include this evaluation in the scope of baseline data collection, it has a unique implementation
model in a shared geographic area with W3A Anekatek that will allow for interesting comparisons in terms of
typical outcomes of RE programming and sustainability. Specifically, data from the first follow up will allow for
a comparison of outcomes between programming using off-grid RE technology and programming using micro-
grid RE technology in East Sumba. Data from both follow-ups will enrich the analysis of sustainability of CBOG
RE grants by allowing for the comparison of a RESCO model to an SPV model in the same geographic area.
Table 7 demonstrates how the evaluation’s two initial major components will collectively serve as the
foundation for responding to these four evaluation questions. The third component is not included in the table
because its instruments will not be developed until the first follow up data collection period.
Table 7: Evaluation Design Overview

EQ Key Outcomes Data source, location Data type
Household and enterprise energy Household survey, Quant.

Community KII/FGD, Qual. Quantitative,
1 consumption (by source), energy
Enterprise Survey, Quant. Qualitative
expenditures
Enterprise KII, Qual.
Household survey, Quant.
Productive uses of electricity, SPV KII protocols, Qual.
Quantitative,
2 occupational and transformed Community KII/FGD, Qual.
Qualitative
agricultural income, employment Enterprise Survey, Quant.
Enterprise KII, Qual.
Household survey, Quant. Quantitative,

3 Greenhouse gas emissions
Gov. official KII, Qual. Grantee KII, Qual. Qualitative
Capabilities of SPV members,
4 All qualitative instruments Qualitative
sustainable operation of facilities
SOCIALIMPACT.COM 47
In choosing which grants to include at minimum using a pre/post methodology in this portfolio evaluation, we
placed the highest emphasis on which grant would lend itself the most to an impact evaluation design, since
such a design is essential to providing valid quantitative responses to evaluation questions 1-3. On this
question W3A Anekatek Solar, East Sumba was the only suitable candidate. All of the Window 3A grants, as
described in the previous section, targeted whole villages in a way that made a household-level experiment
impractical. As such, any grant that could be evaluated quantitatively needed to provide adequate treatment
clusters with similar control clusters nearby. Since W3A Anekatek Solar, East Sumba is operating in eleven
sub-village units with comparable analogs in geographic proximity, we selected it as the subject of our impact
evaluation. The other Window 3A grants were either providing treatment to all villages on an island, for fewer
communities, and/or for relatively unique communities with few options for similar comparisons nearby.
The utilization of an SPV approach for community engagement and sustainability of program outputs is a
fundamental aspect of the design of the Window 3A grants. Any evaluation of the GP Facility’s approach to
community-scale RE programming must evaluate the extent to which the SPV approach contributes to the
achievement (or lack thereof) of program outcomes. This approach differs in specific details and contextual
factors from grant to grant, so we selected the remaining grant with the most compelling potential narratives
in terms of community engagement for a performance evaluation to combine with the impact evaluation of
W3A Anekatek Solar, East Sumba.
On this count, all of the other grants have merits. However, W3A Akuo Energy Solar/Micro-Hydro, Berau has
a variety of factors that will make for interesting qualitative comparison. First, it has a diverse set of villages
for implementation that have varying degrees of history with community cooperatives and distinct socio-
economic backgrounds. Second, the grant includes a micro-hydro component—albeit quite small in the
context if the capacity provided by the Solar PV facilities—that may provide for interesting comparisons with
community management of Solar PV components alone. Finally, it is in a different geographic area from W3A
Anekatek Solar, East Sumba, allowing for a comparative study of how similar program logic applies in different
geographic contexts. By investigating process, outcomes, and sustainability across these two grants, we can
qualitatively explore a variety of factors that mediate results and sustainability.
As stated above, W1 Hivos Solar/Biogas, Sumba/Sulawesi was selected as an ex-post add-on to the
evaluation to broaden the types of CBOG RE programming informing the evaluation’s answers to its core
questions. Specifically, it will allow for the inclusion of off-grid RE technology and a RESCO business plan in
a comparison of program outcomes and sustainability in common geographic settings.
3.2.2 Optional Portfolio Evaluation Add-Ons

It would be ideal to include more grants representing a broader portion of the CBOG RE portfolio in our scope
of inquiry, but this is not financially or logistically feasible at baseline. Overall, we expect the trio of grants we
have selected to produce learning relevant to the CBOG RE portfolio at a broader scale. Funds permitting, it
may be possible to broaden the scope during future data collection periods to cover even more nuances of
what grants within this portfolio were able to achieve and how.
From a technical perspective, it is reasonable to believe that the other 23 grants would provide valuable
learning and increased accountability for MCC utilizing an ex post evaluation design. As none were suitable
for an impact evaluation, it is not necessary to deploy a pre/post evaluation approach to maximize learning
SOCIALIMPACT.COM 48
and accountability from each grant since such an approach would not have been technically feasible to yield
valid, quantitative impact estimates on outcome variables of interest.
Ex post investigations of grant implementation and changes in outcomes of interest compared to grantee
baseline data would provide increased accountability compared to grantee data, alone; and comparative case
studies between grants could yield relevant learning for similar programming in the future. These comparisons
could contrast grants with similar approaches in different geographic areas (as we aim to do with our selected
grants at baseline) or they could contrast grants with different approaches in the same area (as we aim to do
with the inclusion of the W1 Hivos grant during the follow-up periods). Table 8 outlines some non-exhaustive
examples of lines of inquiry which could be pursued based on SI’s initial review of documentation. These lines
of inquiry could be added to either follow-up data collection period, although the second follow up would likely
be more appropriate for themes related to sustainability.
Table 8: Possible Add-On Lines of Inquiry
Additional line of inquiry Grants that could be compared
How do outcomes of interest and sustainability of Window W3A Anekatek Solar, East Sumba vs. W3A Sky Energy
3A-style programming compare between large islands and Solar, Karampuang Island
small islands?
How do increases in access to RE affect the profitability and W3A Akuo Energy Solar/Micro-Hydro, Berau vs. W3A Puriver
sustainability of ecotourism ventures? Solar, Tomia Island
How does the scale/capacity of RE installations deployed W1 HiVOS Solar/Biomass, Sumba/Sulawesi vs. W2 Green
(e.g. household-level solar lamps or biogas digesters vs. Sumba Solar, Central Sumba
school- or community-level micro-grids) affect household-
and community-level outcomes of interest and sustainability?
Does pairing increased access to RE with improved NRM Any Window 3 grant to any Window 2 grant
increase community engagement or improve the
sustainability of programming?
How does the type of RE technology deployed affect W1 HiVOS Solar/Biomass, Sumba/Sulawesi vs. W2 IBEKA
outcomes of interest and/or sustainability via community Micro-Hydro, East Sumba vs. W2 Green Sumba Solar,
engagement? Central Sumba
How has local government policy enhanced or impeded the W1 HiVOS Solar/Biomass, Sumba/Sulawesi vs. W2 Yayasan
sustainability or replicability of RE programming in different Dian Tama Pontianak Solar, Kapuas Hulu vs. W3A Charta
areas of Indonesia? Putra Biomass, Siberut Island
Which new enterprises have successfully demonstrated Multiple, depending on which target and produce successful
productive uses of RE due to GP CBOG RE programming? enterprises
What did these enterprises have in common?
3.3 Quantitative Approach

3.3.1 Methodology
To answer evaluation questions 1-3 which seek to identify the impact of the RE installations, we compare the
outcomes of individuals who have received increased access to electricity through RE sources against the
SOCIALIMPACT.COM 49
counterfactual: the outcomes for these same individuals, if they had not received increased access to RE
sources. Since it is not possible to directly observe the counterfactual, we need a mechanism to estimate it
with as little bias as possible. The ideal method is to randomly assign participation among a sample of potential
participants, creating a treatment and control group. Through random assignment, the treatment and control
groups, on average, are expected to be similar along the characteristics affecting the outcome of interest.
Hence, in the absence of the project, both groups would have the same expected outcome and any differences
between the two groups after project implementation can be attributed to the project.84
For the grants we are evaluating, including Grant W3A Anekatek Solar, East Sumba, participation is not
randomly assigned. Rather, sites were purposefully selected for installation of RE, as described above. One
means of randomization would have been to randomly assign connections (or randomly offer discounted
connection fees to generate random variation in connection status) to the micro-grids within selected villages.
However, for political, logistical, and ethical reasons, nearly all households in selected communities will receive
free connections to the micro-grid, with only very remote households not being offered a connection. Thus, SI
will utilize a quasi-experimental approach which incorporates elements of statistical matching techniques and
difference in differences (DiD) to estimate counterfactual outcomes and program impact for the W3A Anekatek
Solar, East Sumba grant.
We propose to collect panel data from a sample of treatment and comparison households, with the evaluation
sample identified using the following approach:
1.) Identification of comparison kampungs: Given that nearly all households in the 11 treatment
kampungs will be electrified and the few that do not are systematically different, we must identify
comparison households from other kampungs in the same desas or in nearby desas. To do this, we
developed a sample frame of nearby kampungs that (1) had, like the treatment kampungs, been
classified as suitable for a micro-grid according to a recent network planning activity conducted by the
ADB (described below) and (2) based on discussions with key stakeholders, were not slated to receive
electrification in the following year through other planned initiatives, including through Window 2
grants. From this sample frame, we used data on population size and geographic distance to identify
a sample of 17 comparison kampungs. Comparison kampungs (relative to treatment kampungs) were
oversampled in order to increase power (given the fixed and limited number of treatment kampungs),
to generate a buffer in case a small number of comparison kampungs are electrified during the
evaluation period, and to provide a larger pool of potential comparison units from which to draw
matches.
2.) Baseline data collection: Within each treatment and comparison kampung we sampled, on average,
30 households, as described below in Section 3.3.3.
3.) Match similar treatment and comparison households: To generate the final sample of households
for the evaluation, we will use statistical matching techniques to identify similar treatment and
comparison groups. We will conduct two types of matching at the household level, Coarsened Exact
Matching (CEM) and Propensity Score Matching (PSM), and select the matching technique which
84
Assuming a well-run experiment without spillovers, differential attrition, Hawthorne effects, etc.
SOCIALIMPACT.COM 50
maximizes the comparability of the groups, statistical power of the comparison, and external validity.
Given the potential for electrification in comparison areas, which would exclude the electrified
community from the evaluation comparison sample frame, we recommend that final matching is
conducted prior to follow-up data collection. However, we will present the results of a tentative
matching exercise at baseline to illustrate how the groups can be made more comparable.
Follow up data collection with the final matched sample will be conducted once twelve months after the micro-
grids are commissioned and then again thirty-six months after commissioning. As described below, we will
then analyze the results using a DiD regression approach.
The initial selection of similar kampungs and matching of treatment and comparison households helps to
reduce selection bias by minimizing differences along observed household and community characteristics
measured at baseline. However, all matching methods rest on the Conditional Independence Assumption
(CIA). That is, we assume that conditional on the vector of baseline characteristics used in matching, the
expected outcomes of the treatment and comparison groups are independent of the assignment, and selection
bias is removed. However, the potential exists that unobserved variables will differ across the treatment and
comparison group, thus violating the CIA. The DiD approach to analysis will serve to reduce the threats posed
by unobservable differences between the households that do not vary over time.
Also, there is a tradeoff in CEM between the level of coarsening and power that is similar to the common
support condition or assumption other matching approaches. With very fine coarsening of variables
(separating them into higher numbers of strata), we increase the number of strata and reduce the likelihood
of matches. This leads to pruning higher numbers of observations which reduces sample size and power and
limits our ability to generalize to the full evaluation sample (or to those pruned observations). However, if we
use only very loose coarsening of variables (separating them into fewer strata), we increase the likelihood of
matches, preserving a larger proportion of the evaluation sample, but we risk retaining a greater degree of
imbalance between treatment and comparison units. We propose a systematic approach to variable selection
and degree of coarsening, as described below in the baseline analysis section, which optimizes the tradeoff
between imbalance and power.
3.3.2 Timeframe of Exposure

Since the methodology employed to evaluate W3A Anekatek Solar, East Sumba in East Sumba will rely most
critically on an appropriate timeframe of exposure to treatment, SI will select the timeframe of exposure on the
basis of this grant.
Based on discussions of historical implementation of similar programming in East Sumba, SI expects that
beneficiaries will switch from baseline consumption sources (e.g. kerosene, diesel generators) soon after the
commissioning of the new solar micro-grid facilities. In the past, it has taken longer for communities to innovate
productive uses of new RE technology, but stakeholder interviews suggest this should occur starting within
three to six months of exposure to the program. Besides the exposure to treatment required for positive
outcomes to manifest, it is additionally necessary to consider how long it would take for O&M or administrative
challenges to occur that would put the sustainability of grant outputs at risk. The final consideration is the
potential for contamination or electrification of comparison villages, which increases over time.
SOCIALIMPACT.COM 51
With these aspects in mind, SI suggests waiting a full year to collect follow up data for the evaluation. This will
allow sufficient time for productive uses and/or operational challenges for the micro-grids to develop and be
considered in the measurement of program outcomes. Additionally, this will allow for follow up data collection
events to occur during the same season, cancelling out any unobserved bias from seasonal effects on energy
consumption and income.
Given that sustainability and O&M concerns become more pronounced over time, we highly recommend a
smaller, second follow-up three years after installation to further investigate evaluation question 4 and observe
longer-term outcomes of interest in the other evaluation questions.
3.3.3 Study Sample

3.3.3.1 Sample unit
The quantitative portion of the evaluation will take a clustered approach, where individual sample units include
households that are clustered into either “settlement aggregations” in East Sumba or villages/desa in Berau.
For the most part, the settlement aggregations in East Sumba are sub-village units sometimes referred to as
kampungs or RT. Occasionally, a settlement aggregation will encompass a whole village.
3.3.3.2 Sample size
The sample in East Sumba will include approximately 840 households (330 treatment and 510 comparison)
clustered into 11 treatment settlement aggregations and up to 17 comparison settlement aggregations. The
sample in Berau will include approximately 150 households clustered into 3 treatment villages.
3.3.3.2.1 Power Calculation and Assumptions
Given the clustered nature of the intervention and sample, we will collect data from all 11 treatment settlement
aggregations and up to 17 comparison settlement aggregations in East Sumba. To determine the number of
households to sample in each settlement aggregation, we must estimate the intra-cluster correlation (ICC) for
key outcomes and then look at the relationship between minimum detectable effect size (MDES) and cluster
size at the estimated values of ICC. To estimate ICC, we use data from Castlerock’s baseline survey in the
W3A Anekatek Solar, East Sumba target villages and calculate values ranging from 0.00 to 010 (see Table
8).
SOCIALIMPACT.COM 52
The graph in Figure 9 displays the

relationship between cluster size and MDES
using the highest estimated ICC values
(0.06 and 0.10) as well as assuming 22 total
cluster, power of 80 percent, Alpha
(α)=0.05, and R-squared=0.2.
Unsurprisingly, we find an inflection point
around approximately 15 households with
diminishing returns to power for additional
households per cluster beyond that. Given
this relationship and the fact that five of the
eleven treatment kampungs have between
27 and 41 households, we recommend a
sample size of approximately 25
households per cluster85, which
corresponds to an MDES of 0.37 and 0.43
for ICC values of 0.06 and 0.10,
respectively. Based on the baseline data,
this corresponds to an ability to confidently
Figure 11: Relationship between cluster size and MDES
measure a change in monthly electricity
expenditure of at least approximately 60,000 to 70,000 IDR (Indonesian Rupiah) or a change in electricity
access of 1.52 to 1.77 hours per day.86
Table 9: Power Calculation Summary Statistics percent
MDES=0.35 MDES=0.41
Outcome Mean SD ICC
(ICC=0.06) (ICC=0.10)
Monthly electricity expenditure
82,660.93 161,915.2 0.06 59,909 69.623
(IDR)
Monthly kerosene usage (liters) 1.73 32.93 0.00 12.18 14.16
Monthly kerosene usage for
0.43 2.06 0.02 0.76 0.88
lighting only (liters)
Electricity access per day (hours) 3.23 4.12 0.10 1.52 1.77
To account for attrition and pruning during the CEM process, we propose to inflate this sample by 20 percent
at baseline, yielding a total baseline sample size of approximately 840 households in Sumba.
85
Only 2 treatment kampungs have fewer than 25 households.
86
The grant’s CBA indicates that expected benefits include a 19,583 IDR per month reduction in energy expenditures and an increase in energy consumption of
39.19 kwh/month. We would be adequately powered to detect such a change in consumption, although we may not be adequately powered to detect changes in
expenditure unless they exceed those predicted in the CBA.
SOCIALIMPACT.COM 53
Since our evaluation design in Berau will not include a counterfactual approach (e.g. we will not be making
comparisons between a treatment and control group), there is no need to do a power calculation. The sample
size of 150 households has been selected because this number would be adequate to pull representative
samples from each village.
3.3.3.3 Sample frame
Since treatment units have already been selected by the grantee in East Sumba, the sample frame for W3A
Anekatek Solar, East Sumba will include all 909 total households among the 11 treatment kampungs. To
construct this sample frame, we will request a list of these households from the implementer.
For the comparison group, the sample frame will include all settlement aggregations in East Sumba that satisfy
the following conditions:
1.) The Network Planner Activity of ADB TA 8287 indicates that the settlement aggregation was best
suited for electrification via micro-grid or off-grid technology;
2.) The settlement aggregation does not include households that are currently connected to the PLN grid;
and
3.) The settlement aggregation is not targeted by PLN for electrification until after September of 2018.
After selecting settlement aggregations from this sample frame, the household sample frame will be
constructed by requesting a list of all the households in each settlement aggregation.
The sample frame for household data collection in Berau will include all households that will be connected to
the solar or micro-hydro micro grid. This includes 463 households among three villages. We will request a list
of these households from the grantee.
In both kabupatens, the sample frame for enterprises will be constructed by asking local officials upon arrival
about the location of enterprises in each kampung.
3.3.3.4 Sampling strategy
For the evaluation of W3A Anekatek Solar, East Sumba, we will use a random sampling strategy from the
sample frame in treatment areas where settlement aggregations include over 25 households. Where
settlement aggregations include fewer than 25 households, replacement households will be selected randomly
from other treatment settlement aggregations.
Since the objective of selecting comparison settlement aggregations is to match the treatment aggregations
as closely as possible (and not to represent the entire sample frame of potential comparisons), settlement
aggregations will be selected using a non-random method. Specifically, we will calculate the distance
between each of the settlement aggregations that meets the conditions from the list in the previous section
and each of our eleven treatment settlement aggregations and select the seventeen which are closest to a
treatment settlement aggregation, under the assumption that these would be the most similar on important
characteristics in the absence of any other data.87 For the selection of comparison households within
87
The only data in our possession on these settlement aggregations prior to the baseline survey are GIS coordinates
and population figures, so we will verify that the distribution of aggregations on each of these characteristics is
similar to the treatment kampungs prior to sampling.
SOCIALIMPACT.COM 54
comparison aggregations, we will use the same random sampling technique as will be used for treatment
households from lists of households obtained by local officials in selected settlement aggregations. The final
list of settlement aggregations selected for sampling in East Sumba is presented in Table 10.
Table 10: Sampled settlement aggregations and households, Baseline
No. Kampung, Kecamatan - Treatment Kampung, Kecamatan - Control
1 Tawui Northeast, Pinu Pahar Kalimbu Maramba, Mahu
2 Tawui Riyang, Pinu Pahar Tara Amah, Mahu
3 Tawui West, Pinu Pahar Mauhani, Paberiwai
4 Tawui North, Pinu Pahar Pahulu Bandil, Matawai La Pawu
5 Rehi Jara, Karera Lumbuwudi, Pinu Pahar
6 Praiwitu North, Ngadu Ngala Pingi Ailun, Matawai La Pawu
7 Tanah Rong, Karera Linggi Tana, Paberiwai
8 Praiwitu South, Ngadu Ngala Prai Kalu, Paberiwai
9 Tandula Jangga, Karera Laipabundu, Pinu Pahar
10 Lailunggi, Pinu Pahar Undut Maringging, Pinu Pahar
11 Tawui South, Pinu Pahar Rakamau, Pinu Pahar
12 Winumuru, Paberiwai
13 Matawailuri, Pinu Pahar
14 Pada Djara, Ngadu Ngala
15 Prai Maninggat, Paberiwai
16 Laironja, Matawai La Pawu
17 Dusun 2, Matawai La Pawu
Total Total
For W3A Akuo Energy Solar/Micro-Hydro, Berau, households will be sampled using a simple stratified
random sampling technique. The strata will include the three treatment villages, from each of which fifty
households will be randomly selected.
In both kabupatens, up to eight enterprises will be sampled for the enterprise survey per treatment unit.
If fewer than eight enterprises exist, all of them will be surveyed. If more than eight exist, enterprises will
be selected for the survey purposively to cover a broad cross-section of industries.
SOCIALIMPACT.COM 55
3.3.4 Primary Data Collection

3.3.4.1 Instruments
The quantitative approach described above will rely on both household surveys and semi-structured interviews
with enterprises, community leaders, SPV members, and other key stakeholders. The household survey is
described here while the principal semi-structured interviews for the quantitative component are introduced
here and described in more detail below in the qualitative section. At this stage, the household surveys are
the same for both grants (W3A Akuo Energy Solar/Micro-Hydro, Berau and W3A Anekatek Solar, East Sumba)
though we may make contextual revisions as we gather additional grant information and conduct piloting.
The household survey covers all relevant dimensions of the household that might be affected by the new
access to electricity or that might affect the adoption and usage of electricity. The socio-economic living
conditions will be elicited ranging from background variables like age, household size, and educational status
of adult members to variables that potentially change after electrification, for example employment status,
educational investments of children and expenditures. A particular focus is on energy consumption and usage,
i.e. different energy services, fuels, expenditures, and appliances. Moreover, the questionnaire probes into
the activities related to energy usage, for example activities after nightfall, TV usage and appliances. Attention
is dedicated to income generating activities. More specifically, the household questionnaire includes the
following sections and can be found in Annex 3:
Instrument Outcome Area Specific Topics Description/Use

‐ Housing conditions and size
Covariates for energy spending and
Household ‐ Household roster with
consumption as well as potential
Information education
outcomes in household assets
‐ Transportation assets
Covariates for energy spending and
Migration ‐ Migration roster potential outcomes in economic
migration rates
‐ Source of energy
Verification of increases in access to
Energy Sources ‐ Household energy use
RE; key outcomes in energy usage and
and Use ‐ Energy use and spending by
spending
source
‐ Agricultural income by
Household
Agriculture and product Outcomes related to increases in
Survey
Livestock ‐ Livestock assets and agricultural income
income
‐ Banking and savings status Outcomes related to decreases in
Financial Situation
‐ Remittances poverty measured through increased
and Expenditures
‐ Household expenditures expenditures
‐ Time spent on productive, Outcomes related to time use, including
leisure, and household productive, leisure, studying, and
Activity Profile
activities (and studying, for household chores disaggregated by
children) men, women, and children
‐ Experience of key health
Outcomes related to improved health
Health issues including ARI (acute
status due to cleaner energy use
respiratory infection)
SOCIALIMPACT.COM 56
Instrument Outcome Area Specific Topics Description/Use

‐ Mobility and safety at home Outcomes related to increased safety
Security
in night due to access to lighting at night
‐ Attitudes related to Outcomes of community awareness
Green knowledge
environmental practices activities
Electricity access ‐ Type of electricity available Verify increased access to RE
Covariate and important outcomes
‐ Key services offered
Services related to increased services offered
‐ Customer base
and expansion of customer base
‐ Appliance use by energy
Energy Use source Outcome related to productive use of RE
Enterprise
‐ Lighting by energy source
Outcomes related to business
Employment ‐ Employment by type
expansion
Qualitative discussion to explore role of
Production and ‐ Production, expansion, and
energy access in business constraints,
Expansion bottlenecks
investments and expansion
‐ Population size and number
Demographics Covariates of household outcomes
of households
‐ Transportation
Infrastructure and
Community ‐ TV, radio, mobile networks Covariates of household outcomes
services
Leader ‐ Social infrastructure
‐ Energy sources and prices Covariates of household outcomes and
Energy sources
in the community verification of treatment
‐ Enterprises operating in the
Income generating Outcome related to increase productive
community
activities use of energy
‐ Market access
Semi-structured protocols will be held with the chief of the respective sub-villages or a sub-village member
with good knowledge on the population and village dynamics. The protocol comprises modules on basic sub-
village information, availability and quality of infrastructure and services, energy access and use patterns, and
detailed sections on income generation in the sub-village. Lastly, it includes information on community
engagement, development programs and subjective community well-being. Given the semi-structure, the
protocol allows for flexibly gathering village-specific information in-depth and for learning about unexpected
circumstances or developments.
Similarly, semi-structured interviews will be held with all microenterprises of the sub-village. In case of large
enterprise numbers, a non-random sample will be chosen, which includes all different types of enterprises, for
example welders, bakers, shop owners, or carpenters. The protocol includes modules on basic enterprise and
customer information, energy use and production processes, and employment patterns. It is designed to
capture growth potentials of the enterprise, which might be unlocked by electricity access. Particular attention
is given to understanding growth hindering bottlenecks and potential net effects of electrification for the local
economy.
The present version of the questionnaires are draft versions which we designed based on experiences gained
in previous work on electrification in Indonesia and elsewhere. It will be revised based on further document
review, discussions with key stakeholders and after pre-tests in the field.
SOCIALIMPACT.COM 57
3.3.4.2 Rounds and timing

The baseline will be implemented in October and November 2017, with an initial follow-up one year later in
order to track adoption behavior and investigate initial changes on key outcomes. However, many impacts
may evolve over time and technical-economic sustainability issues of the micro-grid materialize rather in the
mid/long-term. Accordingly, we recommend a final follow-up survey around three years after the baseline.
We will revisit the sampled communities in the same season as in baseline to avoid seasonality distortions
and will also have an eye on potential delays in the installation of the micro-grids, which – if substantial –
would entail a rescheduling of the follow-up survey.
3.3.4.3 Respondents within the sample unit
A priori, the desired respondent is the person most responsible for decisions related to energy use and
expenditures, likely the household head. If this person is unavailable, we would permit the survey to be
conducted with another adult household member who is involved in and informed of decisions related to
energy use. We envision maintaining the same respondent for the questions on activity profile, but will consider
the benefits and feasibility of engaging an alternate respondent for that section during pre-testing.
3.3.4.4 Staff
Our Program Manager, Mike Duthie, will lead the quantitative data collection effort, including participating in
instrument piloting and enumerator training. He will be supported by two Research Assistants, Miguel Albornoz
(SI-HQ) and a local researcher that has yet to be determined. Dr. Jörg Peters and Ms. Luciane Lenz are
expected to be in the field leading the qualitative component shortly before the initiation of the household
surveys. Mr. Duthie and Dr. Peters will collaborate to ensure integration across the two efforts. One of SI’s
local research assistants will be trained to provide field monitoring of data collection in both locations under
the direct guidance of Mr. Duthie and Mr. Albornoz.
SI will competitively procure a local data collection company to conduct the field work, and we have already
sent out and received responses to a request for proposals to conduct the surveys. SI will provide expert
guidance in a comprehensive training, at least five days in duration and including field practice, to all field staff
employed by the data collection company.
3.3.4.5 Data processing
Since we intend to conduct electronic data collection, we expect to receive data regularly throughout field
work88, which we will import into Stata and using the SI-developed errout Stata command will check for a
variety of common logic, range, missing value, skip, and outlier errors. This can be conducted in near real time
and generates a log of errors for discussion and verification with the data collection partner, as well as for
further training of staff on common errors. Once SI receives the final dataset, we will conduct data cleaning,
again checking for missing data; logic, range, and skip errors; and outliers, using Stata do and log files.
Identified issues will be discussed with the data collection partner for verification and any changes will be
entered into do files with notes explaining the change. Relevant variables will be transformed for analysis. All
88
Given that we do not expect data collection field work for the survey to last more than two or three weeks, it may not be logistically practical to get interim data
sets, conduct quality checks and feedback information prior to completion of field work, but this will be discussed with the data collection firm as a priority.
SOCIALIMPACT.COM 58
data cleaning, management, and analysis will be conducted through Stata do files to ensure transparency and
reproducibility of results.
3.3.4.6 Data quality
While the specific data quality assurance protocols will be agreed with the data collection partners, the
following represents SI’s standard approach and can be considered representative of the approach we will
take. We expect to conduct electronic data collection which permits regular, timely verification of data quality,
logic and range check in data entry, and additional quality assurance checks related to automatic time stamps
and geocoding.
Data Quality Assurance processes will occur in the field, in real-time, during data collection and during data
entry and in delivery of datasets. The data collection company will provide significant oversight of enumerators
in the field. Specifically, they will provide on-site management of enumerators that is sufficient to observe the
activities of the interviewers, identify problems in their administration of the questionnaires, and correct those
problems. The data collection partner will ensure that all administered surveys are checked at the conclusion
of each day by field supervisors to ensure that they are complete and devoid of inconsistencies. The partner
will be responsible for implementing quality monitoring processes and will identify key personnel ultimately
responsible for data quality. Specific activities include:
‐ A supervisor will accompany 5percent of survey interviews to ensure completeness and to monitor
and record any discrepancies or abnormal responses.
‐ A supervisor will monitor the sampling process and location of completed surveys and should
immediately notify SI upon discovery of any irregularity;
‐ Supervisors will review nightly their interviewers’ instruments to ensure appropriate skips are
accurately followed and answers are properly recorded;
‐ The partner will conduct spot-check interviews of 5percent of surveys, by re-visiting or re-calling
respondents and verifying responses to a subset of 10-20 survey questions;
‐ Full re-interviews will be conducted by supervisors in the event that any interviewer is suspected of
fraudulent behavior;
‐ Weekly summaries of data quality control activities shall be submitted to SI, in addition to a final tally
of interview observations, re-visit spot checks, and complete re-interviews at the completion of data
collection.
‐ SI staff or designates will also conduct independent quality assurance.
At the conclusion of data collection, the partner will deliver a data quality summary with the final dataset. This
will include information about challenges in data collection, any modifications to the data collection protocols,
data quality process, identification of any data quality issues, as well as metadata about the final dataset
(sample replacement, response rate, attrition, average duration of survey, etc.) SI may provide further detailed
outline as needed but data quality reports will include at least the following information:
‐ Data source
‐ Sample size
‐ Samper size of pilot(s)
‐ Dates of pilot(s)
‐ Dates of data collection
SOCIALIMPACT.COM 59
‐ Number of enumerators
‐ Number of supervisors
‐ Number & percent of randomly selected survey responses audited by field supervisor
‐ Number and percent of randomly selected survey responses audited by the firm
‐ Average number of surveys conducted per enumerator per day
‐ Summary of quality checks performed during fieldwork
3.3.4.7 Summary Table
See the summary table included above in Section 3.3.4.1.
3.3.5 Secondary Data

This evaluation will primarily use secondary data in support of our sampling strategy. Due to the sub-village
nature of treatment in East Sumba, most secondary data is not representative at an adequate level to serve
as a covariate in analysis. SI will utilize the following data in support of our sampling approach:
1.) GIS (Geographic Information System) shapefiles of settlement aggregations in East Sumba generated
by ADB TA 8287
2.) Network Planner data generated from the Midline Report of ADB TA 8287
3.) List of desas where PLN is implementing micro-grids in 2017 and 2018
4.) Variables from blocks 4 (Population and Employment) and 12 (Economy) of the 2014 PODES (Village
Potential Statistics) Survey
The first two of these datasets are critical to the development of our sampling frame. They identify treatment
and potential comparison clusters and characterize key conditions of the clusters, namely their population and
their suitability for electrification via micro-grid technology. The third dataset provides critical information for
our sampling approach by identifying potential future contamination sites. The final dataset may be helpful in
providing covariates for sampling or balancing at baseline.
All of this secondary data has been transferred to SI by its owner using appropriate and secure channels. Prior
to using the data from ADB TA 8287, SI assessed its quality by reviewing the methodological section of TA
deliverables and asking follow up questions of staff who collected the data based on any inconsistencies
found. There was no need to independently verify the quality of PLN or PODES data, since these were
generated by qualified actors (PLN and BPS (Badan Pusat Statistik in Indonesia)).
3.3.6 Analysis Plan

3.3.6.1 General Approach
To analyze the project’s impact on key outcomes of interest, we must first verify that the project achieved its
intended outputs. Prior to conducting econometric analyses, we will use a combination of project monitoring
data, SPV records (as available), information gathered from key informant interviews, and household survey
data to determine whether the micro-grids have been successfully installed and that households have been
connected. This will help establish whether the project has indeed increased access the RE. At baseline,
verification of outputs is not applicable.
SOCIALIMPACT.COM 60
Following this, we will analyze the impact of the project on our key indicators, including energy consumption,
energy spending, and household economic and time use outcomes, using statistical and econometric models.
Note that baseline statistical analysis focuses on associations of factors with outcomes rather than analyzing
impacts.
This study is powered to report impacts for all treatment households. We may also explore outcomes for critical
sub-groups, such as poorer households or female-headed households to test the heterogeneity of impact. In
some cases, for example, with female-headed households, estimates of sub-group differential impact will be
made with reduced precision and power due to the smaller sample size available for that sub-group.
3.3.6.2 Baseline
The focus of baseline analyses will be to both identify the matched sample of treatment and control households
and to investigate the current status of, and factors associated with, energy consumption and expenditures in
the target area.
Identification of matched treatment and control households
We will use the CEM approach described above with baseline data to match treatment and control households,
thereby identifying the final sample for the evaluation. Specifically, we will:
1. Identify secondary and baseline variables that correlate with treatment and key outcomes, with
a specific focus on energy consumption and expenditures. To look at variables associated with
treatment, we will estimate a logistic regression, whereas we will use a linear regression model to look
at factors associated with energy consumption and expenditures. Candidate variables will include
‐ household variables such as sex, age, education, and employment of the household head;
household and home size; and household asset index; and
‐ community level variables such as an index for community resources, population size/density,
and geographic location (for example, travel time to the district capital).
Pending the results, we envision selecting each variable that is statistically significant in either model,
though we may include more restrictive criteria if we find that many variables are significant in either
model. Note that this analysis is also a critical input for the second key focus on the baseline analysis
described below.
2. Develop bin sizes for CEM. As a starting point, we will think critically, and based on the literature,
about appropriate bin sizes for each variable. However, given that there are not natural bin sizes for
most of the variables we expect to include in the matching (with the exception of sex of household
head, for example), we will develop a few sets of bin sizes that range from fine to loose coarsening.
3. Conduct CEM based on each set of bin sizes. Using the Stata CEM command, we will match units
under each bin size scenario, pruning observations that fall into treatment-only or control-only strata.
4. Determine the most appropriate bin size scenario. Given the tradeoff between level of imbalance
and power in matching approaches, we will investigate each bin size scenario according to the
following criteria:
‐ Imbalance: We will measure the average absolute standardized difference in means for
variables included in the CEM and other variables associated with the outcomes of interest.
‐ Power: We will recalculate the MDES and power based on the number of households pruned.
SOCIALIMPACT.COM 61
‐ External validity: To investigate the representativeness of the sample to the target population,
we will conduct t-tests to look for differences in means between the pruned sample and full
treatment sample and then calculate an average t-statistic for each bin size scenario.
We will document the results of each of these tests and based on these criteria, the evaluation team
will decide which bin scenario offers the optimal tradeoff.
5. Identify final sample. Based on the selected bin size scenario, the final evaluation sample will be all
non-pruned households. This represents the sample of households for which follow up data collection
and analysis should be conducted. We will present in the baseline report the similarity of the matched
treatment and comparison group, alongside balance in the unmatched groups for reference.
Investigate the current status of energy consumption and expenditures in the target area
The baseline analysis will also be useful in documenting the pre-intervention status in the target area. This
will include:
1. Descriptive statistics of outcome variables and covariates, by socioeconomic status and in some cases
by age or gender.
2. Econometric analysis of factors associated with key outcomes: Regression analysis will be used to
explore the relationship between outcome variables and covariates, and test posited relationships from
the project logic.
3.3.6.3 Follow-up
Follow up analysis will focus on estimating the one-year impacts of the grant using a DiD regression approach
with controls. We will also provide a review of outputs to ensure that the project did indeed increase access
to RE. Specifically, this will include:
1. Descriptive statistics of outcome variables and covariates, by socioeconomic status and in some cases
by age or gender.
2. Analysis of determinants of outcomes: Regression analyses built on baseline models by examining

changes in the marginal effects of household and individual demographic and economic variables on
select outcomes of interest, such as energy consumption and energy expenditures. This builds on the
baseline analysis, looking further at which variables (and how changes in those variables) are related
to the outcomes of interest. While the relationships cannot be confidently considered causal, they may
be instructive in identifying additional questions and research in household energy consumption and
expenditure dynamics.
3. Analysis of output data: Through analysis of project monitoring data, household survey data on
electricity availability, SPV documents, and key informant interviews, we hope to establish whether
the grant was effective in increasing access to RE. If we find, for example, that the RE systems are
non-operational or systems suffer from significant shortage in supply or that comparison areas have
also gained access to RE, then we might expect null or limited results on key outcomes.
SOCIALIMPACT.COM 62
4. Statistical analysis of one-year impacts89: Using the matched sample, we will estimate program
impacts using the following fixed effects panel regression framework:
∙ ,
where Yijt is the outcome of interest for household or other unit i in kampung j at time t; d represents
treatment assignment and is equal to 1 if kampung j is assigned to treatment, and 0 otherwise; Tjt
represents time and is equal to 0 at baseline and 1 at follow-up; Xijt is a vector of time-varying variables
that affect the outcome for unit i in kampung j at time t, and δijt is a time-varying error term. The
coefficient κ will measure the “treatment effect,” or the change in outcome Y for treatment households
or enterprises relative to that for controls. This estimate is unbiased so long as the error term δijt is not
correlated with treatment.
3.4 Qualitative Approach

3.4.1 Methodology
To provide additional data to answer evaluation questions 1-3 (which seek to identify the impact of the RE
installations) and to answer evaluation question 4 (which seeks to identify the level of effectiveness of SPVs
at managing the Solar PV facilities), SI will collect baseline and endline qualitative data in treatment sites
representing W3A Anekatek Solar, East Sumba and W3A Akuo Energy Solar/Micro-Hydro, Berau grants as
well as conduct a mini-survey in W3A Akuo Energy Solar/Micro-Hydro, Berau areas (described in the
quantitative section above). The pre-post PE approach will rely on primary qualitative data from key informants
and focus group participants at the regency and village levels. Observation of SPVs and facilities may be
included at endline, but is not applicable at the baseline.
Pending availability of funds, SI also suggests a follow-up PE (post-intervention, qualitative data collection
only) of other grants referenced in this report (Window 2 and other Window 3A grants not selected for the
baseline) in addition to another round of endline data collection 2-3 years after installation in Berau and East
Sumba to further explore the factors that contribute to micro-grid success, particularly looking at evaluation
question 4. A follow-up PE would allow more time for failure to potentially identify additional factors of success
and failure of the uptake of micro-grids. It would also allow a more comprehensive review of sustainability and
investigation of factors across the different solar grants that affect sustainability. Based on the time it takes to
install grids, transfer management to the community, and allow for challenges/problems to arise that require
SPV response, re-visiting sites two or three years after baseline data collection (as opposed to one-year post-
baseline, as proposed below for the first round of endline data collection) could also prove informative.
As noted in the Literature Review section above, the Window 3A grants innovatively utilize a community
engagement (SPV) approach or scheme to attempt to address challenges identified in the transition of
communities to micro-grids. The SPV approach seeks to transfer ownership to the communities in a way that
will promote sustainability and use of Solar PV facilities by strategically engaging communities from project
89
Note that these impact estimates will serve as quantitative answers to evaluation questions 1 and 2 on their own, and will subsequently feed into a model developed
by ICF International to produce estimated reductions in GHG emissions in response to evaluation question 3. These results will be presented alongside ICF’s original
estimations for context. In the event that primary data calls into question assumptions used as inputs in the ICF model, SI will note this and calculate the resulting
change in GHG emissions if these assumptions were altered.
SOCIALIMPACT.COM 63
inception, establishing (or re-vitalizing existing) community engagement groups, and training SPV members
in the areas of O&M, finance and administration, and sales.
In order to holistically answer evaluation question 4 related to the SPVs ability to generate community buy-in
and sustainability of critical infrastructure, baseline qualitative data collection will be useful to document what
the community is like pre-intervention. Areas or themes of interest would include the current status of existing
community engagement mechanisms/groups; levels of engagement in existing groups by various community
members (men, women, and youth); community past experiences with RE sources or donor/government
energy projects/initiatives; key community needs/challenges in terms of economic growth (or access to income
generating activities); and perceptions (optimism) of the Solar PV facility plan. Endline data collection, in
addition to collecting data along similar lines of inquiry as the baseline, will include themes related to the SPV
intervention in a given site. These could relate to relationship to the implementer/grantee/contractors;
experience with intervention roll-out; preparedness; and productive uses. Qualitative data at this stage is
expected to provide depth to quantitative findings related to key variables that were found to relate to outcomes
of interest. Endline comparative studies may prove useful, but the approach is expected to be substantially
informed by baseline data (both quantitative and qualitative).
Therefore, collecting pre-post qualitative data in both East Sumba in Berau will provide an opportunity to a)
document baseline community engagement conditions and investigate the current status of energy
consumption; and b) explore how each SPV approach/scheme ultimately impacted the achievement of
outcomes of interest (measured via indicators collected in quantitative data in both locations) at endline.
Additionally, a follow-up PE in unique comparative sites (from Window 2) could provide further depth
surrounding the role community engagement plays in the success or failure of uptake of micro-grids.
Specifically, qualitative data collected in East Sumba and Berau will: at baseline, provide baseline context for
indicators of interest related to evaluation questions 1 – 4; and, at endline, provide depth on evaluation
question 1-3 and answer evaluation question 4.
Data will be collected from both treatment and control sites in Sumba and in treatment sites only in Berau,
with the exception of grantee and SPV interviews and beneficiary FGDs which are only relevant in treatment
areas. SI will conduct semi-structured interviews with approximately 250 enterprises and 50 other
stakeholders along with approximately 12 FGDs in six selected villages in East Sumba and Berau at baseline,
further described below. This will allow for discussions with village/regency government officials, community
members (both beneficiaries and SPV members), enterprises, contractors and grant implementers/managers.
3.4.2 Timeframe of Exposure

Similar to section 3.3.2 above, SI suggests waiting a full year to collect endline data for the evaluation. SI
suggests collecting qualitative endline data shortly after collection of quantitative endline data (in November –
December of 2018) so that the latter can inform the former in terms of additional lines of inquiry to include.
This will allow sufficient time for productive uses and/or operational challenges for the micro-grids to develop
and be considered in the measurement of program outcomes. This will also allow for endline data collection
events to occur during approximately the same season, cancelling out any unobserved bias from seasonal
effects on energy consumption and income. Lastly, this will allow time for the project to transition O&M and
administrative responsibility for the Solar PV facilities to SPVs.
SOCIALIMPACT.COM 64
Additionally, we suggest a second follow-up PE (post-intervention data collection only) to be conducted after
three years of community exposure to the intervention/program, under the theory that more challenges to
sustainability and opportunities for investment in productive capital will have had the opportunity to manifest
by this time.
3.4.3 Study Sample

3.4.3.1 Sample unit
SI’s qualitative approach will include a variety of sample units with each type of stakeholder being interviewed
using a distinct protocol (see Annex 3). For all KII protocols, the sample unit will be individuals selected to
represent SPVs90, government entities91, grant implementers/managers92, and private firms contracted to
provide support to the facilities93. For the community beneficiary FGD guide, the sample unit will be
households. For the enterprise beneficiary KII guide, the sample unit will be firms and/or informal community
enterprises. These enterprises are those, in addition to the SPV, that may use energy for productive uses and
may include individuals earning income above a subsistence level by selling a good or service (or producing
a good or service for more than auto consumption).
3.4.3.2 Sample size
The SPV Leadership KII protocol will be issued to three to four individuals in each treatment unit sampled.
Grantee or contractor KII protocols will be issued to one individual each per treatment unit sampled. At times,
these individuals may repeat across treatment units (e.g. one O&M contractor is interviewed in reference to
several desas), but this information is not known at the time of writing. One regency official will be interviewed
per grant, and one to three grantee staff (including MCA-I Window 3 managers) will be interviewed per grant.
Kampung official and enterprise semi-structured interviews will be conducted in all treatment and control
areas.
There will be two FGDs of community beneficiaries per treatment unit (one for male beneficiaries and one for
female beneficiaries), and we will aspire to include eight to ten beneficiaries in each focus group. We expect
a total sample size of 50 - 78 key informants, approximately 250 enterprises and around 120 focus group
participants.
3.4.3.3 Sample frame
In the case of W3A Akuo Energy Solar/Micro-Hydro, Berau, the three villages selected for qualitative study
will include all three treatment units involved in the grant. In the case of W3A Anekatek Solar, East Sumba,
time and feasibility constraints preclude the qualitative team from visiting any more than three villages. The
sample frame for these villages will include all five of the treatment villages targeted by the grant.
The sample frame of stakeholders to serve as key informants in each village will be constructed by soliciting
contact lists from each of the grantees. The sample frame of beneficiaries to serve as focus group participants
will be constructed from beneficiary lists from each grantee. The sample frame of enterprises in each village
90
In W3A Akuo Energy Solar/Micro-Hydro, Berau, this could include the director, secretary, treasurer, O&M division head, sales and collection division head, or
the finance and administration division head. In W3A Anekatek Solar, East Sumba, this could include the head, Finance Manager, Secretary, or other members of
the community appointed to the cooperative.
91
Primarily including the Head of the Village (Kepala Desa) and the Head of the Sub-District (Camat).
92
Primarily including grantee staff (both local and HQ based), MCA-I Window 3 grant managers, and MCC RE Advisors.
93
Primarily including O&M and EFC contractor staff, as relevant for each treatment unit.
SOCIALIMPACT.COM 65
will be constructed by communicating with village officials in advance about how many and which types of
enterprises were present in the village.
3.4.3.4 Sampling strategy
At the village level, the three villages selected for additional qualitative study in East Sumba will be selected
purposively to draw the most interesting comparisons possible both within East Sumba and between East
Sumba and Berau. These will also contain as many treatment kampungs as possible. Barring any constraints
from selecting these, we currently propose to conduct qualitative data collection in Tawui, Lailunggi, and
Praiwitu.
Most key informants will be selected using a purposive sampling technique. In some cases, there may only
be one person or a few specific people that are performing the role whose perspective we require as a key
informant. We will review program documents and work with the grantee before data collection to identify
which role this is in village and regency government offices and in each contractor’s office. In the event that
an identified informant indicates a colleague who could provide additionally illuminating information, we will
attempt to contact this colleague to serve as an additional informant (snowball sampling).
Community beneficiary FGD participants will be selected using a convenience method on the basis of which
community members are available to participate in an FGD when the evaluation team passes through each
village. Since we propose qualitative field work to occur before quantitative field work, it will not be necessary
to avoid community members who may have been fatigued from participating in the quantitative survey. Given
that there are reportedly few enterprises in each village, especially few that are not basic kiosks or shops, we
will use a purposive sampling technique to ensure that the firms selected represent as diverse a cross
section as possible of enterprises in each treatment unit.
3.4.4 Primary Data Collection

3.4.4.1 Instruments
All KIIs and FGDs will be conducted according to pre-developed and tested protocols (see Annex 3). SI has
developed semi-structured protocols to direct each qualitative data collection activity. SI will utilize the same
baseline qualitative questionnaires in both East Sumba and Berau sites. As previously noted, endline
instruments are likely to differ from baseline instruments, depending on baseline findings. Baseline
instruments will document baseline community engagement conditions and investigate the current status of
energy consumption; and endline instruments will explore how each SPV approach/scheme ultimately
impacted the achievement of outcomes of interest (measured via indicators collected in quantitative data in
both locations). The team developed parallel protocols (see Annex 3) with the same or similar questions across
KIIs, FGDs, and mini-surveys (where appropriate/relevant) to enable greater data triangulation (additional
information on the analysis plan provided in section 3.4.5.2). To facilitate analysis, the enterprise and kampung
official protocols will be principally quantitative but will include qualitative elements.
Table 9 below provides a summary of instruments, respondents, and estimated respondent numbers per
treatment unit.
SOCIALIMPACT.COM 66
3.4.4.2 Rounds and timing

Qualitative baseline data collection will occur shortly before quantitative baseline data collection, in September
– October of 2017. While Berau data collection could occur later as commissioning will not occur until
December 2017, it will prove efficient to collect both East Sumba and Berau qualitative baseline data in the
same timeframe.
Endline data collection, as noted in section 3.4.2 above, is proposed to occur after endline quantitative data
collection in both East Sumba and Berau, scheduled for one year after baseline. Therefore, qualitative endline
data collection is proposed for November – December of 2018.
3.4.4.3 Respondents within the sample unit
Respondents within the sample unit (of six villages in East Sumba and Berau) are described in Table 9.
Respondents will represent sub-district and village levels, and range from implementing staff and grant
advisors to community members and local enterprises. If respondents are missing or absent at the scheduled
time, the team will follow the sampling procedures defined in section 3.4.3.4 and identify replacement
respondents (both at baseline and endline).
3.4.4.4 Staff
For the qualitative component, SI proposes a staffing structure of two evaluators, Sr. Analyst Jörg Peters and
Jr. Analyst Luciane Lenz, and two local research assistants (RAs), in addition to an interpreter at baseline.
The two evaluators have expertise in RE and solar power micro-grid programming, while the local RAs will
have expertise in Indonesia and the RE sector. Dr. Peters will serve as the Team Leader for the qualitative
field work and with the support of Mr. Duthie will supervise the evaluation team’s work from design and
fieldwork to data analysis and reporting. Dr. Peters will not conduct KIIs and FGDs (with the exception of those
in English), but rather will allow the RAs to collect primary qualitative data in Bahasa Indonesian. He will be
assisted by an interpreter during fieldwork to facilitate note-taking and on-going analysis as well as by staff
from the data collection partner who will conduct the enterprise interviews.
This team of four will allow for efficient coverage of the six sites over a period of 17 days (September 18 to
October 5). While SI expects that some interviews may take place in English, the use of local team members
that are culturally and linguistically fluent in Bahasa Indonesia will allow interviews to be conducted in the
national language when necessary. See section 4.5 for more details on staffing.
3.4.4.5 Data processing
Interview and discussion notes from qualitative data collection activities will be created during field work with
daily review by the team to ensure clarity. The team will also record all interviews and discussions to lend to
eventual transcription and translation. Transcription and translation may be done through external consultants
and/or members of the evaluation team. Complete transcripts will be a) anonymized for the protection of
respondents and b) uploaded into qualitative data analysis software for analysis and report writing. Qualitative
data will be handled solely by the evaluation team and SI HQ management team members that provide support
during baseline and endline data collection activities.
SOCIALIMPACT.COM 67
3.4.4.6 Data quality

The data processing methods described in section 3.4.4.5 lend to high quality data at the baseline. The team
leader will have the ultimate responsibility to check interview notes for completeness and accuracy during
team de-briefs and review sessions during fieldwork. A local evaluator will be tasked to review transcripts and
translations to ensure accuracy of wording and phraseology used. Additionally, if the team splits and the team
leader is not present during multiple KIIs and FGDs, the team leader will randomly spot check interview notes
and transcripts related to those KIIs/FGDs to ensure the facilitator/interviewer followed protocols and adhered
to best practice for conducting qualitative data collection.
Qualitative Questionnaires Summary Table
No. Type Name Respondents Estimated Number of Respondents
Berau: Director, Secretary, Treasurer,
O&M division head, Sales and
collection division head, or Finance
and administration division head
1 KII SPV Leadership 3-5 individuals per treatment unit
East Sumba: Head, Finance Manager,
Secretary, or other members of the
community appointed to the
cooperative
1 individual per treatment/control
2 KII Village Official Head of Village (Kepala Desa)
unit
1 individual per sub-district
3 KII Regency Official Head of Sub-District (Camat)
including a treatment unit
1-3 individuals per grantee
Project Grantee staff; MCA-I Window 3A (including both local and HQ-based,
4 KII
Grantee/Manager Manager(s); MCC RE Advisors if possible); 2-3 individuals from
MCA-I and MCC
1 individual per treatment unit (may
5 KII EPC Contractor Contractor staff
be duplicates across units)
1 individual per treatment unit (may
6 KII O&M Contractor Contractor staff
be duplicates across units)
Household members (not selected for 2 groups (1 M, 1 F including 8-10
7 FGD Community
quantitative survey) individuals each) per treatment unit
Firms and/or informal community
8 KII Enterprise Up to 8 per community
enterprises
Table 11: Summary of instruments, respondents, and estimated respondent numbers per treatment unit
(Qualitative Questionnaire)
SOCIALIMPACT.COM 68
3.4.5 Analysis Plan

3.4.5.1 Coding
Interview transcripts will be coded using electronic software (Dedoose or similar software) to construct
response categories and identify patterns in data, as relevant. Coding qualitative data through use of electronic
software will allow SI to analyze transcripts with speed and efficiency, easily cataloging and documenting
emergent themes from among respondents. Prior to fieldwork, the team will develop a preliminary coding
scheme based on finalized protocols. During fieldwork, the team leader will adjust the coding scheme as new
themes or areas of interest arise as relate to each evaluation question. The coding scheme will be finalized
post-fieldwork and will include codes across identified themes and evaluation questions.
3.4.5.2 Analysis method/framework
At baseline, the team will utilize interview notes and codes (resulting from aggregated and coded transcripts
detailed in section 3.4.5.1) to detail key indicators related to evaluation questions 1 – 3 (including energy
consumption, energy spending, and household economic and time use) and community engagement levels
(through description of themes related to evaluation question 4). Triangulation will enable the team to cross-
verify and cross-validate the findings that emerge to identify correlations between findings. In particular, the
team developed parallel protocols (see Annex 3) with the same or similar questions across KIIs, FGDs, and
mini-surveys (where appropriate/relevant). This will enable greater data triangulation because each method
addresses sub-sets of the same evaluation questions, and findings will be validated or refuted by the other
techniques. Methodological triangulation will also enable the team to strengthen the potential linkages and
accuracy of its data if the results obtained through one method are less conclusive than another method.
The team plans to employ several data analysis methods to identify key findings from the collected data. The
type of analyses will depend on the specific data being assessed, but will most likely include the following.
The first two listed are likely to be employed at baseline, while all listed are likely to be employed at endline.
1. Comparative Analysis – The team will compare baseline context and endline results across grants,
treatment sites (villages), and stakeholder groups to assess convergence or divergence in perspectives.
2. Trend Analysis – Trend analysis will enable the team to examine different indicators or interest over time
(from baseline to endline) to identify patterns of convergence or divergence of outputs and outcomes
related to the evaluation questions.
3. Contribution Analysis – Contribution Analysis is an approach for assessing and inferring causality in
program evaluations. It provides evidence for drawing conclusions that the grant’s activities have
contributed to positive, documented results identified by the team. Such analysis will be most useful in
confirming the relevance of the program’s theory of change at endline.
4. Gender Analysis – The team will consider at baseline the current status of women and men in the
treatment sites and at endline, consider whether activities and resulting outcomes specifically benefit (or
do not benefit) women or men. All data collected through KIIs, FGDs, and mini-surveys will be
disaggregated by gender and analyzed for effects on women and men beneficiaries of the project.
SOCIALIMPACT.COM 69
3.5 Challenges & Limitations

3.5.1 Limitations of Interpretations of the Results
While the approaches outlined above should generate reliable and representative information for the three
grants studied, we are limited in our ability to generalize these results to other RE grants, particularly those
using other RE technologies or SPV mechanisms. The inclusion of three grants allows us some degree of
analytical purchase in investigating the generalizability of the results across contexts, but analysis of additional
grants, including of other RE technologies, would support further analysis and confidence of the role of context
and intervention type, thus supporting discussion of generalizability and interpretation of results.
3.5.2 Risks to the Study Design

This section describes a variety of potential risks with the evaluation approach as well as our proposed
mitigation strategies. Specifically, we consider the most important risks as summarized in Table 10, and
discuss them in greater detail below.
Violation of CIA assumption. A major risk is related to the assumption that conditional on control for baseline
observable characteristics through the matching process, the expected outcomes of treatment and
comparison groups are similar. Given that the selection of treatment kampungs was neither random nor did it
follow a systematically documented approach, it would be impossible to exhaustively model the treatment
process or control for all relevant factors. However, based on discussions with a variety of stakeholders, we
believe that the potential comparison areas are quite similar to treatment areas along demographic, social,
economic, and energy use characteristics. Moreover, we will collect extensive data at baseline on which to
match, and the DiD approach eliminates this concern for any time-invariant unobserved characteristics.
Type of risk Description Mitigation strategy
-Conduct balance tests at

baseline
-Test for systematic differences
Imperfect control for factors that influence following baseline
Violation of CIA assumption
program outcomes -Use CEM based on variables
from an extensive baseline
-Use DiD approach to control for
time-invariant factors
-Use conservative assumptions in

power calculations
Power may be too low to identify expected -Oversample, particularly in
Lack of statistical power
effect sizes comparison areas, to limit effects
pruning and minimize the effect of
clustering
SOCIALIMPACT.COM 70
Type of risk Description Mitigation strategy
-Review and interviews with

regional stakeholders to identify
plans for electrification
Outcome variables may be affected by time-
varying factors that are not related to -Relatively short measurement
Confounding
treatment (e.g. electrification in comparison time horizon makes installation of
kampungs) new systems during IE unlikely
-Oversampling of comparison
kampungs
1) Power calculations allow for

15percent loss to follow-up
1) Attrition in sample
Other important 2) Spillovers unlikely, but
considerations (not 2) Spillovers/general equilibrium effects oversampling of kampungs to
discussed in detail in permit dropping as needed
3) Incomplete program implementation prior
following text)
to compact close 3) Track monitoring data and
maintain flexibility in data
collection timing
Table 12: Categorization of threats to identification of impacts, and mitigation strategy

Lack of statistical power. Another distinct concern has to do with the potential lack of statistical power to reliably
measure impacts of the MCC investments. This is of particular concern in this case due to the very low number
of treatment kampungs, which makes power very sensitive to ICC. Moreover, any specific events that would
require removal of a treatment kampung from analysis (for example, lack of completion of installation in a
kampung) would have a large effect on power. A third concern relates to the pruning process during matching.
If many households (or even perhaps whole kampungs) are pruned due to lack of a suitable match, power will
further decrease. To mitigate this risk, we have been relatively conservative in estimating parameters for power
calculations. We also propose oversampling at baseline to allow for pruning, as well as sampling additional
comparison kampungs to gain power (due to the diminishing returns of increasing the sample of households
in each treatment kampung).
Confounding. Another potential source of bias in our estimates of impact could emerge from confounding by
time-varying factors affecting treated and untreated comparison units differentially. Perhaps the most
important confounder would be electrification of comparison kampungs, either by PLN or another project. To
mitigate this risk, we will do a systematic review of other stakeholders involved in electrification in E Sumba
and through interviews identify (and exclude from our comparison kampung population) any areas planned
for electrification. Through this review and because of the relatively short duration of the IE, in comparison to
the time typically taken for permitting and installation of electrification, we do not expect this to be a significant
threat in practice. Moreover, we intend to oversample comparison kampungs, which would permit dropping a
very small number of comparison kampungs if they are electrified.
SOCIALIMPACT.COM 71
4 ADMINISTRATIVE
4.1 Summary of IRB Requirements and Clearances
In conjunction with MCC’s commitment to respect and follow the Common Federal Policy for the Protection of
Human Subjects where feasible, SI will pass the approved evaluation design through IRB review prior to data
collection. SI has an in-house Institutional Review Board (IRB) that can review applications for human subjects
research. SI’s internal IRB has established protocols for gathering informed consent, protecting anonymity
and identifying information, and ensuring ethical data collection—including from children and other vulnerable
populations. It is registered with the U.S. Department of Health & Human Service’s Office for Human Research
Protections.
In addition, SI closely monitors and adheres to human subject research regulations in its countries of operation
to ensure all evaluations are registered and fully compliant with local law. In this case, in accordance with
Government Decree No: 41/2006,94 SI will ensure that, if required, research activities under this evaluation
and staff supporting these activities apply for and receive the appropriate permits from the GoI’s Ministry of
Research, Technology, and Higher Education (Ristekdikti).
4.2 Data Protection

SI’s process for respecting privacy of respondents during data collection, transfer, storage, analysis, disposal
and dissemination is governed by SI’s data security guidelines, which are aligned with MCC’s microdata
guidelines.
4.3 Preparing Data for Public Use

SI will adhere to MCC’s open data policy with regard to preparing data for publication. All primary quantitative
data collected by the evaluation will be prepared and submitted to MCC according to the most updated version
of the Disclosure Review Board (DRB) guidelines available at the time of data collection. On an instrument-
by-instrument basis, SI and MCC will weigh the utility of publishing primary qualitative data (even in a
restricted-access database) against (i) the risks of respondent re-identification and (ii) the risks of adverse
effects on data quality from disclosure. In the event that the utility of this data outweighs the risk of re-
identification, and that respondents can be adequately informed via a consent script as to the data’s intended
use without jeopardizing their willingness to be forthcoming with interviewers, SI will submit this primary
qualitative data to MCC as part of the DRB process.
4.4 Dissemination Plan

Since reporting and dissemination must be completed prior to Compact closeout, SI will present the baseline
evaluation findings in draft form after receiving feedback from MCC and local stakeholders on the baseline
draft evaluation report. One presentation will be given in Washington to MCC, while another will be given to
94
The text of which can be found as Annex 1 to this document:
http://www.international.itb.ac.id/web/wp-content/uploads/2010/05/Foreign_Research_Permit_Procedure_2015.pdf
SOCIALIMPACT.COM 72
local stakeholders in Jakarta including MCA-I, implementing grantees, and relevant GoI stakeholders. We
recommend a similar set of presentations in both Jakarta and Washington for follow-up reports given the
importance of this sector to the GoI and other stakeholders.
4.5 Evaluation Team Roles and Responsibilities

The evaluation team will be comprised of a field evaluation team and support staff at SI headquarters. In some
cases, evaluation team members will have a role both as field evaluators and management support staff. The
evaluation team includes all personnel described in Table 11.
Table 13: Evaluation Team
Personnel Role Technical/Support Responsibility
Principal Investigator, responsible for technical
oversight and senior-level evaluation expertise. Will
Mike Program
Both lead evaluation design, data collection, reporting, and
Duthie Manager
dissemination. Also responsible for oversight of overall
contract performance for SI-HQ.
Expert in the evaluation of RE programming,
responsible for advising evaluation team on sector-
Sr. Analyst,
appropriate evaluation design and instruments. Will
Jörg Peters Renewable Technical
contribute to the oversight of field data collection as
Energy
instructed by Principal Investigator, as well as data
analysis and reporting.
Sr. Analyst,
Expert in Solar PV programming, specifically. Will serve
Hussain Renewable
Technical in an advisory role to the team and review evaluation
Samad Energy
methodology and instruments prior to finalization.
(advisory)
Subject matter expert in solar photo-voltaic technology
and programming, will advise on quantitative and
Luciane qualitative instruments and plan for field data collection.
Jr. Analyst Technical
Lenz Will contribute to the oversight of field data collection as
instructed by Principal Investigator as well as data
analysis and reporting.
Amanda Mid-level evaluator, responsible for liaising with local
Jr. Analyst Both
Stek stakeholders and overseeing enumerator training.
Local research assistant responsible for assisting in the
Research arrangement and oversight of both quantitative and
TBD Assistant(s) Technical qualitative data collection. SI anticipates that we will
(local) require one quantitative research assistant and two
qualitative research assistants.
SOCIALIMPACT.COM 73
Personnel Role Technical/Support Responsibility

Will serve as the evaluation manager for SI-HQ support
staff, and thus manage finances, personnel, scheduling,
and contractual compliance for the evaluation. Will also
Miguel Research serve as a research assistant and contribute to
Both
Albornoz Assistant (HQ) evaluation design, data collection, analysis, and
reporting as instructed by the Principal Investigator.
Primarily responsible for managing the data collection
subcontractor and overseeing data quality assurance.
Project assistant responsible for administration and
Julia Administrative
Support project backstopping. Will contribute to data quality
Higgins Assistant
assurance as instructed by the Principal Investigator.
4.6 Evaluation Timeline and Reporting Schedule

Table 12 displays the schedule for the evaluation’s baseline. This schedule would be repeated after one year
for endline data collection.
Table 14: Evaluation Timeline and Reporting Schedule
Name of Data Cleaning & Analysis First Draft Report Final Draft Report
Data Collection
Round Expected Expected
October/2017 – November/2017 -
Baseline January/2018 February/2018
November/2017 December/2017
Follow- October/2018 – December/2018 -

February/2018 March/2019
up 1 December/2018 January/2019
Follow- October/2020 – December/2020 -

February/2020 March/2021
up 2 December/2020 January/2021
Task Deadline
Draft Evaluation Design Report (EDR) Submission July 21, 2017
Institutional Review Board and Ristekdikti Materials Submission July 28, 2017
Feedback on EDR Received Aug. 11, 2017
Data Collection Subcontractor Selected Aug. 25, 2017
Final EDR Submission Aug. 21, 2017
Baseline Qualitative Data Collection Sep. 18 – Oct. 5, 2017
SOCIALIMPACT.COM 74
Task Deadline
Baseline Quantitative Piloting/Enumerator Training Oct. 16 – 20, 2017
Baseline Quantitative Data Collection Oct. 23 – Nov. 30, 2017
Summary of Quality Control Checks Submission Dec. 15, 2017
Draft Baseline Evaluation Report Submission Jan. 12, 2018
Feedback on Baseline Evaluation Report Received Jan. 31, 2018
Draft Baseline Findings Presentation Delivered - Washington Week of Jan. 15, 2018
Draft Baseline Findings Presentation Delivered - Jakarta Week of Jan. 22, 2018
Final Baseline Evaluation Report Submission Feb. 23, 2018
DRB Submission Mar. 2, 2018
5 REFERENCES
Aguirre, J. 2014. “Impact of Rural Electrification on Education: A Case Study from Peru.” Mimeo, Research Center,
Universidad del Pacifico (Peru) and Department of Economics, Universidad de San Andrés (Argentina).
Akella, A.K. 2009. Social, economical and environmental impacts of renewable energy systems. Renewable Energy
34: 390–396
Apergis, N. 2010. Renewable energy consumption and economic growth: Evidence from a panel of OECD
countries. Energy Policy 38(1): 656-660.
Arraiz, Irani, and Carla Calero. From Candles to Light: The Impact of Rural Electrification. Working paper no. IDB-
WP-599. Inter-American Development Bank, May 2015. Web.
Asian Development Bank (2014). Mid-Term Report (Final): Least-Cost Electrification Plan for the Iconic Island; ADB
TA8287
Asian Development Bank (2016). Achieving Universal Electricity Access in Indonesia ADB 2016; TA8287
Bailis R., Drigo R., Ghilardi A. and O. Masera (2015). The carbon footprint of traditional woodfuels. National Climate
Change 5:266–72.
Banerjee, S. G., A. Singh, and Samad, H. (2011). Power and people : the benefits of renewable energy in Nepal.
Washington D.C., World Bank.
Barron, M., and M. Torero. 2014. “Short Term Effects on Household Electrification: Experimental Evidence from
Northern El Salvador.” Mimeo.
Bensch, G., Grimm, M., Huppertz, M., Langbein, J., & Peters, J. (2016). Are promotion programs needed to
establish off-grid solar energy markets? Evidence from rural Burkina Faso (No. 653). Ruhr Economic Papers.
SOCIALIMPACT.COM 75
Bensch, G., J. Peters und M. Sievert (2017), The lighting transition in rural Africa — From kerosene to battery-
powered LED and the emerging disposal problem. Energy for Sustainable Development 39 : 13-20.
Bernard, T. (2012). Impact Analysis of Rural Electrification Projects in Sub-Saharan Africa. World Bank Research
Observer, 27(1), 33–51.
Birol, F. (2010). World energy outlook 2010. International Energy Agency, 1(3).
Chaplin, D., Mamun, A., Protik, A., Schurrer, J., Vohra, D., Bos, K., ... & Cook, T. Grid Electricity Expansion in
Tanzania by MCC: Findings from a Rigorous Impact Evaluation, Final Report (No.
144768f69008442e96369195ed29da85). Mathematica Policy Research.
D'Agostino, A.L., Lund, P.D. and Urpelainen, J., 2016. The business of distributed solar power: a comparative
case study of centralized charging stations and solar microgrids. Wiley Interdisciplinary Reviews: Energy and
Environment.
Feron, S.; Heinrichs, H.; Cordero, R.R. Sustainability of rural electrification programs based on off-grid photovoltaic
(PV) systems in Chile. Energy Sustain. Soc. 2016, 6, 32
https://energsustainsoc.springeropen.com/articles/10.1186/s13705-016-0098-4
Gibson, J., & Olivia, S. (2010). The effect of infrastructure access and quality on non-farm enterprises in rural
Indonesia. World Development, 38(5), 717-726.
Glemarec, Y. Financing off-grid sustainable energy access for the poor. Energy Policy 2012, 47, 87–93.
Global Environment Facility (2009). “Investing in Renewable Energy: the GEF experience”. 2009. Global
Environment Facility.
Gonzalez, M., and M. Rossi. 2006. “The Impact of Electricity Sector Privatization on Public Health.” IDB Working
Paper No. 219, Inter-American Development Bank, Washington, DC.
Grimm, M., & Peters, J. (2016). Solar off-grid markets in Africa. Recent dynamics and the role of branded
products. Field Actions Science Reports. The journal of field actions, (Special Issue 15), 160-163.
Grimm, M., Hartwig, R. & Lay, J. (2013). Electricity Access and the Performance of Micro and Small Enterprises:
Evidence from West Africa. European Journal of Development Research, 25, 815-829.
Grimm, M., Sparrow, R., & Tasciotti, L. (2015). Does electrification spur the fertility transition? Evidence from
Indonesia. Demography, forthcoming.
Grogan, L. & Sadanand, A. (2013). Rural Electrification and Employment in Poor Countries: Evidence from
Nicaragua. World Development, 43(0), 252–265.
Hasan, A. 2013. Indonesian province explores ‘green growth’ amidst economic expansion. CIFOR. <blog.cifor.org>
http://en.rwi-essen.de/publikationen/rwi-materialien/353/
http://prokum.esdm.go.id/pp/2014/PP%20Nomor%2079%202014.pdf pg. 8
http://www.unredd.net/documents/un-redd-partner-countries-181/asia-the-pacific-333/a-p-partner-
countries/indonesia-187/15077-final-evaluation-of-the-un-redd-programme-in-indonesia.html
http://www.unredd.net/documents/un-redd-partner-countries-181/asia-the-pacific-333/a-p-partner-
countries/indonesia-187/15078-indonesia-redd-national-strategy.html
SOCIALIMPACT.COM 76
http://www.unredd.net/regions-and-countries/asia-pacific/indonesia.html
https://openknowledge.worldbank.org/handle/10986/16068
https://www.adb.org/documents/indonesia-energy-sector-assessment-strategy-and-road-map
https://www.adb.org/publications/adb-and-climate-investment-funds-country-fact-sheets
https://www.adb.org/publications/clean-energy-asia-case-studies-adb-investments-low-carbon-growth
https://www.adb.org/publications/climate-change-vulnerability-poverty-indonesia
https://www.adb.org/publications/cobenefits-and-trade-offs-green-and-clean-energy
https://www.adb.org/publications/cobenefits-and-trade-offs-green-and-clean-energy
https://www.adb.org/publications/diagnosing-indonesian-economy-toward-inclusive-and-green-growth
https://www.adb.org/publications/ground-case-studies-community-empowerment
https://www.adb.org/publications/investing-renewable-energy-generation-transmission-eastern-Indonesia
https://www.adb.org/publications/summary-indonesias-energy-sector-assessment
https://www.adb.org/publications/summary-indonesias-poverty-analysis
https://www.bps.go.id/website/pdf_publikasi/Neraca-Energi-Indonesia--2011-2015--.pdf
https://www.bps.go.id/website/pdf_publikasi/Neraca-Energi-Indonesia--2011-2015--.pdf
https://www.cia.gov/library/publications/the-world-factbook/geos/id.html
https://www.illegal-logging.info/sites/files/chlogging/uploads/IndonesiasiaranpdfGreenPaperFinal.pdf
https://www.researchgate.net/profile/Muhammad_Shahbaz29/publication/257548547_Economic_growth_energy_
consumption_financial_development_international_trade_and_CO2_emissions_in_Indonesia/links/56dd1cb308ae
SOCIALIMPACT.COM 77
e1aa5f88cc5d/Economic-growth-energy-consumption-financial-development-international-trade-and-CO2-
emissions-in-Indonesia.pdf
Independent Evaluation Group (IEG). 2008. The Welfare Impacts of Rural Electrification – An IEG Impact
Evaluation. Independent Evaluation Group, World Bank.
Indonesia National Council on Climate Change. Fact Sheet Norway-Indonesia Partnership

REDD+.http://www.norway.or.id/PageFiles/404362/FactSheetIndonesiaGHGEmissionMay252010.pdf
Khandker, S. R., Barnes, D. F., & Samad, H. A. (2013). Welfare Impacts of Rural Electrification: A Panel Data
Analysis from Vietnam. Economic Development and Cultural Change, 61(3), 659–92.
Khandker, S. R., Barnes, D.F. & Samad, H.A. (2012). The Welfare Impacts of Rural Electrification in Bangladesh.
The Energy Journal, 33(1), 187.
Khandker, S.R., Koolwal, G.B. (2010) How Infrastructure and Financial Institutions Affect Rural Income and Poverty:
Evidence from Bangladesh. Journal of Development Studies, Vol. 46 (6), p.1109–1137
Khandker, Shahidur R., Douglas F. Barnes, and Hussain A. Samad. Welfare Impacts of Rural Electrification: A
Case Study from Bangladesh. Working paper no. 4859. The World Bank, Mar. 2009. Web.
Kirubi, C., Jacobson, A., Kammen, D. M., & Mills, A. (2009). Community-based electric micro-grids can contribute
to rural development: evidence from Kenya. World Development, 37(7), 1208-1221.
Lenz, L., A. Munyehirwe, J. Peters und M. Sievert. 2017. Does Large Scale Infrastructure Investment Alleviate
Poverty? Impacts of Rwanda's Electricity Access Roll-Out Program. World Development 89 (17): 88-110.
Luttrell, Cecilia, Ida Aju Pradnja Resosudarmo, Efrian Muharrom, Maria Brockhaus, and Frances Seymour. “The
Political Context of REDD+ in Indonesia: Constituencies for Change.” Environmental Science & Policy 35 (2014):
67–75.
McCarthy, John F, and Kathryn May Robinson. Land and Development in Indonesia: Searching for the People’s
Sovereignty. Singapore: ISEAS – Yusof Ishak Institute, 2016.
National Council on Climate Change. 2010. Setting a course for Indonesia’s green growth.
Neelsen, Sven and Jörg Peters. 2011. “Electricity usage in micro-enterprises — Evidence from Lake Victoria,
Uganda.” Energy for Sustainable Development, 15(1): 21–31.
Oakley, D., P. Harris, et al. (2007). Modern energy ‐ Impact on micro‐enterprise. A report produced by the
Department for International Development. R8145. DFID. AEA Energy and Environment. March 2007.
Obeng, G. Y. and H. D. Evers (2010). Impacts of public solar PV electrification on rural microenterprises: The
case of Ghana. Energy for Sustainable Development 14(3): 223‐231.
Obeng, G.Y.; Evers, H.D. Impacts of public solar PV electrification on rural micro-enterprises: The case of Ghana.
Energy Sustain. Dev. 2010, 14, 223–231
Pacheco, P. 2013. Biofuels and forests: Revisiting the debate. CIFOR. <blog.cifor.org>
Parikh, P., Fu, K., Parikh, H., McRobie, A., & George, G. (2015). Infrastructure Provision, Gender, and Poverty in
Indian Slums. World Development, 66, 468-486.
SOCIALIMPACT.COM 78
Peters, J., & Sievert, M. (2015). The provision of electricity to rural communities through Micro-Hydro Power in
rural Indonesia: Micro Hydro Power pilot programme within the national programme for community development
(PNPM) supported by the Netherlands through energising development (No. 88). RWI Materialien
Peters, J., & Vance, C. (2011). Rural Electrification and Fertility – Evidence from Côte d’Ivoire. Journal of
Development Studies, 47 (5), 753-766.
Peters, J., M. Sievert and C. Vance (2013), Firm Performance and Electricity Usage in Small Manufacturing and
Service Firms in Ghana. In: Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH (ed.),
Productive Use of Energy – PRODUSE ‐ Measuring Impacts of Electrification on Small and Micro‐Enterprises in
Sub‐Saharan Africa. Eschborn: Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH. 75‐94
Peters, Jörg, Colin Vance, and Marek Harsdorff. 2011. “Grid Extension in Rural Benin: Micro-Manufacturers and
the Electrification Trap.” World Development, 39(5): 773–83.
Ramanathan, V., & Carmichael, G. (2008). Global and regional climate changes due to black carbon. Nature
geoscience, 1(4), 221.
Rencana Usaha Penyediaan Tenaga Listrik (or RUPTL) 2015-2024
Rud, J.P. (2012). Electricity provision and industrial development: Evidence from India. Journal of Development
Economics, 97(2), 352–67.
Shindell, D., Kuylenstierna, J. C., Vignati, E., van Dingenen, R., Amann, M., Klimont, Z., ... & Schwartz, J. (2012).
Simultaneously mitigating near-term climate change and improving human health and food security. Science,
335(6065), 183-189.
Trimble, Christopher Philip; Kojima, Masami; Perez Arroyo, Ines; Mohammadzadeh, Farah. 2016. “Financial
viability of electricity sectors in Sub-Saharan Africa : quasi-fiscal deficits and hidden costs”. World Bank Policy
Research Working Paper.
UNEP. 2012. Financing renewable energy in developing countries: Drivers and barriers for private finance in sub-
Saharan Africa.
United Nations Environment Programme (UNEP). 2008. Green Jobs: Towards Decent Work in a Sustainable, Low-
Carbon World.
UN-REDD. 2011c. UN-REDD Programme in Indonesia Semi-Annual Report.
van de Walle, D., Ravallion, M., Mendiratta, V., & and Koolwal, G. (2015). Long-term impacts of house
SOCIALIMPACT.COM 79
6 ANNEXES
6.1 Annex 1: Stakeholder Comments and Evaluator Responses
6.1.1 MCA-I Comments and Evaluator Responses
Reviewer
Page Number
Name/ Comment Evaluator Responses
(in draft EDR)
Institution
Correct abbreviation for PSGIP is Project Social and Gender Integration Plan (please do
8 Corrected.
Dwi Faiz/MCA‐I make necessary correction accordingly throughout the document
positive impact of electrity should also include not only (gender) attitude but gender These outcomes are omitted
equality outcomes where women who are mostly overburden with household chores and from this section because
mobility due to lack of electricity could save time, use it for capacity building and leisure, they are not included in
increase mobility and safety. See Deloitte (2014); ENERGIA (2007); Dinkelman (2011); project M&E documents, but
Suggestion to revise or add the (gender) attitute to positive outcome for gender equality the evaluation does aim to
24
above, or to add not only education for children, but also opportunity for women to capture and report on these
reduce household chores, improved mobility and safety as appropriate in the text outcomes, as appropriate
explaining about positive impact of electricity. Evidence on rural electrification positive (see instruments). We added
impact on women's employment could also be seen in the literature (example from South a footnote to make this
Dwi Faiz/MCA‐I Africa, in addition to India) such as in: Dinkelman, Taryn (2011). more clear.
Added the words "… at the
for primary question 2: additional lines of inquiry n may also be needed to differentiate community or household
32
between productive use of electricity at community/village of at household level" to the question for
Dwi Faiz/MCA‐I clarity
For primary question 4: additional lines of inquiry should also cover: how does the SPV
We will include these
governance ensure transparent and participatory monitoring of community? Has the SPV
probing questions in our
governance/business models taken into account gender equality and social inclusion?
instruments, but feel it is
32 How has the implementer prepared the transition/hand over of share to village
superfluous to include them
enterprise? how has the SPV business model/business plan fit into existing condition of
in the evaluation question
the village in terms of willingness/affordability to pay for electricity? (especially to ensure
itself.
Dwi Faiz/MCA‐I that less affluent household are able to pay for electricity)?
SOCIALIMPACT.COM 80
Reviewer
Page Number
(in draft EDR)
Institution
Household information should also covers the following: number of family members,
primary occupation/livelihoods and composition of gender, to differentiate electricity use
for productive activities. In W3A‐33, most villagers are farmers, where electricity
consumption may reach its peak during the night for leisure and others, the most affluent
households may be family of traders with more opportunity for off‐farm economic activity
that needs electricity. in W3A‐59, agriculture post harvest processing is presumably
contributes to productive use of electrity. Another point is any information about non‐
farm asset/livelihoods should also be included. In W3A‐33 location, there are households
37
opening stalls, or households with handicraft making as source of income in addition to
agriculture activities. In household activities, household chores should also be identified
clearly, to ensure that reproductive activities such as water‐fetching, child
rearing/nurturing be included. On key health issues, outcomes should include other than
cleaner energy use, but longer availability of electricity for key health services, especially
for child and maternal health. Longer availability of electricity can contributes to
availability of cold chain for vaccine, better assistance and support for pregnancy with This information is included
Dwi Faiz/MCA‐I complication (maternal health), better access to clean water and good sanitation. in the household survey.
Security should also include mobility in the community, W3A‐59 includes street lighting, The outcomes are captured
38
Dwi Faiz/MCA‐I security/safety in mobility outcome should also be diseggregated by sex. in the household survey.
For Beneficiary FGDs, there
will be separate male and
female groups to ensure
inclusion of female
perspective. For enterprise
surveys, all enterprises
Just a reminder to also put a gender balance in selecting member of sub‐village for semi‐ (included female‐operated)
38 structured protocols, there may be some social dynamics unable to be captured by one will be included. For HH
gender only. survey, we will interview
whoever is most responsible
in household for decisions
related to electricity usage.
All other instruments will
select people per their roles
Dwi Faiz/MCA‐I in their respective
SOCIALIMPACT.COM 81
Reviewer
Page Number
(in draft EDR)
Institution
organizations, whatever
gender they may be.
The plan is not to interview
household head, but to
Limiting the respondents to household head may have risks of bias view on
interview person in
energy/electricity consumption, not to mention gender bias, as household head is
household responsible for
presumable male. In most households observed in Berau, many questions raised during
decisions related to
consultation come from women who are the household "manager" who oversee
electricity, whether they are
42 household spending, including on electricty. Having both men and women interviewed
male or female. SI will take
has merit in getting better information about electricity consumption, expenditure,
steps during piloting to
clearer information on family members activities and time use. If this cannot be done,
ensure that we are not
steps to avoid gender bias in responses should be taken such as triangulation with other
prevented from speaking
methods. Pilot test should also sensitive to this issue.
with female household
Dwi Faiz/MCA‐I managers
Due to concerns around the
length of the survey, we
Questions for household health status should also incorporate maternal health, filter have not included questions
87
question on female household members pregnancy status and health will be needed related to maternal health
as it is not a primary
Dwi Faiz/MCA‐I intended outcome.
Due to concerns around the
length of the survey, we
In addition to health insurance, type of health service providers accessed should also be have not included questions
captured, such as puskesmas, pustu, house of bidan (midwives), traditional healers, on this in the household
87
halfway houses for labour, in case of maternal health complication (this will give survey. However, we do
information on potential electricity needed for basic service at village level) include questions about
health care options in the
Dwi Faiz/MCA‐I village survey.
SOCIALIMPACT.COM 82
Reviewer
Page Number
(in draft EDR)
Institution
Questions on perception on gender equality among community members is worth to ask,
since as required in PSGIP development, gender equality has to be integrated in key
93
critical stages of SPV development, such as engagament and participation in SPV Included in SPV Leadership
Dwi Faiz/MCA‐I establishment as well as information sharing on SPV. instrument
Population figures
disaggregated by
95 all demographic data should be disaggregated by sex , where appropriate
male/female in regency and
Dwi Faiz/MCA‐I village official protocols.
Regarding educational
attainment, the
questionnaire includes
questions on the highest
level of education, the years
spent on education, and the
time spent studying inside
and outside of school of
children (disagregared by
for the open ended question on living condition, the protocol does not explanatory notes gender). We elicit
on what is considered as 'improved living condition' , while general poverty indicators information on availability of
such as housing condition is understood, the response could be varied. Hence it will be basic services via regency
good if there is an explanatory notes to explain the living condition. It will be good if and community official
100
improvement of living condition could also identify availability of basic social services, instruments. We added
such as availability of (functional) health and education services and basic human maternal health questions
development improvement such as education attainment, health status only to the FGD guides and
(numbers/incidence of maternal mortality and children under two, malnutrition, etc). Village Official
questionnaire, as we do not
expect electricity access to
affect maternal health
variables with a detectable
size, as the timeframe is only
one‐year, the channel of this
impact is only indirect, and
as sample sizes of pregnant
Dwi Faiz/MCA‐I women within this year will
SOCIALIMPACT.COM 83
Reviewer
Page Number
(in draft EDR)
Institution
be too small to detect
effects.
A few questions on gender
awareness were added to
the questionnaire. They may
102 Question no. 22L: please refer to my comment in p. 93
be adopted to the training
context after receiving
Dwi Faiz/MCA‐I training materials.
This is included in the
103 There is no reference on basic social service facilities (Schools, Puskesmas, etc) at Regency regency and village official
Dwi Faiz/MCA‐I protocols
At Regency‐level, information on priority agriculture commodities (produk pertanian
105 unggulan) and value‐addition agro commodities could be available , it should also be on
Dwi Faiz/MCA‐I the list of question. This has been added.
Dwi Faiz/MCA‐I 106 Similar comment on living condition This has been added.
Dwi Faiz/MCA‐I 107 Similar comment on gender equality for question no. 16 This has been added.
Arief/MCA
iii ESMP: Environmental and Social Management Plan Corrected.
Indonesia/M&E
Arief/MCA
iv M&E: Monitoring and Evaluation Corrected.
Indonesia/M&E
Arief/MCA
iv NRE: New and Renewable Energy Corrected.
Indonesia/M&E
SOCIALIMPACT.COM 84
Reviewer
Page Number
(in draft EDR)
Institution
Arief/MCA
iv PE: Performance Evaluation Corrected.
Indonesia/M&E
Arief/MCA
v PMAP 1: Participatory Mapping and Planning 1 Corrected.
Indonesia/M&E
Arief/MCA
v PMAP 2: Participatory Mapping and Planning 2 Corrected.
Indonesia/M&E
Arief/MCA
7 Paragraph 2: PMaP 2 and 8 has been contracted and PMaP 5 are cancelled. Corrected.
Indonesia/M&E
Arief/MCA
8 Para 5: By July of 2015, 21 TAPP grant agreements had been issues Clarified in the text.
Indonesia/M&E
para 2: the secuence should be: there are 21 TAPP has been awarded for window 3A.
Arief/MCA 9 Then 7 grantees out of 21 received the TAPP Extension. Then 5 grantees awarded the Clarified in the text.
Indonesia/M&E Grant Agreements to move to the construction stage.
Arief/MCA The table: according to Grant Agreement, the Implementer for W3A‐59 is PT Anekatek
9 Corrected.
Indonesia/M&E Consultant not Castlerock Consulting
As the methodology will apply impact evaluation for W3A‐59 I East Sumba, there is still
Explanation of determining
unclear explanation on determining the control group. What kind of criteria used to
sample frame and sample
29‐34 determine that these control group does not receive any intervention except MCA‐
for control group is in
Arief/MCA Indonesia project. it is better to have explanation regarding the situation in control group
section 3.3.3
Indonesia/M&E to make us understand the link between criteria and condition in control group.
We will explore these
factors. EQ4 directly asks
about the SPV's effect on
Regarding the Evaluation Question No. 4 related to the SPV, it is also valuable to assess sustainability. Our
29 about the factors that makes the SPV sustainable. This is unclear whether this key success qualitative approach says we
factor will be one of the results of this evaluation. will "explore how each SPV
approach/scheme ultimately
Arief/MCA impacted the achievement
Indonesia/M&E of outcomes of interest."
The evaluation will evaluate the project implementation based on project theory of We will assess the outcomes
change. This assumes that the project theory of change is correct. However, it is also and assumptions of the
29‐34
Arief/MCA possible that the deviation in the implementation because of the development of the theory of change as well as
Indonesia/M&E theory of change is very weak such as not through assessment from previous experience. assess its relevance in our
SOCIALIMPACT.COM 85
Reviewer
Page Number
(in draft EDR)
Institution
therefore, it is valuable also to assess the process of the development of the theory of literature review, but we
change. belive that an analysis of the
development of the theory
of change is outside the
scope of this evaluation.
For SPV model, it is very difficult to see the genesis of the SPV structure. In Indonesia,
there are many community project that tries to develop an organizations to maintain
infrastructure sustainability such as Community water supply project for low income
(Pamsimas), National Community Empowerment for rural area (PNPM Perdesaan) and for
environmental protection (PNPM Green) not to mention other projects on maintaining Incorporated in grantee
29‐34
community infrastructure such as Community electricity implemented by Ministry of protocol
Energy and Minaral resource and community empowermnet in fishery and coastal area
implemented by Ministry of small island and Marine. I think the evaluator should also to
Arief/MCA assess whether the development of the SPV model has also taken into account the best
Indonesia/M&E practices from similar model developed before.
Syarifah/ PMC Sample selection: For Window 3A, W3A‐59 East Sumba and W3A‐33 Berau were
It is not financially feasible
nominated as the sample for the perfomance and impact evaluation. From
at the baseline stage to
methodological perspective and practicality, we understand why these two grants were
expand the scope of the
selected. However, we are wondering if it is possible to consider additional sample(s) with
evaluation past Berau and
different characteristic (at least for performance if not posible for impact evaluation). For
East Sumba. However, we
example, unlike East Sumba and Berau, W3A‐68 Wakatobi and W3A‐80 Karampuang, as
have included a sub‐section
you are aware, are geographically isolated (small Island). Karampuang is also the most
in the Evaluation design
advance in term of the implementation progress which allow the evaluation to explore
overview to provide initial
more. Further, from the design stage, we've learned that there are at least three types of
ideas for optional lines of
grantees in W3A Pprojects: 1) Contractor: Grantee with the main interest in constructing
inquiry that could be
the facilities and then leave e.g. W3A 56‐58; 2) Grantee with the interest to contribute
pursued with an expanded
to local development as part of their Social Responsibility and to improve profile. Grantee
endline scope, budget
might stay 2‐3 year after the project and then leave e.g. W3A‐33, W3A‐59, W3A‐68; and
permitting. We plan to
3) Grantee with the interest to stay for a long tearm and expand business in the area. It
include H1 Hivos in future
will be interesting to how these different characteristic contribute to the project
data collection periods.
effeciveness and sustainability .
SOCIALIMPACT.COM 86
Reviewer
Page Number
(in draft EDR)
Institution
Syarifah/ PMC SI requested updated
documents from MCC and
The Project information/description presented in the EDR is based on the planning from grantees before final
documents, which some might have changed as the implementation started. There is a submission of EDR.
need as such for the Evaluation team to update the information. Information in EDR reflects
the most updated
documents shared with SI.
Syarifah /PMC Data colection Schedule : given the differences in the project start and implementation The first data collection
progress, we have a concern on the project ability to complete the activities and event in the evaluation is a
demonstrate results by the time the evaluation is started i.e. October. W3A‐59 for baseline. Thus, it is not
example, we are aware that they will only start the trainings on economic opportunity meant to capture any
(realistically) in late September and October. How the evaluator will consider the project results. It is meant to serve
implementation progress in starting teh evaluation process. as a point of comparison for
a future point in time when
we expect to be able to
discern results, if they have
indeed occurred.
Syarifah /PMC Length of questionnaires and time of HH survey: The questionnaires for HH survey is We took out the Willigness‐
comprehensive yet long ( 140s questions) and the interview will most likely take more to‐Accept taks, as it may
than 3 hours/respondent. Given the context of the respondents who are mostly fishermen consume time. We
and or farmers, how would you make sure the availability and most importantly maintain furthermore took out: a)
the interest of respondents in answering the questions. Questions 96 on livestock
and animal products sold
within last 12 months; b)
Question 115 on exact cook
stove use, as electric stoves
are asked for beforehand; c)
Questions 119, 120, 121:
membership in associations,
as community participation
is an expected impact of
electrification but it is not an
explicit target of the
program; d) Questions 125
SOCIALIMPACT.COM 87
Reviewer
Page Number
(in draft EDR)
Institution
and 126 on boiling and
filtering water; e) Question
142 on purpose of mobile
phone usage. However, a
few questions were added
as response to the
comments. We expect the
interview to take around 1,5
hours, given that most
questions are simple yes/no
questions. Also note, that
some questions have to be
skipped when not
appropriate. We ask attitude
and awareness questions
towards the end, expecting
to raise interest. However,
we will pretest the
questionnaire in the field
and will revise, if it's
implementation takes longer
than 1,5 hours.
The IE and PE will largely use
During the conference call meeting, it was told the Evaluation type for this Off‐Grid RE will the same instruments
cover both Performance Evaluation and Impact Evaluation. (although the household
As for the Impact Evaluation, the survey tools, analysis plan and questionnaires are very survey will be more brief for
well prepared and thorough. Will SI develop more in‐depth survey the PE). They mainly differ in
Linny instrument/questionnaire and the analysis plan for the PE? such as questions related to that the PE does not utilize

Ayunahati/PMC how the project is being implemented, how is the program management, operational an evaluation design with a
decision making, performance issues,how it is perceived and valued, accomplishments to valid counterfactual. Thus,
date, constraints. The Grantee KII Protocol covers some questions related to the outcomes of interest will be
evaluation on the implementation, is it enough? Who are the other target respondents to characterized by a simple
triangulate the information? pre/post measurement
technique and qualified
SOCIALIMPACT.COM 88
Reviewer
Page Number
(in draft EDR)
Institution
from stakeholder views on
program effectiveness.
We understand the reason for the selection of 2 projects for this evaluation. However, it is
SI appreciates this
suggested to include a sample of project at a more advance level of implementation
recommendation. However,
(W3A‐80) for the Performance Evaluation, with the objective to learn from the design
for the reasons outlined in
Linny phase to the implementation/completion phase of the construction on the performance,
the EDR, we and MCC
Ayunahati/PMC design, effectiveness and operational performance. What worked well, what could be
agreed to target the grants
done differently, what are lesson learned for the implementation of similar projects in
in Berau and East Sumba for
future in Indonesia. Lessons and recommendations will be very useful for the Government
evaluation.
of Indonesia and other stakeholders.
8 by July 2015 ‐more than 7 TAPP grants existed Corrected.
Jeff D. PMC
Corrected based on grant
agreement. Previous figures
were from DFS. Note
9 W3A‐33: check size of combined capacity ‐1671 may not be correct
summary table was
replaced, correction is in
Jeff D. PMC grant‐specific section.
Corrected. Note summary
W3A‐56‐58: They are not using 10 different facilities ‐ they are using 7 gasifiers machines table was replaced,
9
spread over 3 locations. (Additional fact checking shall be conducted to ensure accuracy). correction is in grant‐specific
Jeff D. PMC section.
Summary table was
W3A‐59: They are helping to establish Cooperatives in each village, but there is only 1 replaced, but correction
9
"SPV" made in grant‐specific
Jeff D. PMC section
Corrected based on grant
9 W3A‐68: Combined capapcity of output much higher than report stated 300kW agreement. Previous figures
Jeff D. PMC were from DFS. Note
SOCIALIMPACT.COM 89
Reviewer
Page Number
(in draft EDR)
Institution
summary table was
replaced, correction is in
grant‐specific section.
SPV approach corrected
W3A‐33: They are not using the "KSM" approach. They are using the BUMDes structure
16 based on May 2017 SPV
within the village government
Jeff D. PMC Business Plan
Summary table was
W3A‐56‐58: says 7 facilities here (page 9 uses 10). Is a facility considered the building/site replaced, but grant‐specific
17
used for generation ‐or‐ the number of electricity generating equipment used? section references gasifiers
Jeff D. PMC as facilities
SI did not receive more
updated business plan than
DFS. We note in Evaluation
Design that SPV approaches
W3A‐56‐58: This project is changing the "SPV" structure. No "LVEs" are currently will be updated using
17
envisioned and most recent 'SPV" business plan is different than what is presented here. primary data collection
where possible, and
recommend that these be
updated ex post where
Jeff D. PMC possible.
Updated based on most
18 W3A‐59: construction start was postponed past May 2017
Jeff D. PMC recent work plan
Based on July 2017 SPV
business plan, this grant is
still using a cooperative
approach. We will update in
19 W3A‐59: They are no longer pursuing the Cooperative approach ‐using BUMDes instead
evaluation report based on
primary data collection and
any updated
Jeff D. PMC documentation.
SI did not receive more
updated business plan than
20 W3A‐80: They are no longer pursuing the BUMDes approach ‐using Cooperatives instead
DFS. We note in Evaluation
Jeff D. PMC Design that SPV approaches
SOCIALIMPACT.COM 90
Reviewer
Page Number
(in draft EDR)
Institution
will be updated using
primary data collection
where possible, and
recommend that these be
updated ex post where
possible.
W3A‐33: please note that the "micro‐hydro component is insignificant to the overall Noted and included in the
31
Jeff D. PMC supply of electricity in the mini‐grid. text.
On point 4, please check whether it should instead be stated as 'landscape‐lifescape
Aretha Aprilia. 9 Corrected.
analysis'
PMC
In order to reflect project
documentation and US
industry standard
classifications
(https://www.irena.org/Doc
W3A‐33: "Connect 463 households across 3 villages to grids powered by 3 Solar umentDownloads/Publicatio
Photovoltaic (PV) and 1 Micro‐hydro facilities." It should be 'Pico‐hydro' as the installed ns/RE_Technologies_Cost_A
9
capacity is only 30 kW (when the actual capacity estimates based on PMC's engineering nalysis‐HYDROPOWER.pdf;
team is less than 10 kW). pg. 11) we will continue to
use micro‐hydro. However,
we have updated text to
reflect relative capacity of
Aretha Aprilia. micro‐hydro to solar in
PMC Berau
"The East Sumba regency was targeted by this this project based on previous studies
executed under an ADB Technical Assistance grant (TA 8287) held by the implementer,
12 Corrected.
Castlerock Consulting" ‐‐> The implementer of project is PT Anekatek. Castlerock provided
Aretha Aprilia.
service on the crosscutting deliverables.
PMC
"Over the course of a month while the facilities are under construction, Electric Vine
Aretha Aprilia. 19 Industries (EVI) will conduct Focus Group Discussions (FGD)…" ‐‐> The Implementer of this Corrected
PMC project is PT Anekatek, not Electric Vine.
SOCIALIMPACT.COM 91
Reviewer
Page Number
(in draft EDR)
Institution
“In the case of W3A‐33, these additional streams include increased income for honey and Without commenting on the
boat production and additional resource cost savings on ice for fishing.” The statement on validity of this statement,
honey and boat production came from PT Akuo (the implementer), which needs further we simply note in this
23 revalidated. section that it is included in
the ERR to which we have
Aretha Aprilia. Secondly, the term 'ice for fishing' should perhaps be replaced by 'ice for storage of access. Sentence on "ice for
PMC caught fish' or something to that effect. fishing" corrected.
In regards to the policy relevance, there were no mentioning of the recent development
of RE policies e.g. the Ministerial Decree No. 38/2016 was devised to help expedite
electricity development in the remote villages across the nation. Conversely, the Energy
27
and Mineral Resources Ministry Regulation No. 12/2017 and Industry Ministry Decrees This is included in the
Aretha Aprilia. No. 4 and 5/ 2017 on requirements for local content for solar PV modules in Indonesia literature review section
PMC may inflict negative trends in RE development throughout the country. now.
SI appreciates this
recommendation. However,
for the reasons outlined in
the EDR, we and MCC
Project W3A‐80 has more advanced progress compared these two projects, has overall
agreed to target the grants
better performance in both engineering and crosscutting deliverables, and therefore shall
29 in Berau and East Sumba for
be considered to be investigated further in order to obtain the 'good practices'
evaluation. We note that
perspectives.
W3A‐80 may be considered
for ex post evaluation,
Aretha Aprilia. where good practices could
PMC be noted.
We took out the Willigness‐
to‐Accept taks, as it may
consume time. We
There are 144 Questions in total, which would essentially take a lot of time and may cause furthermore took out: a)
N/A inconvenience to the HH respondents. The authors shall select questions carefully and Questions 96 on livestock
tally them with the main objectives / key questions. and animal products sold
within last 12 months; b)
Aretha Aprilia. Question 115 on exact cook
PMC stove use, as electric stoves
SOCIALIMPACT.COM 92
Reviewer
Page Number
(in draft EDR)
Institution
are asked for beforehand; c)
Questions 119, 120, 121:
membership in associations,
as community participation
is an expected impact of
electrification but it is not an
explicit target of the
program; d) Questions 125
and 126 on boiling and
filtering water; e) Question
142 on purpose of mobile
phone usage. However, a
few questions were added
as response to the
comments. We expect the
interview to take around 1,5
hours, given that most
questions are simple yes/no
questions. Also note, that
some questions have to be
skipped when not
appropriate. We ask attitude
and awareness questions
towards the end, expecting
to raise interest. However,
we will pretest the
questionnaire in the field
and will revise, if it's
implementation takes longer
than 1,5 hours.
Thank you. This was a
Aretha Aprilia. 113 What is "EWSA electricity" as stated in Q11? mistake. It has been
PMC replaced by PLTMH
SOCIALIMPACT.COM 93
Reviewer
Page Number
(in draft EDR)
Institution
Yes‐‐SI will provide primary
empirical inputs to this
Regarding the Outcome 3 on GHG emissions: as we understand that MCAI has hired a
model and qualify any of the
30 consultancy firm to conduct GHG emission evaluations to estimate the avoided emissions,
models assumptions if they
using IPCC methodologies. Does this relate to that assessment results?
Aretha Aprilia. are not supported by our
PMC data collection.
Econ Team 23 For your kind information, ERR for community‐based RE projects have been through some
re‐scopings, such as increased/decreased in project cost, number of HH to be electrified.
And as per today (August 8) ERR for:
MCC has asked SI to use ERR
1. W3A‐33 Project is 24.5.%,
estimates in the report to
2. W3A‐56‐58 Project is 11.82%,
which we had access at the
3. W3A‐59 Project is 16.82%,
time of the draft EDR (July
4. W3A‐68 Project is 15.7%,
2017). We will update these
5. W3A‐80 Project is 34.91%
estimates in our evaluation
Yes indeed, there were additional benefits for W3A‐#33 and W3A‐#80. But MCC
report according to updated
Economist suggested to generalized all ERR models, which is, at least for now, no need to
documentation.
put additional benefits for all W3A projects.
Econ Team 23 Since one of the outcome for these projects is replacement for fossil fuel used, such as These variables are included

kerosene and solar for diesel genset, it would be good to get data and information about in our household survey
how many litters are kerosene/diesel that was replaced after connected to RE grid. instrument.
We added these questions
to the household
questionnaire and similiar
Refine the questionnaire to measure level of increased awareness contributed by the questions to the grantee
questionnaire
project ‐ whether the beneficiaries: (i) understand what is RE; (ii) understand the protocol. Training materials
p 88 ‐ 89
importance of RE; (iii) understand the cost & benefit of using RE; (iv) somewhat reach the will be reviewed and
level of understanding on importance of RE that add/push them to exert effort to sustain potential changes made to
the RE investment. Triangulate the questions by interviewing the grantee and reviewing the instruments upon
Rini/M&E the quality of training materials developed by the grantees. receipt of the materials.
SOCIALIMPACT.COM 94
6.1.2 MCC Evaluation Management Committee Comments and Evaluator Responses

Page
Reviewer Name/ Number
Comment Evaluator Responses
Institution (in draft
EDR)
Remember that there are two types of RE grants under GP: on‐grid/commercial and off‐
grid/community. This evaluation only covers off‐grid, so it should be titled accordingly.
Something along the lines of "Grant Facility Community‐based Off‐grid Renewable Energy
Grant Portfolio". This wil come up again in later comments, but this evaluation should refer
Thank you, we have made this change in
1 mostly to the community‐based off‐grid renewable energy grant portfolio. It's fine to
referencing the grant portfolio.
reference the windows when introducing the project, but after windows closed, MCA‐I
shifted to discussing grants in the context of portfolios. This is at least in part because
MCC Energy eval grants with RE components ended up being awarded under all 3 windows and so the
lead window distinction is no longer meaningful.
MCC Energy eval PLUP is not going to end up working in all 45 districts. Please change to "Overall, PMAP
7 Done.
lead contracts will include implementation in UP TO 45 districts throughout Indonesia"
To make the term "windows" slightly clearer, you could say "funding windows' in this
sentence: Grants will be funded through three schemes, or “windows”: Partnership Grants
7 Done.
MCC Energy eval (Window 1), Community‐based Natural Resource Management Grants (Window 2), and
lead Renewable Energy Grants (Window 3).
For the sake of comprehensiveness of the GP evaluation work after the Compact closes, the
focus of the evaluation should be described as "community‐based off‐grid" not Window 3A.
The EDR was being developed as MCC was getting a better sense of what interventions
were incorporated into the W2 portfolio, as we were not involved in the investment
decisions on W2. Midway through the EDR development process we notified SI that there We have reshaped the report as requested
was actually a significant number of off‐grid RE grants in W2 and asked those to be in this comment, presenting the context by
considered for this evaluation. We can understand if SI was not able to incorporate W2 into which all CBOG RE grants were funded, not
8 the evaluation design for technical or practical reasons, but the existence of those off‐grid just those from Window 3A. We have
grants needs to be acknowledged in the EDR. Similarly, there was one W1 grant with an RE included ways that the evaluation's scope
component (WWF), though it seems the RE component is not going to proceed. Section could be expanded in future data collection
2.1.1. will need to be edited to discuss off‐grid RE under different scenarios: W1, 2, and 3A. periods to include some of the other grants.
The description of the TAPPs and 3A is fine, but the report has to acknowledge that there
were also off‐grid RE projects in W2 that were awarded and designed with a slightly
MCC Energy eval different approach. You can make reference to the one W1 RE grant, though not much
lead needs to be written about that.
SOCIALIMPACT.COM 95
Page
EDR)
Added the following sentence: "Compared
It's important to specify that 3A is off‐grid (in addition to be community‐based) and that 3B
MCC Energy eval 8 to the off‐grid W3A grants, all W3B grants
is on‐grid (in addition to being commercial‐scale)
lead fund on‐grid electrification. "
The description of 3A TAPP grants needs to be updated. Please see the 2017 M&E Plan for
MCC Energy eval 8 more details (both the GP project description in the narrative and Annex V). 21 3A TAPP Corrected based on MCA‐I comment.
lead grants were signed.
Included in the new overall CBOG RE
A 6th 3A grant is likely to be signed in August. If it does go through, it should be referenced portfolio table. It does not have a section in
9
MCC Energy eval in the table with a footnote similar to the Puriver one. the narrative because we were unable to
lead review project documentation.
New Table 1 is a list of all relevant CBOG RE
grants provided by Shreena/MCA‐I. We
included more detailed sections for the ten
2.1.2. would ideally cover the other off‐grid RE grants as well, even if not in as much detail. grants for which we were able to review
10
A list of the relevant W2 grants and a sentence or two on each could be sufficient. project documentation. Note that we only
had the grant agreement for the Puriver
MCC Energy eval grant so we were unable to fill out
lead implementation to date section.
MCC Energy eval Please refer to the analysis as "cost benefit analysis" rather than "ERR analysis". The ERR is
22 Done.
lead just one statistic to come out of a CBA.
We have added a note following the
evaluation questions to indicate that
Given the planned comparison between the two RE grants (Anekatek vs. other), it might findings will be analyzed and presented
make sense to add an evaluation question or sub‐question to one of the existing questions, within and across grants to identify patterns
29 along the lines of: How do energy consumption patterns differ across [type of technology / or differences. However, we also note that
geography / window]? Or are findings consistent across different types of grants? The the lack of a counterfactual in Berau can
wording of this Q will depend on which second grant the group opts for. heavily confound results and is not an equal
MCC Energy eval comparison to the counterfactual‐based
lead design in Sumba.
MCC Energy eval 31 / Let's agree on a name for grants other than the grant number and replace references to the See new naming convention preceding Table

lead General grant numbers after the initial introduction. Apparently there are sometimes two different 1.
SOCIALIMPACT.COM 96
Page
EDR)
numbers in the accounting systems. I would suggest "Sumba Anekatek Solar" for 59, for
example.
Per Anekatek's updated work plan,
commissioning in Sumba will take place
between November of 2017 and February of
Exposure period ‐ note that 1 year after the baseline might only give households an
2018. This will make for exposure period of
exposure to treatment of 6 months (let's say works aren't completed until March 2018).
between 8‐11 months. As discussed in
33 This doesn't seem long enough to expect HHs/businesses to have changed behaviors and
presentations, there is also concern about
bought new appliance, particularly for productive uses. I would conduct the follow‐up in
contamination from PLN electrification after
2019. Is the stakeholder input on adoption of electricity in Sumba documented in a report?
October of 2018. For this reason, and per
MCC Energy eval MCC's approval issued via email on 8/10, we
lead will maintain our proposed exposure period.
Based on email from Vivian following DC
presentation, SI's understanding is that we
We'll need to be clear whether the EDR is approved including the 3rd round of data
34 are to move forward with both follow ups in
collection 2‐3 years post‐installation or not. The wording should be adjusted accordingly.
MCC Energy eval the EDR. We have included both follow ups
lead in the budget.
Added a footnote to describe this. In
How do these MDEs compare to the assumptions in the CBA for Anekatek? Please summary, we are likely adequately powered
35 document this so that we have a sense of how the power of the evaluation compares to the to detect changes in consumption, although
MCC Energy eval investment committee's expectation of impacts. we may be underpowered for changes in
lead expenditure based on CBA's estimates.
Added the following footnote to the TA's
introduction in section 2.1.2.3: "The purpose
of this TA was to “support the GoI’s Sumba
Iconic Island Initiative,” which aims to
Please introduce the ADB TA referenced, at least in a footnote. What did the TA cover, electrify 95% of households on the island of
36
where was it, etc. so that readers understand how it's relevant. Sumba via 100% renewable means by 2025.
The referenced Network Planner exercise
was part of a “comprehensive least‐cost
MCC Energy eval electrification planning exercise” for Sumba,
lead wherein the most cost‐effective and
SOCIALIMPACT.COM 97
Page
EDR)
technically appropriate means for achieving
a 100% electrification ratio were laid out
(ADB 2014). "
When proposing optional work, the EDR needs to be clear about what's actually agreed to New sub‐section included in Evaluation
MCC Energy eval 44 and what's optional for a future decision. Perhaps add a section to the end with Optional Design Overview to separate proposed
lead expansions to the evaluation. evaluation work from optional add‐on work.
Correct abbreviation for PSGIP is Project Social and Gender Integration Plan (please do
v Done.
MCC GSI lead make necessary correction accordingly throughout the document
We have replaced this quote justifying
investment in RE for low‐carbon growth with
SI states the following "Indeed, the Asian Development Bank (ADB) indicates that there is the following "Indeed, the Asian
substantial evidence that “energy efficiency is the best way to mitigate GHG emissions Development Bank (ADB) indicates that
while also meeting the growing requirements for energy services that accompany renewable sources of electricity offer many
6
expanding economic growth". RE under GP is not "energy efficiancy" project, rather “positive cobenefits” in addition to reduced
reducing fossil fuel and GHG reduction. What is the rationale of refering to "energy GHG emissions including rural revitalization,
efficiency" ? jobs and employment, economic
development, and avoided environmental
MCC GSI lead costs of fuel extraction and transport. "
We present GP's objectives in a later
section‐‐this section is referencing GoI's own
The objective of GP is to help Indonesia reduce GHG, as it is one of the worst polluters. Not
6 policy and justification for reducing GHG
because it is suffering from global warming events.
emissions, per the cited Green Paper, which
MCC GSI lead is tied to their policy on climate change.
We felt it important to include downstream
The Theory of change states " community cooperatives will have the capacity to operate
impacts, as these are part of the testable
20 and manage the micro‐grids such that household income will be increased and GHG
theory of change, but we included an
emissions decreased…" . It is better to have a more practical narrative such as "micro‐grids
MCC GSI lead intermediate sentence using the suggested
SOCIALIMPACT.COM 98
Page
EDR)
will be sustainable as communities receive benefits and SPV will continue to provide O&M language to improve the practicality of the
services. narrative.
One very important gender equality outcomes with electricity access is missing. Women are These outcomes are omitted from this
able to save time and efforts by using timesaving appliances and equipment that need section because they are not included in
electricity. This results women having more time for human development, social, project M&E documents, but the evaluation
21 networking and economic opportunities, able to take rest, and feel secure in lighted does aim to capture and report on these
homestead. See Deloitte (2014); ENERGIA (2007); Dinkelman (2011); Evidence on positive "typical" outcomes, as appropriate (see
impact on women's employment and income opportunities from electrification (example instruments). We added a footnote to make
MCC GSI lead from South Africa, India): Dinkelman, Taryn (2011). this more clear.
for primary question 2: need to differentiate between productive use of electricity at Added the words "… at the community or
29
MCC GSI lead community/cooperative and at household levels. household level" to the question for clarity
For primary question 4: additional probing questions can include (a) does the SPV
governance ensure transparent and participatory monitoring of community? (b) Has the
We will include these probing questions in
SPV governance/business models taken into account of equal participation by women and
our instruments, but feel it is superfluous to
marginalized/poor households in SPV formation and management? (c) How has the
29 include them in the evaluation question
implementer prepared the smooth transition/hand over to community SPVs? (d) how has
itself. See updated SPV leadership and
the SPV business model/business plan fit into existing condition of the village in terms of
grantee protocols.
willingness/affordability to pay for electricity? (especially to ensure that less affluent
MCC GSI lead household are able to pay for electricity)?
Household information should also covers the following: number of family members,
primary occupation/livelihoods and composition of gender, to differentiate electricity use
for productive activities. In W3A‐33, most villagers are farmers, where electricity
consumption may reach its peak during the night for leisure and others, the most affluent
households may be family of traders with more opportunity for off‐farm economic activity
that needs electricity. in W3A‐59, agriculture post harvest processing is presumably
37
contributes to productive use of electrity. Another point is any information about non‐farm
asset/livelihoods should also be included. In W3A‐33 location, there are households
opening stalls, or households with handicraft making as source of income in addition to
agriculture activities. In household activities, household chores should also be identified
clearly, to ensure that reproductive activities such as water‐fetching, child rearing/nurturing This information is included in the
MCC GSI lead be included. On key health issues, outcomes should include other than cleaner energy use, household survey.
SOCIALIMPACT.COM 99
Page
EDR)
but longer availability of electricity for key health services, especially for child and maternal
health. Longer availability of electricity can contributes to availability of cold chain for
vaccine, better assistance and support for pregnancy with complication (maternal health),
better access to clean water and good sanitation.
Security should also include mobility in the community/village, W3A‐59 includes street The outcomes are captured in the household
38
MCC GSI lead lighting, security/safety in mobility outcome should also be diseggregated by sex. survey.
For Beneficiary FGDs, there will be separate
male and female groups to ensure inclusion
of female perspective. For enterprise
surveys, all enterprises (included female‐
operated) will be included. For HH survey,
Protocols need to include gender balance in selecting member of sub‐village, women need
38 we will interview whoever is most
to have their voices in all structures.
responsible in household for decisions
related to electricity usage. All other
instruments will select people per their roles
in their respective organizations, whatever
MCC GSI lead gender they may be.
Limiting the respondents to household head may yield biased/onesided view on The plan is not to interview household head,
energy/electricity consumption, not to mention gender bias, as household head is but to interview person in household
considered male. In most households observed in Berau, many questions raised during responsible for decisions related to
42 consultation come from women who are the household "manager" who oversee household electricity, whether they are male or female.
spending, including on electricty. Having both men and women interviewed has merit in SI will take steps during piloting to ensure
getting better information about electricity consumption, expenditure, clearer information that we are not prevented from speaking
MCC GSI lead on family members activities and time use. If this cannot be done, steps to avoid gender with female household managers
SOCIALIMPACT.COM 100
Page
EDR)
bias in responses should be taken such as triangulation with other methods. Pilot test
should also sensitive to this issue.
Due to concerns around the length of the
Questions for household health status should also incorporate maternal health, filter survey, we have not included questions
87
question on female household members pregnancy status and health will be needed related to maternal health as it is not a
MCC GSI lead primary intended outcome.
Due to concerns around the length of the
In addition to health insurance, type of health service providers accessed should also be
survey, we have not included questions on
captured, such as puskesmas, pustu, house of bidan (midwives), traditional healers, halfway
87 this in the household survey. However, we
houses for labour, in case of maternal health complication (this will give information on
do include questions about health care
potential electricity needed for basic service at village level)
MCC GSI lead options in the village survey.
Questions on perception on gender equality among community members need to be asked,
as required in PSGIP development, gender equality has to be integrated in key critical stages
93
of SPV development, such as engagament and participation in SPV establishment as well as SPV leadership protocol updated to include
MCC GSI lead information sharing on SPV. such a question
Population figures disaggregated by
95 all demographic data should be disaggregated by sex , where appropriate male/female in regency and village official
MCC GSI lead protocols.
Regarding educational attainment, the
for the open ended question on living condition, the protocol does not include explanatory questionnaire includes questions on the
notes on what is considered as 'improved living condition' , while general poverty indicators highest level of education, the years spent
such as housing condition is understood, the response could be varied. Hence it will be on education, and the time spent studying
good if there is an explanatory notes to explain the living condition. It will be good if inside and outside of school of children
100
improvement of living condition could also identify availability of basic social services, such (disagregared by gender). We elicit
as availability of (functional) health and education services and basic human development information on availability of basic services
improvement such as education attainment, health status (numbers/incidence of maternal via regency and community official
mortality and children under two, malnutrition, etc). instruments. We added maternal health
MCC GSI lead questions only to the FGD guides and Village
Page
EDR)
Official questionnaire, as we do not expect
electricity access to affect maternal health
variables with a detectable size, as the
timeframe is only one‐year, the channel of
this impact is only indirect, and as sample
sizes of pregnant women within this year
will be too small to detect effects.
A few questions on gender awareness were
added to the questionnaire. They may be
102 Question no. 22L: please refer to comment in p. 93
adopted to the training context after
MCC GSI lead receiving training materials.
This is included in the regency and village
103 There is no reference on basic social service facilities (Schools, Puskesmas, etc) at Regency
MCC GSI lead official protocols
At Regency‐level, information on priority agriculture commodities (produk pertanian
105 unggulan) and value‐addition agro commodities could be available , it should also be on the
MCC GSI lead list of question. This has been added.
MCC GSI lead 106 Similar comment on living condition This has been added.
MCC GSI lead 107 Similar comment on gender equality for question no. 16 This has been added.
6.2 Annex 2: Evaluation Budget
Per MCC’s instructions regarding sensitivities around future procurements, the evaluation budget
corresponding to this Evaluation Design Report has been provided to MCC separately.
6.3 Annex 3: Instruments

This annex includes the English version of all instruments that we plan to use in the evaluation, as they were
referenced in Sections 3.3.4 and 4.3.4. W3A Anekatek Solar, East Sumba/W3A Akuo Energy Solar/Micro-
Hydro, Berau Household Survey.
6.3.1 Household Questionnaire
HOUSEHOLD 1. Questionnaire N°
QUESTIONNAIRE
2. Site code
Impact Evaluation Baseline Study 2017 3. Geo coordinate

Green Prosperity Renewable Energy
Grant
4. Date
Hamlet
5.
6. Interviewer’s 7. Starting Time of : h

name Interview
The walls of the main

8. 9. The main roofing 10. The main flooring
building consist of…
material is … material is…
1 Earth
1 Bamboo 1  Ijuk 2 Bamboo
2 Wood 2  Palm leaves 3 Wood
3 Coconut stem 3  Wood 4 Concrete
4 Unburnt bricks 4  Iron sheets 5 Bricks
5 Burnt bricks 5  Concrete 6 Stones
Other______ 6  Tiles 7 Ceramics
 Other_____ Other_____
11. Are the windows fitted 12. Is the building

with glass? plastered?
1 Yes 1  Yes 
0 No 0  No 
[COMMENTS]
Basic Information
13. 14. 15. 16. 17. 18. IF CODE 2. or 3.

19. 20.
Who are the Sex Age Education First Second Where does How much
permanent residents Occupa- Occupa- he/ she does he/
1. 2.
of this household? tion tion exercise she earn
What relationship does Level of Number of this per
each member have to education years occupation? month?
the head of
Years
household? [Only
include household m / f years code [WITHOUT code code code IDR
RE-
members who are at
PETITIONS
least 11 years of age]
a. _______ 1. 1.
2. 2.
1. 1.
b. _______
2. 2.
c. _______
d. ______
e. _____
f. _____

g. _____
h. _____
i. _____
j. _____
k. _____
l. _____
m. _____
n. _____
21. Who is the head of CODE of Q.13 CODE of Q.17 and 18

household?
1. Head of household 1. Farmer (independent)
LETTER OF Q.13
2.Spouse 2. Farmer (dependent)
3. Father/ mother 3. Civil servant [SPECIFY]
22. Who is the interviewee?
4. Brother/ sister
4. Other dependent
LETTER OF Q.13
5. Son/ daughter occupation [SPECIFY]
6. Grandchild 5. Other independent
23. Is any female household 7. Other relative
occupation [Specify]
member pregnant? 6. Studies
8. Servant
LETTER OF Q.13 9. Other non-relative 7. Domestic work, child rearing
8. Without occupation/work
24.
How many children CODE of Q.16.1 9. Retired

between 6 and 11 years
live in the household? 0. None
CODE of Q.21
1. Primary school
2. Junior high
25. How many children 1. Same village
school
younger than 6 years live
2.Village in same Kecamatan
in the household? 3. Senior high
school. 3. Village in same Kabupaten
4. Vocational
training
5. University
26. [TOTAL NUMBER OF

PERSONS IN
[COMMENTS]
HOUSEHOLD.]
27. 28. 29.

How many buildings How many rooms are How long have you been
does your house have? there in your main living on this plot of land?
house [excl.
bathroom]?
30.
Do you own the following means of transportation?
[IF SEVERAL, GIVE NUMBER]
0 No
1 Bicycle ____ 4 Cart ____
2 Motorcycle ____ 5 Tractor ____
3 Car ____ 6 Other: ______________
2. Persons migrated
1  Yes
Have any former household members migrated? 0  No  q.37

31.
32. 33. 34. 35. 36.

What relationship does he/ she What is What is his/ her Where did he/ For what reason
have to the head of his/ her education level? she migrate to? does he/ she live
household? age? somewhere else?
0. He/ she is the head of household age 0. None 1. Jakarta 1. Seasonal work
1. Father 1. Primary school 2. Village in same 2. Daily wage
Kecamatan
2. Mother 2. Junior high school 3. Regular work
3. Village in same
3. Son 3. Senior high school 4. Scarcity of land
Kabupaten
4. Daughter 4. Vocational training 5. Lack of work
4. Makassar
5. Spouse 5. University 6. Studies
5. Padang
6. Other 7. Marriage
6. Other, – specify
8. Other, specify?
1.
2.
3.
4.
[COMMENTS]
3. Electric energy
37. [COMMENTS]
Do you have the following electricity sources in your
household? [SEVERAL ANSWERS POSSIBLE]
0  None  q.38
1  Car battery (without solar panel)
2  Individual genset
3  Connection to a MHP
4  Individual traditional waterwheel
5  Traditional waterwheel in the village
6  Genset in the village
7  Genset shared with neighbour
8  Solar panel (installed on roof)
9  _________________ kW of solar panel 
10  Solar panel (not installed on roof) 

11  _________________ kW of solar panel 
12  Solar PV Kit 
13  PLN  q.41.3
a b c d e f g
Car Traditional Solar
MHP Genset PLN No
battery water wheel panel
38. Have you ever used an
electricity source in this
1 2 3 4 5 6 0
household? If so, which
type?
39. How many years has it
been since your
household was
disconnected from the -3
electricity source or since years years years years years years
the source become non-
functional?
40. Why are you no longer
connected to the
electricity source? -3
1. No longer interested
2. Not able to pay the bill
3. Other: ____
 q.46
1. 2. 3.
When did you receive the When did you receive the When did you receive this
41.
battery? genset? electricity source?
YEAR YEAR MONTH -YEAR
How many times per year do Which fuel do you use for the How much did you pay for the
42.
you charge the battery? genset? connection and the electric
installation in your house?
1  petrol
2  diesel
____________ IDR
TIMES
43. How much do you pay for How many litres of this fuel do How did you pay for it?
charging the car battery? you consume per month?
1  Cash
2  Credit
3  sell sth, what?

_____________
IDR LITRES 4  Donation
How long does it take you to How much do you pay per litre How much did you pay for the
44.
reach the place where you for the corresponding fuel? current line last month?
charge the battery?
IDR IDR IDR
How much did you pay for the How much did you pay for the
45.
reparation of this electricity reparation of this electricity
source last year? source last year?
IDR IDR
 q.41.2 or 41.3, if other electricity  q.41.3, if other electricity

source in household source in household
4. Energy for appliances and lighting
47. 48.
46.
Do you use any of these Do you use the Does any household member
appliances or machines appliance(s)/ machine(s) use any of the appliances/
in your home? to produce goods to sell machines outside the
[READ ALL]
at home? If yes, for how household?
much time?
If yes, how many? If yes, where?
1 = At a friend’s place 2 = At work
3 = At a neighbour’s house 4 = other,
0. None   q.49 specify
1. Iron 
a. Charcoal No Yes YEARS:
b. Electric No Yes YEARS:
2. Refrigerator 
a. Fuel-run No Yes YEARS:
3. Electric stove No Yes YEARS:


4. Electric kettle ------------
5. Rice cooker No Yes YEARS: 

6. Magic Jar No Yes YEARS: 
------------
7. Ventilator
8. Landline telephone No Yes YEARS: No Yes PLACE:
9. Mobile phone No Yes YEARS: No Yes PLACE:
10. Radio ------------ No Yes PLACE:
a. Battery only ------------
b. Bivalent ------------
c. Line power only ------------
11. CD / VCD No Yes YEARS:
12. TV  No Yes PLACE:
a. Black and white No Yes YEARS: 
b. Color No Yes 

YEARS:
13. Satellite receiver ------------


14. Computer No Yes YEARS:
15. Printer No Yes YEARS:


16. Mill
a. Fuel-run No Yes YEARS:
b. Electric No Yes YEARS: 

17. Sewing machine
a. Mechanical No Yes YEARS:


18.
Other :__________ No Yes YEARS:
49. 50. 51.

Do household members Who is the household Which are the three most important electric
use electric appliance(s)/ member [use code appliance(s)/ machine(s) the household
machine(s) to produce Q.13]? member uses?
goods/ offer service

outside home?
A _______________
1
1 Yes _______________ B _______________
0 No  q.52 C _______________

1 Yes A _______________
2
0 No  q.52 _______________ B _______________
C _______________

1 Yes A _______________
3
0 No  q.52 _______________ B _______________
C _______________

1 Yes A _______________
4
0 No  q.52 _______________ B _______________
C _______________

1 Yes A _______________
5
0 No  q.52 _______________ B _______________
C _______________
[COMMENTS]
52. 53. 54. 55. 56.

Do you charge What is the distance to How much How often How many times
your mobile the place where you do you pay did you did you
phone(s) at charge the battery? per charge? charge personally use
home? your mobile your mobile
phone last phone in the last
week? week?
yes
1   q.55
IDR TIMES TIMES
0 No
-3  No mobile 1  Metres -3  You do not
phone in the have a
household 2  Min. by foot mobile
phone
 q.57  ___________
57. 58. 59.
How many flash How many How often do you use candles?
lights sockets are
[PORTABLE] are there in the
there in the household? 1 Minimum once per day
household?
2 Minimum once per week
3 Only in the case of fuel shortage
4 Only in the case of blackout
5 Never
Other: __________________
1. 3. 4. 5.
Which lighting 1. Hurricane lamp 

60.
sources do you Normal Neon/ Energy 2. Tin lamp 
use in your electric fluorescent saver 3. Gas lamp 
household bulb tube
[INCLUDING 4. Battery-driven LED 
EXTERIOR 5. Rechargeable bulb 
LIGHTING]?
  
How many of Outside Outside Outside

61.
these lamps do ______
you use? ______ ______
Inside Inside Inside
______ ________
______ ______
62. How many hours Outside Outside Outside
per day do you
use the lamp(s)? ___ ___ HOURS ___ HOURS
HOURS
Inside Inside Inside
________
___ ___ HOURS ___ HOURS
HOURS
63. How often are

you satisfied with
the lighting
quality of the
lamp?
1. Always 2. Often ________ _________
3. Seldom 4. Never _______ _______
64. How many rooms
do you illuminate
with these _________
lamps? _________
65. What is this room

used for?
1. Living room
2. Head of HH’s room
3. room of other HH
members
4. Kitchen 5. Toilet
6. Other [SPECIFY]
66. Within the last

year, how many of
these bulbs/lighting
sources you had to
replace because
they were broken?
67. What do you do with the neon 1. Throw away with garbage
lights / energy savers when they 2. Throw away in the toilet
are broken?
3. Throw away into nature
4. Return it to the place where I bought it
5. Other (SPECIFY)
5. Energy sources
1. 2. 3. 4. 5. 6.
Candles Gas Kerosene Charcoal Fire wood Coconu Batteries
ts
kg litres kg bundles number
68. How
many
units of for lighting for cooking collected for cooking for lighting
____ do
you
consume
CANDLES KG
per for cooking for ironing bought for radio
month?
for other other other

purposes purposes purposes
69.
How
much do IDR per
IDR per litre IDR per kg IDR per IDR per
bundle battery
you pay IDR per candle kg
per unit?
MIN MIN MIN MIN MIN MIN
70. Throw away-where:
[If HH uses batteries] 1  Throw away Where ?
What do you do with the ____________________ 1. Into garbage

batteries when they are 2  other : 2. Into toilet
empty? ____________________
3. Into nature
71.
Do you sell charcoal or other forest products? If yes, how much do you earn per month in sales?
Yes ___________
0 No ,
IDR
Which negative impacts have you

observed?
72. Do you see negative impacts induced 73.
by electricity?
1  Yes
0  No  q.74
74. [ HOUSEHOLD HAS A MODERN ELECTRICITY SOURCE  Q.75

HOUSEHOLD DOES NOT HAVE AMODERN ELECTRICITY SOURCE Q.80]
Which are the main

75. advantages of your
1.
electricity source?
2.
3.
 -6 None
Have any of this household’s 77. Which appliance has been damaged?
76.
appliances been damaged due
to voltage fluctuation? If yes,
which appliance(s)?
1 Light bulb/energy saver/neon

2 TV
2 Rice cooker
1  No  q.75
3 Water cooker
4 Radio
0  Yes 5 Other, SPECIFY
Please specify.
78. Do you wish to see any improvement in 79.
the electricity supply?
1 Yes
0 No  q.77
6. Agriculture
80. 81.
Do you cultivate farm What is the property status of
land? your farm land?

1 Yes 1  Your property
0 No  q.95 2  Rented
 3  Bagi Hasil
[COMMENTS]
[EXPLAIN THE DIFFERENCE BETWEEN NON-TRANSFORMED AND TRANSFORMED PRODUCTS]
82. 83. 84. 85.

Please indicate your five Which products did How much did you sell For how many IDR do you
most important agricultural you sell in a non- within the last 12 months sell each unit?
products: transformed way last in a non-transformed [UNIT OF Q.82]
year? way?
1 Apple  No Yes _______ kg
2 Shallot  No Yes _______ kg

3 Hot Pepper  No Yes _______ kg
4 Cocoa  No Yes _______ kg
5 Maize  No Yes _______ kg
6 Orange  No Yes _______ kg
7 Soy Bean  No Yes _______ kg
8 Beans  No Yes _______ kg
9 Peanut  No Yes _______ kg
10 Kangkung  No Yes _______ kg
11 Rubber  No Yes _______ kg
12 Potato  No Yes _______ kg
13 Cucumber  No Yes _______ kg
14 Coffee  No Yes _______ kg
15 Cabbage  No Yes _______ kg
16 Pumpkin  No Yes _______ kg
17 Mango  No Yes _______ kg
18 Mangosteen  No Yes _______ kg
19 Pineapple  No Yes _______ kg
20 Rice  No Yes _______ kg
21 Papaya  No Yes _______ kg
22 Banana  No Yes _______ bunches
23 Watermelon  No Yes _______ kg
24 Cassava  No Yes _______ kg
25 Sugar  No Yes _______ kg
26 Tea  No Yes _______ kg
27 Tobacco  No Yes _______ kg
28 Eggplant  No Yes _______ kg
29 Sweet Potato  No Yes _______ kg

30
__________ __________ __________
31 __________ 
__________ __________

32
__________ __________ __________
86. How much do you earn per year selling non-transformed agricultural
products?
IDR
[COMMENTS]
Do you transform agricultural products? 1  Yes

87.
0  No  q.95
88. 89. 90. 91. 92. 93. 94.

What is Who trans- By which Into what? What is the What are the For how
the basic forms the means does unit? approximate much do
product? product? he transform quantities you sell
the product? 1. De-shelled that you sell each unit?
rice
per year?
[USE THE 1. Family 1.Motorized 2. Hulled coffee Sack of x kg,
CODE OF Member appliance Bottle of x ml,
(male) 3. Flour ...
Q.105] 2.Electric
2. Family appliance 4. Beverage [IN UNITS OF [UNIT OF
Member Q.90] Q.901]
3. Tools 5. Oil
(female)
4. By hand 6. Grilled
3.Employee product
5. Other, specify IDR
4.Other, specify 7. Other- what?
1.
2.
3.
4.
7. Livestock
Do you own domestic animals? 1  Yes

95.
0  No  q.98
96. 97.
Which animals do you How many of these animals do you own?
currently own?
1.  Pig
2.  Sheep
3.  Goat
4.  Rabbit
5.  Buffalo
6.  Horse
7.  Cow
8.  Poultry
9.  Dog
Other,specify
10.  ________
8. Financial Situation
98. 99. 100. 101.

Do you have an account Do you save Did the household Where? [SEVERAL
at a bank or savings money at take up a loan ANSWERS POSSIBLE]
association? home? during the last two
years? 1. Kepada keluarga atau
orang lain
2. Di toko
1  Yes, at a bank 1  Yes Ddi lembaga keuangan
Bbank
2  2 No 3. Lainnya – sebutkan
Yes, at a savings

association
1 Yes
0  No 0 No  q.102 ___________
103. [COMMENTS]
102.
How many remittances do To cover family needs,
you receive per month? your household income
is…
 1  Sufficient
 2  Tight
 IDR 3  Not sufficient
9. Expenditures
104. a. b. c.
Do you spend money on the following expenditures? per per per year
week month
If Yes, how much do you roughly spend?
[TRY TO GET THE INFORMATION ON MONTHLY LEVEL]
IDR IDR IDR
-9. Paid in kind
1. 1  Yes 0  NoRent (house and fields) (in money)
2. 1  yes 0  no
Food (for the whole family)
3. 1  yes 0  no
Crop transformation
4. 1  yes 0  no
Transport (public and private)
5. 1  yes 0  no
Telecommunication
6. 1  yes 0  no
Water
7. 1  yes 0  no
Schooling expenses for children (material,
school fees, transport, etc.)
8. 1  yes 0  no Agricultural expenses (seeds, fertilizer,
dung, pesticides, and worker)
9. 1  yes 0  no Livestock breeding
10. 1  yes 0  no Family and religious ceremonies
11. 1  yes 0  no Remittances to family members who do not
live at home
12. 1  yes 0  no Medical expenses [excl. health insureance]
13. 1  yes 0  no Cigarettes
14. 1  yes 0  no Clothes (for the whole family)
105. 106.
What other large investment [>230.000 [SEVERAL Who manages the household
IDR] did you make during the last 12 ANSWERS budget?
months? POSSIBLE]
1. 
2.  1. Male 
3.  2. Female 
107. 108.
On working days, when does the Father/ man Mother/ woman
... in the household usually...
0.  No father/ man in household  No mother/ woman in
 q.107 household  q.108
1. wake up? _ _: _ _ h _ _: _ _ h
2. perform income generating From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
activities [INCLUDING
FARMING]? From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
3. perform household duties? From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h

4. watch television? From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h

5. perform other leisure From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
activities?
6. go to bed? _ _: _ _ h _ _: _ _ h
111.
109. 110.
On working days, children of age 6-11 male children female children
when do the ... in the
of age 12-17 of age 12-17
household usually...
0.  No male children in  No female children in the
 No children of age 6-11
household of age 12-17 household of age 12-17
in household  q.109
 q.110  q.111
1. wake up? _ _: _ _ h _ _: _ _ h _ _: _ _ h
2. study at home From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
after school?
From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
3. study outside the
house after From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
school?
4. watch TV? From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h
From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h From _ _: _ _ h till _ _: _ _ h

5. go to bed? _ _: _ _ h _ _: _ _ h _ _: _ _ h
[COMMENTS]
112. [ HOUSEHOLD WATCHES TV AT HOME  Q.113

HOUSEHOLD DOES NOT WATCH TV AT HOME  Q.115]
Who decides what kind of program you watch on TV? 1  Adult male
113.
2  Adult female
3  Child < 18
114. 115.
Which TV programs do the Which other activities [THAN Q.135
household members watch? – 136] do the household members
carry out after nightfall?
[DO NOT READ] 1. Radio 2. Reading 3. Praying
1. Cartoons 2. Movies 4. Playing 5. Going out
3. News 4. Soap operas 6. Household duties
7.Other, specify
5. Sports 6. Other, specify
a. Father/ man 1. ____________

2. ____________
b. Mother/ woman 1. ____________
2. ____________
Does any member of the household collect firewood? 1  Yes

116.
0  No  q.117
Who normally collects 1. 2. 3. 4.

117.
wood?
Code Q. 13 Code Q. 13 Code Q. 13 Code Q. 13
118. How much time does

he/ she need to collect _______ _______ _______ _______
wood per week? HOURS HOURS HOURS HOURS
11. Health
1. 2.
119.
Did any members of your household in the Adults >=18 years Children <18 years
last six month suffer from ...?
m. f. m. f.
Male Female Male Female
a. Headaches    
b. Respiratory disease    
c. Eye disease    
120. Do you have a health insurance? 121.

How much do you pay per _____?
1  Yes 
0  No IDR.
 Year / Month /
12. Security
122. 123. 124.

How many days per week Are you concerned for their safety Do you think that
do the members of your when they go out? darkness is
household go out after dangerous?
1. Yes 0. No
nightfall?
-3. Not applicable
1. Man 1. Are you outside after 1  Yes

nightfall?
2. ___ 0  No
Woman
3. 2. Are your female children
Boys 12-17
outside after nightfall?
4. ___
Girls 12-17
3. Are your male children
5. Children <12 outside after nightfall? ___
13. Environmental awareness
125.
Which environmental
issue concerns you the
most? Why?
 -6 None
1. 3. 4. 5.
How much do you Strongly Strongly Agree

agree with the disagree Somewhat Somewhat
following statements: disagree agree
  
Good air quality is a

depletable good
126.
Solar power is a
depletable good
127.
Wood is a depletable
128.
good
129. I consciously try to

conserve energy.
130. I am interested to
know about
environmental
problems
131. I dispose of garbage
in dustbins
132. Everyone has the

responsibility to
preserve the
environment.
A B Please explain. [Write down keywords]
Do you know what 1 Yes

“Renewable Energy “ 0 No
is?  Q.134
133.
Do you think your 1 Yes
community should 0 No
use Renewable  Q.135
Energy?
134.
Do you think 1 Yes
135.
Renewable Energy 0 No
is better for the  Q.136
environment than
alternative electricity
sources?
136. Do you know how to 1 Yes
support longevity of 0 No
a community mini-  Q.137
grid as community
member?
14. Gender Equality Awareness
Do you think that…
Women should take care of housework 0 No 1 Yes

137.
2 No opinion
Women are good in making business 0 No 1 Yes
138. 2 No opinion
Women have the same capacities to gain 0 No 1 Yes
money as men 2 No opinion
139.
Women should do what their husbands tell 0 No 1 Yes
them to do 2 No opinion
140.
Men are better political leaders than women 0 No 1 Yes
141.
2 No opinion
Do you think it is justified that men use violence against women in the following situations
She burns food 0 No 1 Yes

She leaves the house without informing him 0 No 1 Yes
She neglects her children 0 No 1 Yes
She argues with him 0 No 1 Yes
She wants to earn money independently 0 No 1 Yes
146.
2 No opinion
[COMMENTS]
15. Conclusion
became became stayed became became

much better the slightly much
better same worse worse
147. In comparison with 1. In your
the situation 1 year family…     
ago, the living
conditions… 2. In your
village…     
How? 1.
148.
2.
Which is your main 1 Radio 3  Neighbour/ friends

149. source of
information? 2 TV 4  Other
3 Newspaper  _________________
150. Household has mobile phone 1  Yes 0  No
[COMMENTS]
151.
152.
Please, could you give us Could you give us your
your first and your family telephone number?
name?
[FINAL COMMENTS / QUESTIONS BY INTERVIEWEE]
[FINAL COMMENTS BY ENUMERATOR]
153. Finishing time of interview  : h
THANK YOU VERY MUCH FOR YOUR PARTICIPATION!
6.3.2 SPV Leadership KII Protocol

This KII should be issued at minimum with the following roles (or equivalents) of SPV leadership:
1. SPV Head
2. Secretary
3. Treasurer
4. Other division heads (e.g. O&M, sales and collection, finance and administration,
environment/community officers)
Questions EQ KII Theme

What do you understand the SPV’s responsibilities to be as a whole 4 All Preparedness
with respect to [grant] and the Solar PV facility in your area? What
are the specific responsibilities of your role on the SPV?
Have you had a leadership role in your community before? If so, 4 All Preparedness,
explain what your role was. existing
organization
What is your role in your community? Do you expect conflicts with 4 ALL Preparedness
community members due to your role in the SPV?
Do you expect the SPV to be prepared better to provide sustainable 4 ALL Sustainability
energy systems in the longer term than a private enterprise? Why
(not)?
Do you expect high or low payment moral among electrified 4 ALL Sustainability,
households? What factors will be decisive for payment moral? Optimism
How would you describe your existing relationship with [grantee] to 4 All Relationship with
this point? grantee/contractors
How would you describe your existing relationship with [O&M 4 O&M Relationship with
contractor] to this point? grantee/contractors
What challenges do you anticipate will occur in your role with the SPV 4 All Preparedness,
given your knowledge of your community? [If SPV will include optimism,
cooperation among treatment units] How do you think [treatment cooperation with
units] will cooperate with one another? other villages
How would you describe your existing relationship with the other 4 All Existing
members of SPV leadership? Have you collaborated with them before organization
on community initiatives? If so, what was your relationship with them
then?
What sorts of enterprises do you anticipate will take advantage of the 2 Head Productive uses
new renewable energy resource? Do you anticipate that community
members will start new business once the micro-grid is
commissioned? If so, what kinds of businesses?
Why did you decide to pursue participating in the management of the 4 All Optimism
micro-grid?
How would you generally describe members of your community with 2, Head, Existing
respect to: 3, Community organization,
 Motivation and work ethic 4 Officer optimism, gender
Questions EQ KII Theme

 Environmental consciousness
 Community engagement
 Gender equality
How does your community generally address community-level 4 Head Existing
problems or goals? organization
What is your assessment of micro-grids as a resource for providing 4 All Optimism
electricity to communities? Have you heard of them being used in
other communities? If so, what have other communities experienced
with this technology?
[If SPV is set up already] How are responsibilities within the SPV 4 ALL Preparedness
distributed between females and males?
[IF SPV is set up already] How will responsibilities in day-to-day 4 O&M Preparedness
operation and maintenance be handed over to you? Does this
process seem reasonable to you?
[If vocational training has already commenced] How confident are you 4 All Preparedness,
that your training will prepare you for your role in the SPV? What part relationship with
or parts of your training have seemed the most useful? grantee/contractors
What might affect people in your community’s willingness to pay for 2, All Optimism
electricity? 4
Who stands to benefit the most in your community from increased 2, All Optimism,
access to electricity? 4 productive uses
Will women be affected proportionally by access to RE? 4 All Optimism, Gender
How might your SPV choose to use surplus electricity or revenue, if 2 Head, Productive uses
a surplus exists? Treasurer
Do you expect any challenges in payments or sustainability of the 2,4 All Sustainability
system?
How does your SPV plan to ensure transparent and participatory 4 Head, Sustainability
monitoring of the community? Community
Officer
How will your SPV ensure gender equality and social inclusion in 4 Head, Gender
benefits from the new RE systems? Community
Officer
How confident are you that the SPV will be prepared, in terms of 4 Head Optimism
capacity, equipment, and legal status, to operate the infrastructure
after construction has ended?
6.3.3 Village Official KII Protocol

*This protocol will be used with Village Heads (Kepala Desa) in treatment and comparison areas in East
Sumba, and in treatment areas in Berau. This protocol, in comparison to others for the qualitative component
of data collection, includes mostly closed questions with several open-ended questions.
Date:
I. Basic Sub-village Data
Name of Data collector: ________________________________
Name of sub-village: ________________________________
Site code : ________________________________
Name of interviewee : ________________________________
Role of interviewee : ________________________________
Phone number of interviewee: _______________________________
 All questions shall refer to the sub-village listed above
1. Demographic Data Sub-village

1.1. Population, male
1.2. Population, female
1.3. Population, total
1.4. Number of households, total
II. Infrastructure and Services in the sub-village
2. Availability and conditions of basic infrastructure

a. Roads: (road condition, construction work, access during rainy season)
a.1 a.2 a.3 a.4

Distance To which city does the Access to main road Can the road be
from main main road connect? (circled the appropriate travelled year-round
road (the nearest town or one) by four-wheeled
rural center) vehicles?
1. Asphalt pavement 1. Yes
....................... 2. Stone pavement 0. No
3. Earth pavement
b. Transportation:
b.1 b.2 b.3
Transport possibilities Price to reach the next urban If public transport is
in the village (circle the center (for each option circled available, how frequently
appropriate) in b.1) does it arrive per week?
1. Bus/ public transport 1. ________
2. Mototaxi 2. ________
3. Taxi 3. ________
4. Donkey cart 4. ________
5. Other, define: _______ 5. ________
c. TV, radio and mobile phone network reception:

Receivable? If YES : quality of reception?
Type of network Don’t Don’t
Yes No Good Medium Bad
know know
1. Radio 1 0 -1 1 2 3 -1
2. Mobile Phone Network 1 0 -1 1 2 3 -1
3. TV 1 0 -1 1 2 3 -1
4. Internet mobile phone 1 0 -1 1 2 3 -1
5. Internet landline 1 0 -1 1 2 3 -1
[COMMENTS]
3. Availability and conditions of social infrastructure (SI)

Type of SI Public or Private Uses electricity source (M)?
code 1. Public 1. PLTMH 2. Battery
2. Private 3. Solar panel 4. Genset
5. PLN 6. Kincir
1. Primary school (SD) [COMMENT]

2. Junior high school (SMP)
3. Senior high school (SMA)
4. Islamic boarding school (Pesantren)
5. Other school – specify
6. Community health center (Puskesmas)
7. Community health subcentre (Pustu)
8. Health service post (Posyandu)
9. Midwife house (house of bidan)
10. Traditional Healers
11. Other health structure, specify
12. Church
13. Mosque
14. Other religious building

15. Administrative office, specify
Question EQ Theme
3.1. What are the challenges health facilities frequented by this Community
1, 2
community face? details
3.2. Do you expect health service quality to be affected by access to RE? Community
1, 2
Why? details
3.3. How would you describe maternal health services in your community
Community
(consider public and private facilities, Midwifes and traditional 1, 2
details
healers)?
3.4. How many cases of maternal deaths have you had in your Community
1, 2
community in the last 12 months? details
Productive
Uses,
3.5. Do you think access to RE can improve health and wellbeing of
2 Gender,
pregnant women? How?
Community
details
Community
3.6 What are the challenges schools frequented by this village face? 1, 2
details
Productive
3.7 Do you expect school service quality to be affected by access to RE? Uses,
2
Why? Community
details
4. Availability and conditions of social infrastructure

a
Main access to water (circle the appropriate response)
1. River or lake 3. Fountain (unprotected)

2. Fountain (protected) 4. Private connection
5. Other, specify __________________________
III. Energy
Question EQ Theme
1. How do you dispose of used/empty batteries and broken energy savers? 1 Energy
(If thrown away, where?) Consumption
2. Do you know what “Renewable Energy“ is? Pease explain. 1, 4 Sustainability,
Preparedness
Question EQ Theme
a. Do you think your community should rely on RE? Explain 1, 4 Sustainability,
Preparedness
b. Do you think Renewable Energy is better for the environment than 1, 3 Environment
alternative electricity sources? Explain.
c. Do you know how to support longevity of a community mini-grid as 1, 4 Sustainability,
community member? Preparedness
IV. Income generation

3. Enterprises:
Type of business unit Number Electricity Sources of each Gender of Owner
0. None of each
1. PLTMH 0. Male,
2. Kincir 1. Female
3. Battery,
4. Genset
5. Solar panel
6. Other, specify
Kiosk /warung
Store
Carpenter
Wall-maker/ builder
Tailor
Beauty salon
Flour miller
Rice huller
Sawmill
Auto workshop
Welding workshop
4. Economic opportunities
Question EQ Theme
a. Are there economic activities in this community, which may grow in case 2 Productive
of electricity access? Uses
b. What are these activities (for example boat production, honey making, or 2 Productive
fishing)? Uses
c. Why or why not may they grow? 2 Productive
Uses
d. What are other factors hindering their growth of economic activities in 2 Productive
your community? What type of training or support may help reduce them? Uses
e. Do you expect any new or existing businesses would use the RE resource? 2 Productive
In what ways? Do you anticipate that community members will start new Uses
business once the micro-grid is commissioned? If so, what kinds of
businesses?
f. What are typical productive activities pursued by women? How could 2 Productive
economic activities of females be encouraged? Uses,
Gender
5. Quality of land in sub-village (fertility, acidity, erosion)

Fertility – majority of land 1. Very fertile 2. Fertile 3. Less fertile 4. Not fertile
Erosion 1. Often eroded 2. Seldom eroded 3. Never eroded
6. Sub-village market (held at least once per week)

Is there a market in the sub-village?
1. Yes, there is 
0. No, there is not  Where is the nearest market (distance)? _________km
V. Socio-economic issues
7. Involvement in sub-village activities:
*Include definition of organization. Should include SPV if already formed at time of interview (in
treatment sites).
Type of organization: Activity How many
1. Religious Frequency per participants
Organization 2. Non- religious Main activity month 1).0-10 2).10-
Type of organization: Activity How many

1. Religious Frequency per participants
Organization 2. Non- religious Main activity month 1).0-10 2).10-
8. How would you generally describe members of your community with respect to:
a. Motivation and work ethic
b. Environmental consciousness
c. Community engagement
d. Gender Equality
Question EQ Theme
9. How does your community generally address community-level 4 Community
problems or goals? details,
Preparedness
10. Are there development projects in the sub-village? What do 4 Community
they do? details,
Optimism
11. Have the general living conditions (particularly poverty level) in 4 Sustainability
the sub-village changed within the last 2 years? (Explain)
1. Improved significantly  2. Improved slightly 
3. Stayed constant 
4. Deteriorated slightly  5. Deteriorated significantly 
12. Why 4 Sustainability
13. What factors are hindering an improvement in living 4 Sustainability
conditions in this sub-village? (Explain)
14. Security
a. Do people in this community feel safe? 4 Community
details
b. Have there been crimes of any sort in your community in 4 Community
the last year? Please explain. details
VI. SPV (for treatment site Kepala Desa only)

Question EQ Theme
a. What do you understand the SPV’s responsibilities to be as a 4 Preparedness
whole with respect to [grant] and the Solar PV facility in your
area?
Question EQ Theme
b. If you are involved in the SPV, what are the specific 4
responsibilities of your role? If you are not involved in the SPV,
how would you describe your existing relationship with the
members of SPV leadership?
c. What challenges do you anticipate will occur with the SPV 4 Preparedness,
given your knowledge of your community? Sustainability
d. How would you describe your village’s relationship with 4 Project details,
[grantee] to this point? grantee
relationship
e. How would you describe your village’s relationship with [O&M 4 Project details,
contractor] to this point? grantee
relationship
f. What is your assessment of micro-grids as a resource for 4 Preparedness,
providing electricity to communities? Have you heard of them Sustainability
being used in other communities? If so, what have other
communities experienced with this technology?
Conclusion
g. Final comments/ questions by the interviewee NA NA
h. Final comments by enumerator
6.3.4 Regency (sub-district) Official KII Protocol
Date:
I. Basic Regency Data
Name of Data collector: ________________________________
Name of Regency: ________________________________
Site code : ________________________________
Name of interviewee : ________________________________
Role of interviewee : ________________________________
Phone number of interviewee: _______________________________
 All questions shall refer to the regency listed above
Demographic Data Regency

1. Population, total
2. Number of households, total
II. Infrastructure and Services in the Regency
3. Availability and conditions of basic infrastructure

Question EQ Theme
a. What percentage of villages in your regency are connected to Community
1, 2
roads? details
b. What are the main town centers (cities) in this regency? Community
1, 2
Please list. details
c. Please describe the majority of roads in your regency
Community
[asphalt pavement, stone pavement, earth pavement…] Can 1, 2
details
these roads be used year-round?
d. Please describe the transportation options in your regency Community
1, 2
[bus/public transport, mototaxi, taxi, donkey cart, other…] details
Question EQ Theme
e. How frequently is this transportation available for regency Community
1, 2
residents? details
f. What percentage of regency residents receive radio, mobile
Community
phone network, TV, internet mobile phone and internet 1, 2
details
landline reception?
g. Please describe access to public, private, and informal health Community
1, 2
services in your regency. details
h. How would you describe maternal health services in your
Community
community (consider public and private facilities, Midwifes 1, 2
details
and traditional healers)?
i. What are the challenges health facilities face? Community
1, 2
details
j. Do you expect health service quality to be affected by access Community
1, 2
to RE? Why? details
k. Please describe access to public and private schools in your Community
1, 2
regency. details
l. What are the challenges schools frequented by this village Community
1, 2
face? details
m. Do you expect school service quality to be affected by access Community
1, 2
to RE? Why? details
III Energy
4. Main energy sources and prices (other than electricity):
Please describe in general the energy sources in your regency overall.
Energy source Used by people?
Candles 1. Yes  0. No 
2. Only in exceptional cases 
Gas (LPG) 1. Yes  0. No 
2.Only in exceptional cases 
Diesel 1. Yes  0. No 
Petrol 1. Yes  0. No 
Kerosene 1. Yes  0. No 
Charcoal 1. Yes  0. No 
Energy source Used by people?
Firewood 1. Yes  0. No 
Batteries (large) 1. Yes  0. No 
Batteries (small) 1. Yes  0. No 
5. Electricity sources used by households in this regency (car batteries, gensets, solar
panels, PLTMH, traditional waterwheel (kincir) – individually vs. commonly used
Electricity source Approximate % of Individual or shared
households owning use?
this source
Car battery
Genset
Solar Panel
PLN*)
Biodigesters
Kincir
PLTMH
Question EQ Theme
6. Do you know what “Renewable Energy“ is? Please explain 1, 4 Sustainability,
Preparedness
7. Do you think your regency should rely on RE? Explain 1, 4 Sustainability,
Preparedness
8. Do you think Renewable Energy is better for the environment than 1, 4 Sustainability,
alternative electricity sources? Explain. Preparedness
9. What is required to ensure longevity of mini-grids in your regency 1, 4 Sustainability,
in your opinion? Preparedness
Income Generation
10. Enterprises
a. What types of enterprises are most common in this regency 2 Productive
(also inquire boat production, honey making, and fishing)? Uses
Please list.
b. If you are aware, what electricity sources do they use? 2 Productive
Uses
11. Economic Opportunities

a. What are factors hindering growth of economic activities 2 Productive
in your regency? Uses
b. What type of training or support may help reduce them? 2 Productive
Uses
c. What are typical productive activities pursued by women? 2 Productive
Uses, Gender
d. How could economic activities of women be encouraged? 2 Productive
Uses, Gender
e. Are there economic activities in this community, which 2 Productive
may grow in case of electricity access? Why or why not Uses
may they grow?
f. Do you expect any new or existing businesses would use 2 Productive
the RE resource? In what ways? Do you anticipate that Uses
community members will start new business once the
micro-grid is commissioned? If so, what kinds of
businesses?
12. Agriculture
a. Describe the quality of land in the regency (fertility, acidity, 1 Community
erosion) details
Fertility – 1. Very fertile 2. Fertile 3. Less
majority of land fertile 4. Not fertile
1. Often eroded 2. Seldom eroded
Erosion
3. Never eroded
b. What are the main agricultural commodities (Product 1 Community
pertanian unggulan) in the regency? details
c. What type of value-addition and aggro processing is 1 Community
performed in the regency? details
d. Are there any particularities in agriculture in your regency? 1 Community
details
13. Security
a. What type of security problems do the communities in your 1 Community
Regency face? details
b. Are security problems increasing or decreasing? 1 Community
details
c. What is needed to improve security in your Regency? 1 Community
details
d. Do you expect access to RE to improve security? 1 Community
details
14. How would you generally describe members of your community 3, 4 Community
with respect to: organization,
a. Motivation and work ethic Preparedness,
b. Environmental consciousness Environment
c. Community engagement
d. Gender equality
15. How does your community generally address community-level 4 Preparedness,
problems or goals? Sustainability
16. Are there development projects in the regency? What do they do? 4 Preparedness,
Optimism
17. Have the general living conditions (particularly poverty level) in 4 Sustainability
the regency changed within the last 2 years? (Explain)
1. Improved significantly  2. Improved slightly 
3. Stayed constant 
4. Deteriorated slightly  5. Deteriorated significantly 
18. Why? 4 Sustainability
19. What factors are hindering an improvement in living conditions in 4 Sustainability
this regency? (Explain)
20. Are there other particularities to note in the regency? NA NA
SPV (for treatment site Camat only)
1. What do you understand the SPV’s responsibilities to be as a 4 Preparedness
whole with respect to [grant] and the Solar PV facility in your area?
2. If you are involved in the SPV, what are the specific responsibilities 4 Preparedness
of your role? If you are not involved in the SPV, how would you
describe your existing relationship with the members of SPV
leadership?
3. What challenges do you anticipate will occur with the SPV given 4 Preparedness,
your knowledge of your sub-district? Sustainability
4. What is your assessment of micro-grids as a resource for providing 4 Preparedness
electricity to communities? Have you heard of them being used in
other communities? If so, what have other communities
experienced with this technology?
5. Final comments/ Questions by the interviewee NA NA
6. Final comments by enumerator NA NA
6.3.5 Grantee KII Protocol

**Note: Some questions in this protocol may be skipped based on the informant’s role in the implementing
organization, or the implementing organization’s role in an implementing consortium. For example,
engineering firms will not be asked about community engagement plans for SPVs, unless they are somehow
involved.
Question EQ Theme
1. Please describe your role on this grant. 4 NA
2. Please describe the grant’s work in [village name; 4 Project

regency name]. Details,
a. When did you begin work, and with what Preparedness
activities?
b. What stage of implementation are you currently
in?
c. What is the ultimate goal of your work, and when
will it be completed?
3. Please describe your progress with licensing and 4 Project

permitting for the development of the Solar PV Facility. Is Details,
this going according to schedule? If no, how will this Preparedness
impact your work? How has this impacted your
engagement work with the community?
4. Please describe your project’s stakeholder engagement 4 Project

plan in this village/regency. Details,
Preparedness
5. Have you implemented any of this plan yet (for example, 4 Project
have you met with the Village Head, village leaders, held Details,
FGDs, etc)? If yes, what has been the community’s Preparedness
response/level of engagement?
6. Have you facilitated the formation of the SPV yet in this 4 Project
village/regency? If yes, what steps have been completed Details,
(have members been selected)? If no, when will you do Preparedness
so?
7. Please describe the business plan you are pursuing in 4 Project

creating the SPV(s) for this project. Who will comprise the Details,
SPV? How will you attempt to gain community buy in and Preparedness
build the community’s capacity to participate? What will
the governance process be in terms of managing cash
Question EQ Theme
flow and assets? Will dividends be reinvested in the
community and, if so, how?
8. Have you finalized the selling price for electricity in each 1 Energy
village/kampung? Please describe how you arrived at this Consumption
figure and, if it has changed, why.
9. How did you decide on the current business and 4 Project

community engagement plans with respect to the SPV(s)? Details,
If these were based on similar models employed by your Preparedness
organization or others in other projects, please describe
how you learned about them.
10. How do you plan to transition ownership to the SPV after 4 Preparedness,
construction has been completed? Sustainability
11. What do you believe the likelihood to be that PLN will 1 and 4 Preparedness
expand into the villages/kampungs targeted by this grant
in the near term? Describe how the SPV may mitigate
this, if it came to fruition.
12. Do you plan to tap into additional resources besides those 4 Preparedness,
provided by MCA-I to ensure the sustainability of the Sustainability
project? If so, what do you plan to do?
13. At this stage of implementation, what are the main 4 (though Preparedness,
challenges you see to the Solar PV Facility development? potentially Sustainability
(ask about SPV leadership and role, if not mentioned) all EQs)
What challenges do you expect in terms of sustainability?
14. In each of the targeted areas, what do you expect will be 1-4 Preparedness,
the main outcomes from your project? Do you expect new Sustainability
or expanded businesses? If so, what kinds? How long do
you expect it would take before these businesses are
developed or expanded?
6.3.6 Community Beneficiary FGD Guide

Question EQ Theme
Energy
1. What type(s) of electricity source(s) do you use in your 1 Energy
homes? [options to probe for are in HH survey Q.37. Note how Consumption
many respondents have a modern electricity source.]
2. How long have you used these sources? 1 Energy

Consumption
3. How much does this source(s) cost you/your family (per 1 Energy
month, per year)? Consumption
4. Please explain if there are sources you have been 1 Energy

disconnected from or have become non-functional. When did Consumption
this occur, and why?
5. Please discuss challenges you face with accessing electricity 1 Energy

in this village. Does this differ by HH/area in the village? Consumption
6. What is your HH main use for electricity (appliances, lighting, 1, 2 Energy

productive uses)? Consumption,
Productive
Uses
7. Are you satisfied with the source(s) of electricity your family 1 Energy
uses currently? What are the main advantages/disadvantages Consumption
of this source(s)? Please discuss.
8. Please discuss what you think of micro-grids as a resource for 1 Energy

providing electricity to communities - Have you heard of them Consumption
being used in other communities? If so, what have other
communities experienced with this technology?
9. Would you prefer other types of electricity? What and why? 1 Energy
Consumption
10. Do you think electricity access can bring growth in economic 2 Productive
activities? How? Uses
11. What else is needed in your community to raise economic 1 Energy

wellbeing? Consumption
Question EQ Theme
12. Do you think households should pay for energy from a RE 1 Energy
mini-grid? What would be the best billing system? Why? Consumption
Equality, Gender, Security

1. Who would benefit most from energy access in these 1 Energy
communities? Consumption,
Gender
2. Do you think female community members will be affected equally 1 Energy
by electricity access as male members? How will electricity access Consumption,
change the life of women, and their rights and roles within the Gender
community?
3. Do you think electricity access can affect security in your 1 Energy
community? Please discuss. Consumption
4. What do you think about mini-grids that are managed by a team of 4 Preparedness,
community members? How will such a management system affect Community
payment morale within the community? How will it affect dynamics Organization
between community members?
Environment
1. Which environmental issue concerns this community the 3 GHG
most? Why? Emissions
Community (engagement and work ethic)

1. Please discuss the main source of income for HH in this 1, 2, 4 Preparedness
community. In your opinion, do individuals have a strong work
ethic in this village (do they work hard)?
2. Please discuss whether your community/village has other 4 Preparedness,

groups/organizations like the SPV. How have these worked, Optimism
and were they successful at managing a community good?
How many of you have participated in a community
group/organization/initiative?
3. How does your community generally address community-level 4 Preparedness

problems or goals?
4. Please discuss your village’s previous experiences with donor 4 Preparedness,

projects, if any. Did you consider these projects a success? Optimism
Why or why not?
Question EQ Theme
Project Details
1. [Project name/grantee name] is working in this village to NA Project Details
develop a Solar PV Facility. Please discuss the work they
have done thus far.
2. Please discuss how you have interacted with [project NA Project Details
name/grantee name] in the last 3 months. Have you attended
any meetings/FGDs/events/activities or received information
about the project goal? If yes, please discuss the purpose of
these events and how you were invited.
3. If you are you aware of the SPV in this village, please discuss 4 Preparedness
their role/function as related to the Solar PV Facility.
Conclusion
1. In comparison with the situation 2 years ago, have the living 1, 4 Sustainability
conditions in this village improved? If yes, how? If not, why not?
6.3.7 Enterprise KII Guide
ENTERPRISE QUESTIONNAIRE 1 Date: _________
SUB-VILLAGE NAME
SUB-VILLAGE SITE
INTERVIEWEE/ENTERPRISE NAME
Impact Evaluation Baseline Study 2017 MALE/FEMALE
OWNER OR MANAGER/STAFF EMPLOYEE
Green Prosperity Renewable Energy Grant
INTERVIEWER NAME
STARTING TIME:
A. Basic Information and Customers

Q1. Line of business
Q2. Enterprise age

Q3. Type of electricity available
Q4. Since when is it available?
Q5. In case of solar panel, what’s the size of the panel (kW)?
None
1
Connection to a MHP
2 Since when (Month, Year) __________________________
Car battery (without solar

3 panel) Since when (Month, Year) __________________________
Solar panel (installed on

4 roof) Since when (Month, Year) __________________________ kW of solar panel_________________
Solar panel (not installed

5 on roof) Since when (Month, Year) __________________________
Individual genset
6 Since when (Month, Year) __________________________ kW of solar panel_________________
Genset in the village
Genset shared with

8 neighbors Since when (Month, Year) __________________________
PLN
Individual traditional
10 waterwheel Since when (Month, Year) __________________________
Traditional waterwheel in

11 village Since when (Month, Year) __________________________
Q6. Kind of products and services offered by the enterprise (USE CODES) organize
price
Q7. Price per piece or unit (define) hierarchically

1

2

3

4

5

6
CODE of Q.13
1. Sale of small products (for example cigarettes,
11. Rice hulling
batteries, petrol)
12. Coffee milling
2. Food or Drinks
13. Coffee procession
3. Cupboard
14. Coconut milling
4. Tables
15. Baking
5. Chairs
16. Metal products
6. Bedsteads
17. Welding products
7. Window and door frames
18. Woven products
8. Doors
19. Hair cutting
9. New clothing
20. Wedding styling
10. Cloth repair and alteration
21. Make-up
This sub-village ___percent; This village ___percent; Other villages ____percent;

Q8. Structure of customers Traders ____percent | Others ______percent Next city _____percent [Specify]
_______________
Number of Customers (supplied) per day: _________
B. ENERGY AND PRODUCTION
Q10. What powers the appliance?
Q9. Which of the following appliances does this a) Electricity
Appliance b) Diesel/Petrol
enterprise use?
c) Mechanic
d) Other, define.
1 Lighting
2 Sewing machine
3 Refrigerator
4 Rice cooker
5 Carpentry equipment
6 Brush
7 Coconut grinder
8 Chili grinding machine
9 Blender
10 Mill
11 Other:
12 Other:
13 Other:
14 Other:
Q11.
Which of the following energy Q12. Q13.
sources does this enterprise use for In a regular month, how much does this
For which of the following
its production process (including enterprise spend on …?
purposes do you use...[use Codes
lighting)? Multiple entries are from Q1. or define]?
possible.
Operating equipment
Lighting
ENERGY SOURCE Reg

SPECIFY
1 PLTMH
2 Diesel/petrol for generator Litre
3 Kerosene Litre
4 Candles
5 Gas (LPG / LNG)
6 Charcoal / briquettes
7 Firewood
8 Car or other rechargeable battery
9 Solar Panel
10 Other:
C. LIGHTING
Q14. Operation time of enterprise on regular day?
Q15.
Q16.
How many of the following lighting devices
What is the number of hours you use lighting per day?
does this enterprise use?
ENERGY SAVER
INCANDESCENT BULB (ORDINARY BULB)
FLUORESCENT TUBE (NEON)
TIN LAMP (KEROSENE)
HURRICANE LANTERN
CANDLE
BATTERY-RUN LANTERN
GAS LAMP (PRESSURIZED)
Other (specify):
D. EMPLOYMENT
Q17.
How many employees does this
enterprise have in total (including
owner)
Q18.
How many of the employees work
the more than 7 hours per day on 5
days?
Q19.
How many of the employees receive
payment?
Q20.
How many of the employees are
family members?
Q21. Discuss how micro-grid access could change use of labor
E. PRODUCTION AND BUSINESS EXPANSION
Q22. Would you purchase machinery/appliances in case of electrification?
Q23. In case, new machinery/appliances were/would be purchased, why didn’t you/don’t you buy a generator to
run machinery appliances?
Q24. Why don’t you produce more of products you produce? (Bottlenecks…)
Q25. In case of bottlenecks, what is needed to overcome them?
Q26. Do you think that access to a micro-grid would help to overcome these obstacles?
Q27. Do you think micro-grid connection could change your production and prices? If yes, how?
Q28. In your opinion, if you were able to produce/offer more of your product – through, for instance, longer hours,
better equipment, more workers - would there be sufficient demand for the additional products?
Q29. Are you currently in a high/low demand period compared to the rest of the year?
SOCIAL IMPACT, INC.

2300 CLARENDON BLVD. SUITE 1000
ARLINGTON, VA 22201

Designreport Idn GP Cbog Feb18

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Designreport Idn GP Cbog Feb18

Diunggah oleh

Hak Cipta:

Format Tersedia

Evaluation Design Report

MCC Indonesia Green Prosperity Project

MCC Indonesia Green Prosperity Project

Submitted: August 2017

2 Overview of Compact & Interventions .......................................................................... 11

3 Evaluation design ........................................................................................................... 45

3.4 Qualitative Approach ...................................................................................................................... 63

4 Administrative ................................................................................................................. 72

5 References ....................................................................................................................... 75

6 Annexes ........................................................................................................................... 80

6.3.6 Community Beneficiary FGD Guide ...................................................................................... 153

TABLES & FIGURES

AEI Akuo Energy Indonesia

APDS Asosiasi Periau Danau Sentarum

ARI Acute respiratory infection

BPS Badan Pusat Statistik

BUMD Badan Usaha Milik Daerah

BUMDes Badan Usaha Milik Desa

CBA Cost benefit analysis

CBOG Community-Based Off Grid

CEM Coarsened exact matching

CEO Chief Executive Officer

CIA Conditional Independence Assumption

CIFOR Center for International Forestry Research

CoK Center of Knowledge

CPI Charta Putra Indonesia

CUKK Koperasi Kredit Keling Kuman

DFS Detailed Feasibility Study

DiD Difference in differences

DRB Disclosure Review Board

EDR Evaluation Design Report

EPC Engineering, Procurement and Construction

ERR Economic Rate of Return

ESMP Environmental and Social Management Plan

EVI Electric Vine Industries

FEED Front End Engineering Design

FGD Focus Group Discussion

GEF Global Environment Facility

GHG Greenhouse gas

GIS Geographic Information System

GOI Government of Indonesia

GSK Green Sumba Consortium

IBEKA Inisiatif Bisnis dan Ekonomi Kerakyatan

ICC Intra-Cluster Correlation

IEA International Energy Agency

IIEE Indonesian Institute for Energy Economics

IRB Institutional Review Board

ISEAS Yusof Ishak Institute

KEN Kebijakan Energi Nasional/ National Energy Policy

KII Key Informant Interview

KKI Warsi Komunitas Konservasi Indonesia WARSI

KSM Kelompok Swakelola Masyarakat

kWp Kilowatt peak value

LATIN Yayasan Lembaga Alam Tropika Indonesia

LED Light Emitting Diode

LLPSLH Lingkungan Hidup

M&E Monitoring and Evaluation

MCA-I Millennium Challenge Account- Indonesia

MDES Minimum detectable effect size

MHP Micro Hydro Power

MKS PT Mikro Kisi Sumba

NRE New and Renewable Energy

NRM Natural Resource Management