February 2013
Table of Content
Abbreviation......................................................................................................................3
Executive summary.......................................................................................................4
1.0. Introduction.............................................................................................................9
2.0: Methodological Approach................................................................................9
2.1: Evaluation Framework.............................................................................................9
2.2: Study Area...................................................................................................................11
2.3: Data Collection..........................................................................................................11
2.3.1 Literature Review...................................................................................................11
2.3.2 Key Informant Interview......................................................................................11
2.3.3 Focus Group Discussion.......................................................................................12
2.3.4 Observation..............................................................................................................13
2.3.5 Challenges during evaluation............................................................................13
3.0: Evaluation Findings...........................................................................................13
3.1: Performance of the Project....................................................................................13
3.1.1: Relevance of the Objectives.............................................................................13
3.1.2: Effectiveness of the Project..............................................................................16
3.1.3: Efficiency of the Project......................................................................................20
3.2: Impact of the Project...............................................................................................21
3.2.1Overall impact of the SATF/FCS funded project..........................................21
3.2.2 Sustainability of the Projects.............................................................................39
3.2.3Lessons Learnt.........................................................................................................40
3.2.4Challenges.................................................................................................................41
3.3 Performance of the Partners..................................................................................41
3.3.1Performance of FCS................................................................................................41
3.3.2Performance of SATF..............................................................................................42
3.3.3 Performance of NGO Implementing Partners..............................................43
4.0 Discussion.................................................................................................................43
5.0 Recommendations...............................................................................................44
Annexes..............................................................................................................................46
Annex 1: List of Respondents Iinterviewed at Partners Level...........................46
Annex 2: List of Consultants and Data Collectors involved in the
assignment...........................................................................................................................47
Annex 3: Data Collection Tool.......................................................................................48
Abbreviation
AIDS
Acquired Immunodeficiency Syndrome
CEO
Chief Executive Officer
FCS
Foundation for Civil Society
HIV
Human Immunodeficiency Virus
KARADEA Karagwe Development Association
LISAWE
Lindi Support Agency for Welfare
MKUKI
Mkakati wa Kubidhibiti UKIMWI Kilimanjaro
MKUKUTA Mkakati wa Kukuza Uchumi na Kupunguza Umaskini Tanzania
MOU
Memorandum of Understanding
MVC
Most Vulnerable Children
NEREFO
Neema Resource Foundation
NCPA
National Costed Plan of Action
NGO
Non Government Organization
NSGRP
National Strategy for Growth and Reduction of Poverty
PADI
Tanzania Mission to the Poor and Disabled
PSS
Psychosocial Support
SATF
Social Action Trust Fund
SADC
Southern Africa Development Cooperation
TOT
Training of Trainers
VSI
Vijana Simama Imara (Youth stand firm model)
Executive summary
The Rolling out of PSS support capacity building project in 13 Regions of
Tanzania was built on the lessons learnt from the PSS project that was
funded by FCS in 2007. The lessons learnt is the result of the findings from
the monitoring and evaluation which indicated that majority of caregivers
at the community level did not have the knowledge on how to provide PSS
services to MVCs. Therefore, the project aimed at building the capacity of
caregivers through partner NGOs providing PSS to support Orphans and
vulnerable children.
The overall objective of the assignment was to assess the impact of
project activities from project inception to completion. The
evaluation team employed the outcome model which assisted
in capturing different project elements which included:
Relevance, Efficiency, Effectiveness, Impact, Sustainability,
Lessons Learned and Documentation of Success Stories. The
evaluation also assessed the achievements and impacts
through different three program areas which included;
performance of the program, Impact of the Program and
Performance of the Partners.
Different methods were also employed to collect the data from
respondents which included literature review, key informant
interview, FGD guide, and Observation. In summary, the
evaluation activities involved review of relevant documents
such as narrative reports, review of national policies and
strategies, meeting with FCS and SATF officials, field data
collection in six districts, data analysis and report writing.
Findings from the evaluation
Relevance of the project:
SATF-PSS project aimed at building the capacity of Implementing partners,
caregivers on how best they can support Most Vulnerable Children who
were exposed to the devastating effects of poverty, conflicts and HIV and
AIDS and were in need of care for their emotional, intellectual, physical,
spiritual and social well-being. The aim of the project aligned with several
national policies, laws, strategies and project that focusing on addressing
the needs and priority of children and their linkages to national
development. These includes; The Law of the Child Act, 2009, The child
Development Policy, 1996 as revised in 2008, The Tanzania Development
Vision 2025 and The education and Training Policy, 1995. The project also
aligns with the National Strategy for Growth and Reduction of Poverty
(NSGRP), The National Costed Plan of Action for MVC, 2012-2016, The
National Costed Plan of Action for MVC, 2007-2010 and Tanzania Health
Policy, 1990.
Effectiveness of the project: Overall achievement on project outcomes
is very good; the project managed to perform remarkably in almost all
areas as explained in further detail in the main body of the report in the
progress towards log frame indicator page 16-18
4
1.0. Introduction
About Rolling out PSS Capacity Building Project
The Rolling out of PSS capacity building project in 13 Regions of Tanzania
was built on the lessons learnt from the PSS project that was funded by
Foundation for Civil Society (FCS) in 2007/08. The project aim was to build
the capacity of partner NGOs, district officials and caregivers to provide
PSS to Orphans and Vulnerable Children (OVC) that were supported under
the SATF MVC education Programme.
However, during the 2007/08 monitoring and evaluation of the project,
SATF and its implementing partners learnt that, despite the great need of
providing PSS to OVC most Caregivers at the community level did not
know the PSS needs faced by children and how to help them cope with the
PSS problems. Also caregivers believed that material support was most
important for OVC and often did not care about their psychological
wellbeing.
On the other hand during evaluation they found that, despite the training
provided to partner NGOs, they had not been able to adequately provide
PSS to OVC as they were yet to mainstream the PSS component into their
policies and plans.
Therefore taking into account the lessons learnt from the previous project
implementation, SATF with funding support from FCS, rolled out a three
years (2009-2012) PSS capacity building project at District level to ensure
its sustainability. The objective of the project was to at least reach 52
members from 26 NGOs with TOT training and train 520 Caregivers at
community level. Also at District level to sensitize at least 260 District
Officials from departments namely: Community development, Health and
Social welfare, Education, Police gender desk, on the importance of
mainstreaming Psychosocial Support in their District plans. The other aim
of the project was to ensure that all 26 Implementing Partners are
supported to mainstream PSS issues into their support. The project also
adapted the Vijana Simama Imara model as one of the initiative to
empower young people to generate income.
2.0: Methodological Approach
2.1: Evaluation Framework
The programmes relevance, efficiency, effectiveness, impact and
sustainability were assessed using the outcomes model depicted below
Baseline
Information
OUTPUTS
Objectives
OUTCOMES
Human Resources
Financial
Resources
10
Output Indicators
MEASUREMENT
Outcome
Indicators
PSS project, assess the aspects of the project which are most and least
sustainable, and finally assess if the project has Potential Avenue for
continuity after withdrawal of the donor support.
2.2: Study Area
The evaluation was conducted in PSS Project catchment areas in six
districts which include Dodoma Municipal Council, Moshi rural District
council, Lindi Municipal council, Songea Municipal council, , Karagwe
District council and Morogoro rural district council. In each district, one
implementing partner was involved in the evaluation. These include
TIKVAH in Dodoma, MKUKI in Moshi Rural, LISAWE in Lindi Urban, PADI in
Songea, KARADEA in Kagera and NEREFO in Morogoro.
2.3: Data Collection
2.3.1 Literature Review
Various literatures were reviewed to form the base for argument on the
relevance, efficiency, effectiveness, impact and sustainability of the
project and the documents reviewed included but not limited to: project
proposal, project reports for 2012, 2011, and 2009 respectively. National
documents were reviewed as well including Laws, policies, and strategies.
These are: The Law of the Child Act, 2009, the child Development Policy,
1996 as revised in 2008, The Tanzania Development Vision 2025 and the
education and Training Policy, 1995. Other documents reviewed were the
National Strategy for Growth and Reduction of Poverty (NSGRP), The
National Costed Plan of Action for MVCI, 2007-2010, The National Costed
Plan of Action for MVC II, 2013-2017 and the Tanzania Health Policy, 1990.
Also the evaluation team reviewed National and International reports,
guidelines and conventions to support the argument. These included The
Tanzania Human Rights Report, 2008, National Guidelines for Improving
Quality of Care, Support and Protection for Most Vulnerable Children in
Tanzania,2009, UN Convention on child Rights, 2009 and the African
Charter on the Rights and Welfare of the Child 1990.
2.3.2 Key Informant Interview
A total of 95 key informant interviews were conducted with District
officials, that is Heads of community development, social welfare,
education, Health departments and Gender Desk in the Police Force.
Other respondents included NGOs heads, Training of Trainers (PSS TOTs)
participants, and Caregivers who received PSS training from
implementing partners. The Information obtained from the key informant
interview was supplemented with the quantitative data, particularly on
how the project was implemented and results obtained, which were
collected from official files and reports.
Participants
FCS
SATF staff
REPSSI
District Officials
Partner NGOs
TOT
Caregivers
Total
Source: Field data
KI Informants
Male
Female
1
3
1
16
2
2
15
40
Total
1
2
1
19
5
7
20
55
2
5
2
35
7
9
35
95
NGO
Male
1
2
3
4
5
6
Total
Source:
TIKVAH
MKUKI
PADI
LISAWE
NEREFO
KARADEA
FGD
Female
7
12
4
14
10
11
58
TOTAL
14
10
11
11
15
14
75
21
22
15
25
25
25
133
Field data
13
NGO
Male
1
2
3
4
5
6
Total
Source:
TIKVAH
MKUKI
PADI
LISAWE
NEREFO
KARADEA
FGD
Female
0
5
0
0
1
9
0
4
0
0
0
0
1
18
TOTAL
5
0
10
4
0
0
19
Field data
2.3.4 Observation
The team observed if there were any visible information (i.e. Leaflets,
brochures, outdoor banners/posters and T-shirts etc.) done by SATF
Implementing partners with regards to its objectives of addressing PSS
services . This was done during visiting to the Implementing partners
NGOs, community groups and in schools where MVC are receiving their
education services.
2.3.5 Challenges during evaluation
Busy schedule of some of district officials due to district budget
preparation
Some of district official who were trained on PSS had retired from Public
Services e.g. three officials in Karagwe Districts
Some of district official who were trained on PSS were on study leave
and some were on leave
Resistance to participate in the interview to some of MVCs
parents/guardian who received PSS support from caregivers demanding
to be paid before being engaged in the interview.
Some of MVCs supported were not available at the time of evaluation
assignment
17
the Objectively
verifiable
activity
indicators
of
achievement as indicated in
the project (in 2008)
Contribute to the improvement of wellbeing of OVC
OVC Children in 13 Regions
To build the Capacity 52 Staff from Implementing
of
Implementing partner Organization trained
partners in order to as TOT
to support the
improve their work implementation
of
PSS
performance
in services at the grassroots
providing
PSS level
services
Exchange
visit
for
knowledge sharing, gaining
experience and adopting the
VSI model to 34 members of
partner NGOs to Humuliza
hence mainstreamed into
their routine activities
Remarks
children
districts
520 OVC caregivers at
grassroots level enabled to
identify
MVCs
to
be
supported in their areas
26
Districts
Councils
sensitized
on
the
importance
of
mainstreaming PSS in their
district plans
260 Staff from departments
namely:
Community
Development, Health and
supporting
implementing
partners to increase the
number of caregivers
520 caregivers identified 2600 Target reached.
MVC (Each caregiver identified 5
MVC in her/his vicinity.
There are more MVCs in the
community who still need
more PSS support. Therefore
SATF PSS project should plan
on
how
to
continue
supporting
partners
to
produce
more
caregivers
who will support more MVCs
2600
MVCs
identified
and Target reached
supported
365 MVCs identified and supported
(MKUKI 20, WOY 25, UHAI 100,
MEDI 20, USHIVIMWA 30, AOPN 20,
TAHEA-Shinyanga 25, KIDI 80, HWF
20 and KARADEA 30)
20
21
SATF PSS Project central focus was to ensure children wellbeing at family
levels are well observed and taken care of by their parents or guardians.
Therefore, the PSS capacity building training was meant to enhance
caregivers to understand MVCs psychological problems and therefore offer
MVCs with a better assistance at both individual and family level.
During evaluation, respondents were asked whether the PSS Project have
changed their behaviors relating to how they can care for their children by
observing the children rights and needs and as well as those children they
were supporting. It is evident that Majority (Over 85%) of respondents at all
level that is district officials, NGOs, TOTs and Caregivers commended that to
the great extent the PSS training they have received have changed their
behaviors at individual level. The histogram below indicates the percentage
of response from different groups on how they recognized PSS training in
changing their behaviors
and skills on how to care for her family. She also commended that despite
being the chairperson for the Organization, she is also caring for orphans at
her family. Therefore the PSS training was provided on the right time as she
is now an expert on caring for orphans.
Elimu hii ya PSS imesaidia kuongeza uelewa wangu wa
kuwasaidia watoto. Japokuwa mimi ni mwenyekiti katika shirika,
pia nalea watoto yatima pale nyumbani kwangu, na sasa najua
jinsi ya kuwalea vizuri kwa kuwapa mahitaji yote ya msingi na
najua jinsi ya kuwafanya wajisikie sehemu ya jamii. TIKVAH
Dodoma
This means, the PSS education has improved my knowledge in
terms of providing better services to my children. Despite being
a chairperson for TIKVAH Organization am also caring for
orphans in my family. Therefore the PSS knowledge have made
me not only to provide better services but also to make them
feel part and parcel of the community
Furthermore, other respondents applauded the effort made by SATF PSS
Project and the changes the project brought to staff and the general
community. He said that before the training he was in the opinion that no any
MVC support that can be provided without having money. But after the
training he realized that there are other programmes and services that can
be provided even without incurring any cost.
Kabla ya mradi wa PSS, hapa Lisawe kama shirika tulikuwa
tunatoa huduma bila kujua kama kuna baadhi ya huduma
ambazo zinaweza kutolewa bila ya hata kuingia gharama. Lakini
baada ya elimu, tumeweza kuona kuwa ni jambo
linalowezekana na tumeweza kufanikiwa Chairperson Lisawe
Lindi
Before PSS project, LISAWE as an organization were providing
support to MVC without knowing that there is another support
which we can offer without incurring money, but after training
we have managed to do that and we have succeeded
The finding is also supported by other respondents who gave the testimony
that the training has improved the organization working relations with other
partners, MVC who are supported as well as the community in general.
Despite improving in the working relationship with other partners, the
knowledge obtained from the training helped in improving the service
delivery. As before the project they used to believe that providing material
28
support only satisfies the needs of MVC. But after the training they realized
that material support not only satisfy the needs and priority of the child
rather they need also to be provided with other psychological support.
Hapo awali kabla ya kupata elimu tulihisi vitu tunavyopewa ndio
kitu cha muhimu kwa mtoto. Kumbe elimu ya saikolojia kwa
mtoto ni muhimu zaidi kwa sababu inamjenga zaidi kuliko kumpa
material thing pekee. Baada ya elimu tukaelimika na sasa
tunatoa pia elimu ya PSS kwa watoto, jambo lilichangia kuongeza
uelewa na kujiamini kwa watoto. Ahsante sana kwa SATF na
mradi wao. Chairperson, KARADEA Karagwe
At the beginning we believed that material things are the only
thing which satisfies the needs of a child. But psychosocial
education is more important to the development of a child than
providing only material support. After attending the training we
have improved our services to PSS to children and this improved
childs knowledge and confidence.
c) Improving the knowledge and skill of Caregivers
During the evaluation, caregivers interviewed were asked if real the PSS
training they have received have improved their knowledge in providing PSS
to MVCs in their surrounding areas. It is evident from the evaluation that
SATF PSS project objective of building the capacity of caregivers through
training was well implemented as majority (97.1%) of caregivers interviewed
commended the content and techniques used during training. Only 2.9%
were not satisfied with the way the programme was implemented. 97.1%
also commended that the knowledge they have acquired from the training
have helped them to first be in a position to know and identify themselves
who they are and how they can care for their children. Also they are
knowledgeable on the criteria they can use to select MVCs to be supported
as well as know the problems facing children and the type of the services
they should provide to them. For those who were not satisfied were asked
why they are not satisfied. They responded that they expected to see
material support as one of the contribution
meeting and received PSS knowledge. One of the question they were asked
was if they ever heard about PSS project. It is evident that majority of
respondents who responded to this question said that they are aware of the
PSS . When further asked , if the PSS knowledge they received have any
added value to their lives and daily undertakings, majority of them reported
that the knowledge they have acquired from the training real changed their
lives
One of the respondent supported the statement and also applaud the efforts
made by SATF through the PSS project to build the capacity of individual and
enhance them to provide PSS services at their family level but as well MVCs
and the general community. She said;
Nawashukuru sana SATF kupitia programu yao ya PSS, kwanza
imeniongezea uelewa wa jinsi ya kuwalea watoto wangu. Ni
elimu muhimu kama wanajamii wote, wangeweza kuipata
natumai tusingekuwa na kizazi cha watoto walio katika
mazingira magumu Education officer for secondary schoolsDodoma
This means Thanks for the SATF PSS Project, as through the
training, I have gained the knoweldge on how to care for my
children. The PSS Programme is an important component if well
disseminated to all community members, then we could not
have the MVCs generation in the community
She also commended that through the PSS knowledge, she is now able to
include the PSS issues in the departmental plans and budget. She said
Kupitia mafunzo haya ya PSS nimeweza kujua umuhimu wa
kuhuisha maswala ya PSS kwenye mipango na bajeti yetu ya
idara ya elimu kwa upande wa secondary
This means that; Through PSS advocacy meeting, I now know
how to mainstream PSS issues in the departmental plans and
budget for the secondary section
Another respondent also supported the statement that PSS is an important
component in the provision of comprehensive services to MVC. Therefore the
need to incorporate this component into our plans, budgets and
implementation of activities is paramount. He said he gained more
knowledge on the important components and how to incorporate them in the
departmental plans. He said
33
and ensure well-being of these MVCs, it is evident that PSS is not a standing
alone programme. It needs to be supported with other programmes in order
to improve the wellbeing of these MVCs.
WHO, 2009, defined psychosocial wellbeing as a state of complete physical,
mental and social well-being and not merely the absence of the disease or
infirmity. WHO continued defining that psychosocial well being of an
individual or community is best interpreted with respect of three core
domains which includes; Human capacity which refers to physical and mental
health of an individuals knowledge, capacity and skills; Social ecology which
refers to social connections and support including relationship, social
networks and support systems of individual and the community and finally
the culture and values which refers to cultural norms and behaviors that are
linked to the value systems in each society together with individual and
social expectations. Therefore, psychosocial well being depends on the
capacity to draw on resources from these core domains in response to the
challenge of experienced events and conditions.
Also SADC, 2011 defined psychosocial well-being as an integral part of a
comprehensive service delivery for children and youth. It is influenced by all
the factors that affect human and social development which are: the
material, cognitive, emotional, cultural and spiritual aspects of an individual,
and the socio-cultural, economic and political environment in which people
live. Therefore, availability of psychosocial well-being enables vulnerable
children and youth to: make appropriate decisions that benefit their
development in the short and long term; assume and maintain social
responsibility and healthy social relationships and behaviors; and maintain a
condition of mental capacity and a reduction or absence of temporary or
long-term mental impairment. SADC concluded that psychosocial support
should not be a standing alone services rather it should be provided in
combination with any other services that are required by children and youth.
The National Guideline for improving quality of care, support and protection
for Most Vulnerable Children in Tanzania, 2009 identified eight key
components/services for addressing the needs of MVCs. These include food
and nutrition, shelter, family-based care and support, social protection and
security, primary health care, psychosocial care and support, education and
vocational training and household economic strengthening. When all these
needs are met then satisfies the wellbeing of MVC. SATF PSS Project
objectives focused only on one component the psychosocial support.
During evaluation, children were asked to state whether they have received
any support from caregivers. The data from the evaluation revealed that
35
68.4% had received the support from caregivers and 31.6% reported that
they have not received any support from caregivers.
Figure 3: MVC received support from caregivers
For those who received the support from caregivers, they were then asked to
state how the support they have received have helped in changing their wellbeing, they responded that real the support they have received from
caregivers helped some of the children to change their bullying behaviors
they used to have before they received the counseling support from
caregivers. Others were able to respond that they have experienced changes
in their families as of the time when the evaluation was conducted their
families were taking good care of them. Some of the testimonies from
children were as;
Nashukuru sana kupata huduma ya ushauri toka kwa Kaka
Barton(Cargiver) amenisaidia sana kubadili tabia na sasa naona
naweza kusoma vizuri na nafanya vizuri darasani.MVC Mpunguzi
Dodoma
Thanks for the counseling services from Mr. Barton. He helped to
change my behaviors and I am now rolled back to school and am
performing well in class.
Another MVC supported the achievement from support they acquired from
caregivers that it has helped her with her family as before their parents used
to treat them badly but after meeting a caregiver then they have changed
and they even engage her in family matters. She said
Naona kuna mabadiliko makubwa sana katika familia yangu.
Kabla Yule mlezi hajaja mama alikuwa akinichapa sana hata
kama sijafanya kosa, hadi ikafikia hatua nikachukia kukaa
nyumba. Lakini baada ya kuwapa ushauri wakaona umuhimu wa
kukaa na mimi na kunishauri ninapokosea. Sasa naona mambo
yamekuwa mazuri.MVC Karagwe
This means I have seen changes within my family. Before
caregiver came to my home, my mother used to slap me every
time even when I have not done mistake. This resulted for me
stay away from home for quite sometimes. After my family got
counseling services from the caregiver, they have now changed
and they have seen the importance of engaging me in decision
relating to family matters. Things now better so better than it
was before.
The other respondent (MVC) Interviewed reported that she is applauding the
counseling support she received on psychosocial. As when she was attending
her primary education, her life was full of discouragement and punishment
as she was raised with her step mother. So the mother was treating her
37
miserably. But she received the counseling support from the caregiver which
helped her to tolerate any punishment from her step mother. She is now
attending secondary school education and performs better. She had this to
say
Mimi naishi na mama wa kambo baada ya mama yangu
kufariki. Nilipokuwa shule ya msingi mama alikuwa akinitesa na
kunipiga sana kiasi nikawa sina furaha na hata shuleni sikuwa na
amani kabisa. Nilipokutana na mshauri akanipa ushauri na
kunieleza kuwa haya ni mambo ya kupita tu jambo la msingi ni
kuzingatia masomo na kweli hiyo ilinibadili tabia nikaanza
kusoma hadi nikafaulu mtihani na sasa nipo kidato cha pili.
Namshukuru mshauri wangu.-MVC,Maharaka Morogoro
I am currently living with my step mother after the death of my
biological mother. When I was in primary school, my mother
punished me seriously which resulted for me to live unhappy live
both at home and school. After meeting and having counseling
services with the caregiver, she advised me to concentrate with
school and ignore the mistreatment from my mother. The advice
from the caregiver helped to draw me back to my normal life and
also improve my academic performance. I am now in form two.
Many thanks to my counselor.
Although MVC gave out their views on how they see the support they have
been provided with caregivers, the evaluation team wanted also to get the
feelings of caregivers on how they see as changes to MVCs they have been
supporting. When asked if the PSS services they have been providing have
made remarkable changes in the lives of MVCs, majority of them responded
that the PSS have real made changes to majority of children they have been
supporting.
One of the caregiver interviewed supported the statement that she has been
receiving messages in writings and some through sms from MVC and their
parents appreciating the support they have been receiving. One of the
messages was as follows:
Mama umekuwa msaada mkubwa kwangu kwa kipindi hiki
ambacho ni kigumu sana. Nakushukuru kwa kunifanya na mimi
kuwa mtu.MVC, TIKVAH Dodoma
Mother, you have been very supportive during the hard time that
I am passing through. Thanks for making me looking the way I
am.
38
Other caregivers also supported the statement that they have been
observing some changes in the lives of MVCs as there is some of MVC who
dropped from school due to psychological problems. But after receiving the
PSS services they have now drawn back to school, behaviors have changed
and their performance in school has improved. She said
Kwa kweli mimi naona mabadiliko mengi kwa watoto
ninaowapa huduma ya msaada wa kisaikolojia. Wengi wao
walikuwa wameacha shule na wengine walikuwa wapweke baada
ya kuondokewa na wazazi wao jambo lililofanya waache shule.
Baada ya kuwapa elimu ya msaada wa kisaikolojia nimegundua
wameweza kurudi shule na wanasoma vizuri, wanacheza na
wenzao na kwa kweli maendeleo yao ni mazuri sana
darasani.Caregiver, Lindi
This means that I have seen a lot of changes among childrens I
have been supporting through PSS. Before the services, majority
of them dropped from school and some were lonely due to the
death of their parents. After providing them with education on
PSS, I discovered that majority was rolled back to school, they
are interacting with their colleagues and as well they have
improved their academic performance.
Despite all the achievements that resulted from PSS, still caregivers have
been experiencing lots of challenge in ensuring that they provide
comprehensive PSS services hence improving the well being of children. One
of the challenges they have been experiencing at the family level is the
resistance of MVC family members to cooperate with caregivers. Also, some
of the family members have now withdrawn their parental roles to leave it to
caregivers. This is due to the fact that they assume these caregivers were
given financial support to provide to these children and therefore, they have
to take the role of caring for their children. This is an indication that some of
family members are still not aware of what they are supposed to do when it
comes to the issue of caring for their children
Improving MVC Wellbeing through VSI Model
According to HUMULIZA, 2006 the VSI Vijana Simama Imara (Youth stand
firm) is an instrument for enabling children and young people to participate
in finding solutions for their own problems. Children/youths organize
themselves in groups for the purpose of supporting each other socially,
economically and psychologically which helps them to build their self
esteem.
39
TShs.100 million for year one, TShs.100 million for year two and TShs.99,
787,000 for year three. It is also evident that during the implementation, the
funds were released on quarterly basis upon receiving the narrative and
financial report of the previous quarter from SATF.
During discussion with FCS team, it was revealed that sometime SATF
delayed in submitting the report both narrative and financial as per the
agreed calendar. This was due to ever adjustment of budget done by SATF
due to increased price and cost of implementing activities. This was also
supported by SATF management that the project budget that was prepared
in 2008 could not feature the price fluctuation and inflation. Therefore they
were compelled to adjust the budget time to time to feature the market
price. This also interferes with the terms and condition that was set before
between FCS and SATF
3.3.2 Performance of SATF
We assessed the extent to which support provided by SATF ensured sound
project design; facilitated participation project beneficiaries; resulted in a
realistic appraisal proposal with an appropriate strategy; supported
implementation effectively; ensured arrangements were adequate during the
implementation of the project, and provided a participatory evaluation,
learning partnerships and adoption of lessons.
Reports from SATF narrative report from May 2009 to September 2012
indicated that several capacity building activities were conducted for
partners. This includes training on PSS to 26 partners, 52 staff from NGOs
implementing partners who were trained as TOTs.
SATF also conducted advocacy training to District heads of departments in all
districts where the project was implemented. The departments included;
community development, Health and social welfare, Education, Police gender
desk and in some areas the agriculture department. It is evident that 260
heads of departments and other departmental staff were sensitized on the
importance of PSS and how to mainstream in to planning and implementing
PSS in their core activities
The other activity conducted by SATF was also to train 260 staff that is 10
staff from each district involved in the implementation of the project (2 staff
from each of the five departments). They were trained on the needs and
commitment to continue supporting the PSS activities through
mainstreaming PSS in to planning and budgeting.
Based on the above assessment from project reports and interviews with
partners, it can be noted that the services provided by SATF were adequate
47
5.0 Recommendations
The evidence from the evaluation indicated that PSS need not to be a
standing alone project and therefore calls for the inclusion of other seven
needs as outlined in the National Strategy for addressing the needs of
MVC. These includes Food and Nutrition, Shelter, Family based care and
support, Social Protection and security, Primary Health care, Education
and vocational training and household economic strengthening.
Creating Safe Space for Children/Recreational centre: One of the
challenge that have been facing children or MVC in Tanzania is for them to
have enough space for them to share experience, feeling and thoughts
49
about the challenges of life and how to address them. This was also
revealed during evaluation that there is no centre/space where MVC can
share their experiences. This call for SATF to continue supporting partners
on the need to establish safe space for them to meet.
The findings from the evaluation indicated that, despite the fact that SATF
have supported 520 caregivers who are providing services to 2600 MVC in
their community, still the demand is very high as majority of MVCs are yet
to be reached. Therefore, there is need for SATF to scale up the project to
other areas where the project was not implemented. This will ensure
many children are reached and supported.
Continue strengthening coordination, cooperation and relationship with
local authority leaders and partner organization in the operation areas.
This will create a strong working relationship as they understand the
project and build the base for sustainability.
Continues documenting of good practices, case studies and lessons
learned. This will enhance the experiential learning of partners whereby
good practices are continuously built on and improved on an ongoing
basis
Strengthening leadership commitment within the district. It was observed
during evaluation that there is lack of commitment among districts official
to implement the PSS activities. Therefore, commitments from leaders
enhances implementation and continuation of the programme and
ensures allocation of resources in terms of time, finance and human
resources within the district
Encourage motivation for caregivers. The findings from the evaluation
indicated that caregivers have been working on voluntary basis that is
without any support of resources i.e. incentives etc. The provision of
motivation ensure sustainability even after the withdrawal of donor
funding
Encourage full participation of all community members towards
achievement of the SATF PSS overall objectives of increasing knowledge
and awareness on provision of PSS services hence improving MVC
wellbeing. This will promote ownership of the project and enhance
participation of community
Strengthening MVC families with economic empowerment activities. It
was revealed that one of the major drawback facing MVC and their
families is the lack of income to support needs of MVCs despite the
introduction of the VSI model in some areas. Therefore economic
empowerment activities will enhance families with stable income to
support the needs of their children
Continue supporting MVCs through Education project to higher education.
One of the problems observed during evaluation which faces some of
MVCs and results into PSS problem is withdrawal of support (Education
support) especially for those children completing primary education and
50
Annexes
Annex 1: List of Respondents Iinterviewed at Partners Level
SN
Name
Title
NGOs
1
Isihaka Msingwa
CEO
PADI
Rachel Sephen
Programme Manager
Mrs Mlay
Director
Dr. Minja
Programme Manager
M/s Milenia
Executive Secretary
Karadea
Rev.
Alphonce
Mushenyela
Mrs Winfrida
Executive Chairperson
NEREFO
Caroline Kisinza
Chairperson
Huruma
Women Group
Necksan William
Chairperson
TIKVAH
MKUKI
51
10
Prisca Hunga
Secretary
LISAWE
11
Asha Fundi
Chairperson
LISAWE
52
Fredrick Ananga
Team Leader
Patrick
Kanyamwenge
Co-Team
Leader
Sarah Pambe
Data
Collection
Edna Barongo
Data
Collection
Michael Sigstance
Data
Collection
Amini Mikidadi
Data
Collection
53
Would you
participate?
be
willing
to
Yes
No
54
Key informant tool for SATF Staff, Partner NGOs and District Official
How beneficiaries (NGOs, Care givers and District officials) were engaged
from the beginning of the project and end?
___________________________________________________________________________
___________________________________________________________________________
55
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
Did the project achieve what it set out to achieve? If yes, How?(Probe to
get more information)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
How are the implemented interventions responding to the needs and
priorities of most vulnerable children?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________
How did implementing partners contribute towards SATF projects goal
(Building the capacity of partner NGOs in PSS so that they can provide PSS
to MVC hence improving their wellbeing)? If yes, How? (Probe to get
more information: Probe more on their supervisory roles and
provision of technical assistance to care givers)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________
Did the project contribute to any Government policy? If yes, How? (Probe
to get more information on the contribution of the project to
relevant MVC national documents such as Laws, Policies,
Strategies and Plans)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
2) Effectiveness
Extent to which project realized outputs and results as set out in the log
frame
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________
Were the objectives of the project clear? Please elaborate
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
Were the planned and implemented activities sufficient to achieve
the objectives/Output? If yes, how? Please explain
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
What factors influenced the achievement or non achievement of the
objectives/outputs?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________
Do you think the project achieved the intended objectives? If yes,
how could it have been done differently?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________
Did the project have any monitoring and evaluation framework? If
yes, how often was this changed if ever changed? What reasons
necessitated the changes?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________
3) Efficiency
Can you explain how the project was managed? Do you think this affected
the project implementation in any way? (explain) How could it have been
made better?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_
Do you think SATF project management/staffs were appropriate for the
task? (Probe for training, qualification, attitude, remuneration/motivation
etc)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___
Was the project cost effective? Were the project funds managed
satisfactorily? (explain) How could it have been made better? Were the
project funds sufficient? Were funds released on time as planned? (Probe
for any bottlenecks)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___
Did the project reach the expected number of beneficiaries? Were they
involved in project management? Was this beneficial? (Probe for any
difficulties experienced), were beneficiaries satisfied with the quality and
delivery of services? What would have been done differently to meet their
expectations?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___
How do you see the working relationship between SATF, Partner NGOs and
Government officials/district (good, poor, bad, very good) and other
partners
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___
Were there any observable risks during implementation? If yes, how were
they managed or anticipated?
___________________________________________________________________________
___________________________________________________________________________
58
___________________________________________________________________________
________________________________________________________________________
4) Impact
What change did the project bring to children who are the most affected
groups and the community in general?
To what extend did the project address the target group? Was the
coverage adequate as per plan/objective? (Probe on how the
identification and selection process of care givers who were
trained to provide PSS was conducted).
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
5) Sustainability
Potential avenues for project continuation after donor withdrawal
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
Did the Organization create Potential Avenue for continuity after the
end of donor support? If yes, Give examples
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
For partner NGOs
Consider those NGO supported in year 1 of implementation
(MKUKI in Moshi Rural Kilimanjaro),
Consider those NGO supported in Year 2 of implementation
(NEREFO in Morogoro Rural-Morogoro and KARADEA in
Karagwe-Kagera)
Consider those NGO supported in Year 3 of implementation
(Huruma Women Group in Dodoma Municipal-Dodoma,
LISAWE in Lindi Municipal-Lindi and PADI in SongeaRuvuma)
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________
Was there a sustainability plan drawn beforehand? If yes, please
share with us if any.
This includes also mainstreaming of PSS issues in the
core mandate of the partner organizations and district
plans
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________________________________________
60
To
what
extent
did
you
collaborate
with
other
partners/stakeholders? Can this contribute towards project
sustainability? Give examples.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________________________________________
6) Lessons learnt
What lessons do we learn from this project that can be adopted to
implement future programmes
Did the project change any of its strategies at any stage? If yes,
what are the reasons that led to these changes? Can these changes
provide useful lessons for application elsewhere?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_________________________________________________________
To what extend did the project design change based on lessons
learned during implementation?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
Any lessons learned from the project?
Also ask about the lessons learnt from the study tour
to HUMULIZA-VSI Model
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_________________________________________________________
Partner Organization
Introductory session:
When did this project start?
What is the coverage of the project? (Probe for more information on PSS
interventions that was supported by SAFT in their implementing areas)
Who are the beneficiaries?
How many beneficiaries were reached by this project? (Probe for more
information on those care givers who
received training on PSS from
Implementing partners and MVC reached by care givers)
1) Relevance
Relevance of the project with respect to specific needs and relevance to
country strategy for improving the wellbeing of most vulnerable children in
Tanzania
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________________________________________
How beneficiaries (Care givers and MVC) were engaged from the
beginning of the project and end?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
Did the project achieve what it set out to achieve? If yes, How?(Probe to
get more information)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
How are the implemented interventions responding to the needs and
priorities of most vulnerable children?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________
How did implementing partners contribute towards SATF projects goal
(Building the capacity of partner NGOs in PSS so that they can provide PSS
to MVC hence improving their wellbeing)? If yes, How? (Probe to get
more information: on their supervisory roles and provision of
technical assistance to care givers)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________
63
Did the project contribute to any Government policy? If yes, How? (Probe
to get more information on the contribution of the project to
relevant OVC national documents such as Laws, Policies,
Strategies and Plans)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
2) Impact
What change did the project bring to children who are the most affected
groups and the community in general?
To what extend did the project address the target group? Was the
coverage adequate as per plan/objective? (Probe on how the
identification and selection process of care givers who were
trained to provide PSS was conducted).
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
64
_____________________________________________________________________
___________________________________________________________
3) Sustainability
Potential avenues for project continuation after donor withdrawal
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________
Was there a sustainability plan drawn beforehand? If yes, please
share with us if any.
65
4) Lessons learnt
What lessons do we learn from this project that can be adopted to
implement future programmes
Did the project change any of its strategies at any stage? If yes,
what are the reasons that led to these changes? Can these changes
provide useful lessons for application elsewhere?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_________________________________________________________
To what extend did the project design change based on lessons
learned during implementation?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
Any lessons learned from the project?
Also ask about the lessons learnt from the study tour
to HUMULIZA-VSI Model
66
_____________________________________________________________________
_____________________________________________________________________
________________________________________________________________
What are your recommendations for improving the programme in
future?
District Official
1) Relevance
Relevance of the project with respect to specific needs and relevance to
country strategy for improving the wellbeing of most vulnerable children in
Tanzania
Have you ever heard about the PSS project? If yes, what have you heard
or what do you know about the project?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
_____________________________
To what extend did the project address the target group? Was the
coverage adequate as per plan/objective? (Probe on how the
identification and selection process of care givers who were
trained to provide PSS was conducted).
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________________________________________
68
3) Sustainability
Potential avenues for project continuation after donor withdrawal
4) Lessons learnt
What lessons do we learn from this project that can be adopted to
implement future programmes
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
Any recommendation for future improvement of the programme
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
__________________________
70
What change did the project bring to children (MVCs) and other
community members in terms of improving the wellbeing of MVCs in the
community? Probe for changes in terms of;
Emotional change
Social change
Intellectual change
Spiritual change
Physical change
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
Were the beneficiaries (At this level we mean MVCs who received
support from care givers) engaged from the beginning to the end of
the project? If yes, please explain how they were engaged?
The engagement includes the identification process of
MVC to be supported, specific services provided,
recommendation on the best ways of supporting these
MVCs and as well barriers.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
2) Impact
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
To what extent do you attribute this change to the project?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
3) Lessons learnt
What lessons do we learn from this project?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
___
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
What are your recommendations for future improvement of the
programme?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
____________________________
For Children: Who received the direct PSS support from care givers
Consent form
SOCIAL ACTION TRUST FUND
73
SATF has been implementing a three years Psychosocial Support (PSS) Capacity
Building Project in 21 Districts in 12 Regions from 2009-2012. The project was
funded by Foundation for Civil Society and it builds upon PSS Project which was
implemented in 2008 with funds from the same Organization.
The Project goal was to contribute to the improvement of the wellbeing of Orphans
and vulnerable children in Tanzania. This was done through enabling care givers at
the grassroots level to use a practical M&E tool to provide PSS services to MVC. The
project also build the capacity of partner NGOs in providing training to care givers
to increase their understanding of OVCs psychosocial problems and offer them
better assistance and enable partner NGOs to mainstream PSS in their policies,
planning and interventions.
As the project is reaching its end January 2013, SATF has planned to conduct final
evaluation of the PSS project and AMCA INTER-CONSULT LTD has been contracted to
undertake the assignment which include assessing the impact and achievement as
a result of project implementation.
Confidentiality and consent: I would like to ask your children some few
questions about the support he/she has received from care giver. His/her
answers will be completely confidential. His/her name will not be written in
this form, and will never be used in connection with any of the information
he/she tell me. Her/his participation to this assignment is voluntary and
therefore the child may respond or reject to any questions that will be asked.
However, her/his response to these questions will help us better understand
the impact of the support she/he has received from the caregiver. The
interview will take about 20 minutes. Would you be willing to allow and
consent for your children to participate in this important assignment?
Yes ___________________________________________________________(Signature of
interviewer certifying that informed consent has been given verbally by
respondent)
No ____________________________________________________________ (If no please
look for replacement of other respondent)
N
o
1.
Question /Issue
Techniques
for
interviewing
children and documentation
Introduction
What is the evaluation is all about?
How long it will take. Do the children
consent to take part?
Throwing a ball and introducing
themselves/ Kurudishiana mpira
Have you received any support Raise up of hands/Kuonyesha
from care givers? If yes, what vidole (to capture number of those
support did you receive from care who know about the support they were
givers? Probe for the support provided )
74
Brainstorming
(to
capture
information to what extent and how
they have received the support from
care givers
Dala dala (capture numbers of those
who mention in support of same
responses)
Emotional support
Social support
Intellectual support
Spiritual support
Physical support
2.
3.
Emotional change
Social change
Intellectual change
Spiritual change
Physical change
Song/dance by children
Look
for interesting
quotations if any.
cases
and
Social support
Intellectual support
Spiritual support
Physical support
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
Did the support you receive from care givers in any way change
your family life/children? If yes, how did the support change your
life? Probe for
Probe for changes before and after receiving the support from care
givers
Probe for the changes in terms of
Emotional change
Social change
Intellectual change
Spiritual change
Physical change
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
___________________________
What lessons do we learn from the support you received from care
givers? Document any story from Parents/guardian
77