Osmose & ASAD

Environmental Education Evaluation 2012

0. Introduction

Osmose is a not-for-profit organization based in Siem Reap, Cambodia. It started to work in 1999 in Prek Toal, a floating village in the
Tonle Sap, the largest lake in South East Asia; a unique flooded forest ecosystem. Since 2003, Osmose has worked in two further floating
villages; Peak Kantel and Kbal Taol. Their activities started because of the serious threats facing the water birds that are dependent upon
the biologically rich and diverse flooded forest within the Core Area. These wetlands had become the most important breeding site in the
South-East Asian region for many large water birds but egg and bird collection for sale or consumption purposes was having dire
consequences for their populations, pushing many to the brink of extinction.
Osmose activities in three villages bordering the Core Area were implemented to address that threat in parallel with the direct
conservation activities of the Wildlife Conservation Society. With the conservation activities, many villagers had lost a vital source of
income and so Osmose established a three-pronged approach to minimise the harm caused by that loss, and reduce the likelihood of
falling back into environmentally damaging practices.
That approach involved the establishment of three main projects, across these three villages:
i) Ecotourism and handicraft programmes in order to replace lost income streams and encourage villagers to understand the
value of preserving their environment;
ii) Environmental Education programme for the children of the villages in which we worked, and;
iii) Social Support and local development for families that Osmose brought together to encourage them to refrain from their
environmentally destructive practices.

Osmose has also grown with the incorporation of the Saray water hyacinth weaving and the Community Based Ecotourism projects. Now,
almost 130 families in Prek Toal and Pech Kantiel villages benefit from participation in Osmose income generation activities carried out in
these villages (Saray, the Water Hyacinth Cooperative and Community Based Ecotourism).
These activities are becoming more and more important as other dangers facing the whole of the Tonle Sap Lake are beginning to make
themselves known. Built structures on the Mekong threaten to reduce or even arrest fish migration and even the extraordinary
phenomenon by which the Tonle Sap River every year reverses itself flooding the plains around the Tonle Sap Lake, the source of the
massive abundance of fish and wildlife on which 3 million lakeside Cambodians livelihoods are dependent.


















ASAD is a Spanish NGO for development that has worked in Cambodia since 2008. Its relation with this country started with the
identification and implementation of a project in the Tonle Sap Lake in partnership with UNV and UNDP.
In July 2010, Osmose and ASAD began to implement a one-year project called Environmental awareness, budget and business
management, alternative livelihood development for the floating communities of Tonle Sap (Prek Toal, Pech Kantiel and Kbal Taol). The
project focused on three of the Osmose programmes:
1. Environmental awareness;
2. Business and financial management training;
3. Alternative livelihoods support.
Related to the first issue of the project, the environmental awareness program, Osmose has brought together 1000s of children on the
lake working in an environmental education program. As a result, these children have engaged in indoor and outdoor classes, bird
watching, and waste management and recycling activities. The program was initiated with idea of reaching all of the children in the
villages in which Osmose operates and providing them the information needed to protect and preserve their unique environment. One of
the objectives was to contribute towards creating the next generation of guardians for this unique and fragile environment.
The children followed a curriculum that was initially (2002) developed by a group of four organisations (Osmose, MLUP Baitong, FAO and
Save Cambodias Wildlife) with funds from FAO, and reedited in 2007 with funds from UNDP / TSCP.
The children were collected by Osmose in a boat for this purpose and returned to their families after class. Through formal education
methods, art, outdoor classes (bird watching, flora and fauna identification), waste management and recycling lessons, the children were
shown different perspectives on their environment.
Implemented since 2004, this programme needed to be renewed and strengthened. Preparing an environmentally responsible generation
for the future is an important part of the response to the environmental threats on the Tonle Sap. But, as some major environmental
issues are becoming more and more real (over fishing, global warming, hydroelectric construction on Mekong basin), delivering the
message to the actual adult generation has become crucial. Although Osmose environmental awareness programme has been mainly
dedicated to children, as the core idea was to aware the future generation of adults, the environmental education program was extended
to adults in the lake in order to work with both groups of population.
Osmose, in collaboration with the Spanish NGO ASAD, started to implement a reform of the environmental education and awareness
program on July 2010. It was designed and elaborated from July to December 2010, and implemented and evaluated from beginning of
February until July 15th 2011. The program included the following issues:

1. Ongoing program for children (OP)
2. New program for children (NP)
3. New Program for adults (AP)

For these three programs, a curriculum with 6 subjects was designed. Content addressed the following subjects:
1. General Environment
2. Fish
3. Wild Life
4. Water and Pollution
5. Flooded Forest
6. Natural Resources and Conservation

Ongoing environmental awareness program for children (OP)
The ongoing program was related to the objectives of the UN in the Tonle Sap Lake, their general environmental and development
policies and strategies, specially, UNDP and UNESCO ones.
The ongoing program continued with 35 groups of children (426 children).

Environmental awareness for adults (AP)
A curriculum with six lessons was designed to work with 153 adults. The program was followed by the adults involved in the income
generating activities:
1. Peak Kanteil: 40 villagers 2 groups
2. Prek Toal: 113 villagers 6 groups

Each group was invited to join one class per month, during six months. The program started in February 1st, 2011 and finished in July 30
th

2011. Some of the adults are parents of the children attending the environmental education training program.

New environmental education program for children (participatory program) (NP)
The new program was adapted from the ongoing program curriculum with the followings inclusions:
1. added information from The Community Environmental Awareness Flip Chart Published by Live and Learn Environmental
Education and WWF Great Mekong in 2006,
2. an update of the environmental issues on the lake.

A curriculum with six lessons was designed to work with 90 children (6 groups of 15 children each) in a participatory program. Each group
chose an environmental subject such as the flooded forest, water bird colonies, fish, waste management or water pollution. This
training was implemented during six months (February 2011-July 2011), in two phases:
a) First phase: 3 months, 2 classes per month. They received complete teaching on the subject during indoor and outdoor
classes (visit in the flooded forest, bird watching, waste collection, waste management or water pollution):
- 4 groups of 15 students in Preak Toal, with 3 teachers (Pichorat, Kimly, Kim Hourn). The students belong to: 8 rich families,
18 medium, 34 poor families.
- 2 groups of 15 students in Pech Kantiel, with 2 teachers (Samon, Mara).

b) Second phase: 3 months. With a participatory methodology and the assistance of a media expert whose expertise was
participatory video techniques, teachers and students prepare an exhibition for the community during two months. The exhibition
consisted of several multi-media (interviews registered or filmed, pictures, drawing, theatre) related to subjects addressed in the
program, such as flooded forest, water bird colonies, fish, waste management or water pollution.
A general exhibition (screening) presenting the contribution of each group would be displayed at the Osmose centre in each
village at the end of the environmental programs. A movie on the environment would be shown at the end of the day, with some
debates and activities to share the experience and the program.
After presentation for the entire villages, the exhibitions would then be available on the Osmose platforms for tourists and
community members to see and enjoy.
If it was successful, the project would be later replicated with new groups of children in these villages, starting with new funds of
new donors.

Preparation of the new environmental education program:
The preparation for the program started in September 2010. Dara (Environmental expert in Osmose) and the teachers prepared a
curriculum on each of the subjects that could be chosen by the students. A meeting with the local authorities to assess their
expectations and possible contributions to this new program took place in November 2010. On December 14
th
Osmose staff had a
meeting in Phnom Penh with Mrs. Navirak, from UNDP, environment specialist, to present the program and the research and to
consider potential partnership for the research and potential funds for future environmental programs. Asad coordinator met as
well the UNDP specialist to present the evaluation research and incorporate its commentaries into the process. The program finally
started at the beginning of February 2011.

1. RESEARCH METHODOLOGY

Quantitative research has been used due to the following reasons:
1.1. The existence of a previous qualitative assessment (2007), which gave useful results and was used to elaborate the
questionnaire of this research.
1.2. Quantitative methodologies allow larger scopes regarding the sample, generalisation and finding comparison.

2. HYPOTHESIS


HYPOTHESIS: Educational intervention addressing specific environmental issues of the local area and targeted at school children and
their parents will enhance their environmental knowledge, awareness, attitude and behaviour regarding their own region.

2.1. Research goals:

This research aims to analyse the impact of an environmental educational program on beneficiaries, which are inhabitants of Osmose in
the Tonle Sap lake, in order to improve their knowledge, concern and behaviour in the future. To that end, it is necessary:

2.2. Specific goals

This research aims to meet the following specific goals:
a. To measure the impact of the various environmental programs implemented on the population of Tonle Sap Lake.
b. To establish which type of environmental educative program is most effective
c. To know the impact of the various activities by the different profiles of population (i.e., to identify the individuals in whom the
environmental education should invest)
d. To assess the potential knowledge transfer (interactions adults/children) to the population living in the surrounding areas which do
not directly benefit from the intervention (to understand the interactions adults/children, in order to know at which group, children
or adults, we should target future environmental programs).
e. To establish the potential differential incidence of the environmental educational activities regarding the type of income-generating
activity for adults.

3. Scope

The scope of this research is limited to the population of Tonle Sap Lake, mainly people living the municipalities of Prek Toal and Peak
Kantiel, where the educational programs were conducted and questionnaires were collected.
Work concerning the control group was carried out in a village outside this area.

4. Population:

Although the socio-demographic section will further describe in detail the target population, the basic population profiles are briefly
outlined here:
Infant students divided into two groups attending two different educational environmental programs each.
o By grade of secondary education
Third and Fourth year
Fifth and Sixth year
o By program
Students attending an old environmental program (6 lessons, once a month)
Students attending a new environmental program (6 lessons, twice a month during 3 month + a participatory
program for them to record videos on their own environment )
Parents attending an environmental educational program for adults
Relatives of these students and their parents (not attending the program)
Control Group (in a different village to those where programs have been implemented )

The target population of this study is divided as follow:

A) Children (526)
436 children attending the ongoing environmental education program
o Prek Toal: 321 children in 18 groups
o Pech Kanteil: 115 children in 7 groups
90 children attending the new participatory environmental education program
o Prek Toal: 4 groups of 15 students each
o Pech Kanteil: 2 groups of 15 students each
B) Adults (153)
153 adults who had environmental education training
Prek Toal: 113 adults.
Pech Kanteil: 40 adults
Moreover, 3 trainers had training for environmental education trainers
Upon data collection through questionnaires, it was noticeable that the final sample was divided in the following population groups:











.
5. Research design


This research design aimed to asses the outcomes of the various programs observed in the various groups of beneficiaries and in the
control group. The outcomes were assessed at four different times, as shown in the following table. Thus, the various assessments were
carried out at the following times:
- T1: before the beginning of each program for all the beneficiaries;

Population groups
Count %
Students (3-4) 258 30
Students( 5-6) 188 22
Adult 219 25
Parents 207 24
Total 872 100
Source: The author relied on the Osmose-Asad Survey
- T2: three months after the end of the new program course for those who attend this program,
- T3: six months after the beginning of each program, after the old program course for children, after the Adult Program course for adults
and after the completion of the participatory audiovisual program of the new program,
- T4: after the last intervention, that of the screening at the Lake of the participatory videos which had been recorded during the new
program course for children, for all the beneficiaries.

The control groups of students were assessed at T1, T3 and T4, while the control group of adults was evaluated at T1 and T4.


6. Measurement tools


The fact that the analysis would be carried out with population groups of great differences was considered to design measurement tools.
Thus, a questionnaire was elaborated for adults and 5
th
and 6
th
grade children, and a second questionnaire was also adapted for 3
rd
and
4
th
grade children. Both questionnaires collect socio-demographic data such as age and sex. However, there are some specific socio-
demographic questions for each group.
In questionnaires for adults and parents, there are questions on the profession and the number of children.
In children questionnaires, there are specific questions on their parents.
Key variables are also included, such as that concerning origin (allowing the establishment of the group control with population in the
other village) or the attendance to a given education program. In all surveys variables concerning knowledge, concern and behaviour are
included, since potential variations, if any, will be measured on these parameters to know the impact of the education interventions.
The item package concerning knowledge has 20 nominal categorical variables with three categories of answer: True, False
and Dont Know/No answer
The item package addressing concern has 10 ordinal variables with 5 categories of answer: Totally disagree, Disagree but
not totally, No agree and neither disagree, Agree but not totally, Totally Agree.
The item package addressing behaviour has 12 variables ordinal variables with 5 categories of answer: Never, Almost never,
Sometimes, Often, Very often.
In order to elaborate the questions of the survey, ASAD team carried out a preliminary research of similar studies on the environment and
environmental education programs in other countries. Afterwards, a draft was submitted for the OSMOSE technicians to review and polish
the latter. Upon review of the questionnaire draft, it was also tested with other OSMOSE technicians, with some teachers of the education
programs and with a sample of the Lake. Therefore potential errors of understanding could be identified, both concerning the content of
the questions and ratings. Final versions of questionnaires can be read at the end of the report (APPENDIX I, II and III)


7. Fieldwork

Fieldwork was managed and coordinated by OSMOSE team. With the supervision of Lindsay and Eric, the survey was conducted with the
help of the teachers of the environmental education programs who delivered the questionnaires at the beginning and at the end of the
session. Teachers were locals who collected the information in classrooms and in the houses of beneficiaries, as well as at different
meeting points.
As shown in the research design, the same survey was conducted at four different times, January, April, July and August. We aimed at
conducting the same survey with the same people at the four different moments established:
A: Before the beginning of the education programs
B: After three months, only with children attending the new program course, before initiating the participatory audiovisual activities.
C: At the end of every program, before the public screening of the videos recorded in the new education program course
D: Immediately after the screenings

After the first survey, which revealed that students mainly answered 1 or 5, a new questionnaire was administered many weeks later
regarding concern and behaviour variables. Those data have been used for data cleansing and have been mixed up with the data of the
first survey, adding the new cases to questionnaire 1 or mixing up the data of the existing cases after previous data comparison.

8. Brief Summary of the Results

The results of the study are describe extensively in the statistical report attached, however, here we present a summary of the main
outcomes obtained with study.

Previously to describe the main results, we will introduce the procedure to analyse the database and the basic statistical concepts needed
to understand the statistical report and results obtained. Three databases were built up: the 3-4 grade students, 5-6 grade students and
adults. Each of the databases has the variables corresponding to each step of the study (Time T1 to T4) and the questions of the
corresponding population study. In the adults database there is an extra variable that distinguishes between adults and parents. In the
students databases there are three study groups, depending on the environmental programme undertaken: old programme, new
programme and control group. The adult database has two study groups: the new programme and the control. In the full statistical report
attached the results from each dimension study on the questionnaire are described between study groups and by each population sample
along the time of study (T1 to T4). As well, the full report contains the mean values of the dimensions, the standard deviation (which
represent the variability on the results, that is how different are responses between participants and how far are one to another).
Throughout the report statistical significant results are indicated. When a significant result is found the interpretation is that the results are
considerable different and therefore there is a different effect on the results of participants depending on the study group or time of study.

Since not all participants were providing responses for all questionnaires across T1 to T4, they can be divided into two further groups:
dependent participants and independent participants.

A dependent participant is the one who reported results on the questionnaire at several times (for example at time T1 and T2 or T1 and
T3 or any other possible combination of time). In other words, are participant that filled in the questionnaire repeated times along the
study.

An independent participant is the one who only filled in the questionnaire once during the whole process of the study, for example a
participant that answered the questionnaire at T1 but not at T2, T3 or T4. Therefore, different statistical analyses were carried out
regarding to dependent participants o independent participants. Nevertheless, in this brief summary we will report only the main results of
the study in order to provide some guidelines to conclude the study. Note that in this brief summary we are not providing detailed of the
statistical tool used to obtain the results or any type of graph and table, we only provide summary trends and differences observed
between participants and group of study.

Before checking for differences amongst group of study (new programme, old programme and control), we had to check that the
questionnaire designed for the study was appropriate. The questionnaire designed for each population group was an adequate tool to
measure the dimensions: knowledge, concern and behaviour, obtaining a Cronbachs alpha between 0.4 and 0.7, except for the
population group of 3-4 grade students where the concern dimension was not well described by the questions proposed in the
questionnaire. Thus, for future studies should be considered to review and modified the items corresponding to the concern for 3-4 grade
students. Note that Cronbachs alpha is a statistical measure for related items within one dimension that indicates if the items of a
questionnaire are appropriate to describe a certain dimension or characteristic. Values of this measure over 0.4 indicates moderate to
good relation between items to described the dimension, whilst values under 0.4 indicate a poor relation and therefore, items or questions
should be considered for revision.

Since dependent participants were few, we mainly consider here the results from independent participants to draw main results (see full
statistical report for detail about dependent participants results).

For independent participants across steps (non paired samples at any combination of step 1, 3 and 4), it was observed that for the 3-4
students of new programme their knowledge at the beginning of the study was significantly worse than for the control group, but once the
programme was finished the students obtained better knowledge than the control and old programme groups. This implies a superior
effect of the new programme over the old programme and non-programme at all on the knowledge of 3-4 grade students. That is the
knowledge of 3-4 grade students improved with the new programme with respect to the old and control. Also differences on the
knowledge between girls and boys were observed, having girls a significantly higher knowledge than boys. No differences were found
depending on the age. Regarding to the concern, the new programme did not improve the concern of the students; at the end of the study
all groups of activities had similar concern. The concern of students increases with the age of the person (independently of the group of
activities). The behaviour of the control group was initially significantly higher than the new activities group, however, at step 3 and 4 the
new group improved their behaviour resulting in significantly higher scales than the control group. Therefore the behaviour in the new
activities was superior to non-activities at all. However, no differences were observed between the old and new programme, in other
words the old and new programme obtained similar results on the behaviour. The behaviour of students with different ages in the new and
old programme group is similar, however, when comparing students from the control group it is observed than the behaviour increase
significantly with the age.

As a result of the study for 3-4 grade students:

The knowledge improved with the new activities in compare with the old activities and control group. That is better
knowledge of students undertaken the new programme
The concern did not change. Students across group of activities had the same concern. The new programme and the old
programme did not have any effect on their concern
The behaviour improved when the child undertakes the new and the old activities in compare with non-activity at all. The
behaviour is similar when the child undertakes the new or the old programme. The new programme did not improve with
respect to the old, but it does with respect to none activities.

Results from 5-6 grade students show that the new students had lower knowledge at the beginning of the study than the control group.
However, the new programme improved the knowledge of the new group in comparison to the rest of students from the other groups. No
differences were found between genders. Initially older students had lower knowledge than younger students, changing those results after
step 3 when the knowledge of older student overcomes the one from younger students. The knowledge between students of same age
but from the experimental groups varies significantly; in the old programme older students had higher knowledge than students with same
age in the new programme. With respect to concern, at step 4 improved significantly with respect to step 1; but a similar increase was
observed for all groups, having not found any difference between groups of activities. No differences on concern between genders and
students from different ages were found. It is observed that the behaviour of the new group was lower than the control group at the
beginning of the study. However, at the end of study the new group had significantly better behaviour than the control group. Initially older
students had worse behaviour than younger students, changing this attitude for the old group.
As a result of the study for 5-6 grade students same conclusion as for 3-4 grade students about the effect of the new programme with
respect to the old programme can be drawn (see points 1 to 3 in previous paragraph).

Results from independent adults show that no significant difference was found between parents and adults from the new programme
and the control group. It was observed that the results at step 3 and 4 were significantly higher than at the beginning of the study, same
tendency from all type of activities. Knowledge results depending on gender and age did not differ. The concern scale only depended on
the age of the adults, increasing their concern with an increase of the age. The behaviour of adults also increases with the age, and in
particular, adults and parents from the new group had higher behaviour than adults in the control group at step 4 (in previous steps not
differences were found between adults from different groups). The results between different genders did not differ significantly.
As a result of the study for adults: at the end of the study the adults from the new group improved their behaviour with respect to the
control group, having a positive effect the new programme in the behaviour, however, not clear improvement was observed for the
knowledge and concern.
We have to note that the results above are on independent participants, which mean that at each step the participants evaluated were
different people. For the case of students, few participants repeated the questionnaire from step 1 to step 3 or 4, therefore no robust
conclusions could be obtained about those responses.
However, for adults, participants form step 1 provided results at step 3 and 4 and so we could analyse those responses and check if the
new programme has a positive effect on those participants along the time. In particular, when comparing those adult participants from
step 1 to step 3 and 4 a higher knowledge was gained with the new programme, indicating that the new programme had a positive effect
on the knowledge of those adults, whilst in the control group the knowledge remained the same from the beginning to the end of the
study.
Although it seems that the knowledge of students improved with the new programme with respect to the old programme and control
groups, it is not clear from the effect an statistical perspective due to the inconsistencies found on the data, such as, different people
where involved in the study at different steps. This issue about different participants across steps implies that the conclusions and
changes on the knowledge may be due to person characteristic and not to the programme undertake. As well, the few people in common
across steps make results to be not robust enough to draw clear conclusion about if the new programme is better than the old
programme. Only for common adults across steps we can conclude that the new programme improved their knowledge with respect to
non-programme.