Chapter 4: The Evaluation Process: Carrying Out the

Study and Reporting

Presented by: Carrie Gosset & Takiela Langley

Conducting the Data Collection

Before data are collected, obtain the necessary clearances and permission
Provide incentives (personal feedback, a workshop discussion on the findings,
or a report) for people or organizations who take the time to participate in your
evaluation

Educate participants on why the data is being collected and how

the results will be used
Train data collectors in the procedures of data collection &
supervise them to maintain consistency

Get data from as many members of your sample as possible

Be considerate of your participants time and schedules; cause as little
disruption as possible

Analyzing the Data

Check data for responses that may be out of line or unlikely; i.e. selecting more
than one answer when only one can be selected or always choosing the third
alternative on a multiple-choice test of science concepts

Carry out the data analysis specified in the evaluation plan

Conduct a second set of analyses to further address questions

For example, the first analysis looked at overall teacher performance, a second
analysis might subdivide the total group into subunits of particular interest
e.g., more experienced versus less experienced teachers; teachers rated very
successful by mentors versus teachers rated less successfuland examine
whether any significant differences were found between them.

Integrate and synthesize findings; draw a conclusion based on the findings

Formal reports developed by evaluators usually

include these 6 major sections:

Background: (1) the problem or needs addressed, (2) a literature review,

if relevant, (3) the stakeholders and their information needs, (4) the
participants, (5) the projects objectives, (6) the activities and components, (7)
location and planned longevity of the project, (8) the resources used to
implement the project, and (9) the projects expected measurable outcomes.

Evaluation Study
Questions

There are many questions to be asked about a project, and they cannot be
answered at one time.
This section of the report describes the questions that the study addressed. It also
points out some important questions that could not be addressed.

Evaluation Study
Questions:

Has the project achieved its objectives?

Were the required resources for the program clearly defined?
How well was the project managed?
Did the project have the desired impact?

Evaluation Procedures:

The systematic method for assembling, analyzing, and using information to

answer questions about strategies, procedures and programs

Data
Analyses

Describes the techniques used to analyze the data that were collected.
Describes the various stages of analysis that were implemented and the checks
that were carried out to make sure that the data were free of as many
confounding factors as possible.

Findings

This section presents the results of the analyses described previously.

Usually organized in terms of the questions presented in the section on
evaluation study questions.
Each question is addressed, regardless of whether or not a satisfactory answer
can be provided.

Conclusions (and
recommendations)

This section reports the findings with more broad-based and summative
statements

Example of
Formal Report
outline:
Exhibit 12. Formal report outline
I. Summary sections:
A. Abstract
B. Executive summary
II. Background:
A. Problems or needs addressed
B. Literature review
C. Stakeholders and their information needs
D. Participants
E. Projects objectives
F. Activities and components
G. Location and planned longevity of the project
H. Resources used to implement the project
I. Projects expected measurable outcomes
J. Constraints

Data collection
1. Methods
III. Evaluation study questions:
2. Instruments
B. Questions that could not be addressed by
C. Summary matrix
the study
1. Evaluation questions
(when relevant)
2. Variables
IV. Evaluation procedures:
3. Data gathering approaches
A. Sample
4. Respondents
1. Selection procedures
5. Data collection schedule
2. Representativeness of the sample
V. Findings:
3. Use of comparison or control groups, if
A. Results of the analyses organized by study
applicable
question
VI. Conclusions:
A. Broad-based, summative statements
B. Recommendations, when applicable

Photo Credits
https://www.flickr.com/photos/monkeyc/324659432
https://www.flickr.com/photos/paurian/3550755709
http://www.thebluediamondgallery.com/tablet/a/analyze.html
https://www.flickr.com/photos/indi/15738318
https://www.flickr.com/photos/drachmann/327122302
https://www.flickr.com/photos/10557450@N04/6632470867
http://www.thebluediamondgallery.com/scrabble/d/data.html
https://www.flickr.com/photos/nearsoft/21087957719