Program Evaluation

Early Intervention Program

FRIT 8435

Spring 2010

Group # 4
Poovndrie Jansen
Jennifer Prall-Barfield
Martha Meloy

1
 Table of Contents

Executive Summary ……………………………………………….….….3

Introduction……………………………………………………………….5

Focus of the Evaluation……………………………………………….…..7

Brief Overview of Evaluation Plan and Procedures………………………8

Presentation of Evaluation Results………………………….………..…...9

Conclusions and Recommendations…………………….………….…….11

Appendix………………………………………………….……..……….12

2
 Executive Summary

Purpose of Evaluation

The purpose of this evaluation is to determine the effectiveness of the Early Intervention

Program at Loganville Elementary School. The goal of the Early Intervention Program is to

provide additional instructional resources to help students who are performing below grade level

to obtain the necessary academic skills to reach grade level performance in the shortest time

possible.

The following questions were posed for this evaluation:

1) How many of Loganville Elementary students achieve grade level success after completing

the EIP program?

2) How many points are students improving on the Criterion Referenced Competency Test after

completing the EIP program at Loganville Elementary School for one year?

Obtaining Data for the Evaluation

In determining the effectiveness of the Early Intervention Program at Loganville

Elementary School evaluators used data from the Criterion Referenced Competency Test.

Scores were analyzed and compared using data from April 2008 and data from April 2009. Data

was obtained from CRCT scores of 5th grade students who qualified for EIP instruction and

completed a year of EIP instruction involving Math concepts.

Conclusions

After reviewing the testing data used to determine the effectiveness of the EIP program at

Loganville Elementary School, the evaluators of this program have determined that the EIP

program was successful. It met the EIP instructional objective of providing additional

3
instructional support to students with below grade level math competencies to meet or exceed

math standards as measured on the standardized test.

Recommendations

The evaluators recommend that the EIP program at Loganville Elementary School should

continue using the same model. However for future evaluation projects the impact of the pull out

model as administered at Loganville Elementary should be compared and contrasted by other

models used in schools within the district and with other districts. This would provide a clearer

picture to the effectiveness of this program.

4
 Introduction
Purpose of Evaluation

The purpose of this evaluation is to describe the effect the Fifth Grade Math Early

Intervention Program (EIP) has on assisting students at Loganville Elementary School to achieve

grade level success after completing the program. The evaluation seeks to find if students are

achieving success set forth by the Department of Education for EIP Programs in the state of

Georgia. How many students achieve grade level success after completing the EIP Program?

How many points are students improving on the Criterion Referenced Competency Test after

completing the EIP program at Loganville Elementary School for one year?

The Early Intervention Program is a statewide program run individually by schools under

standards provided by the Georgia Department of Education. The Early Intervention Program

(EIP) is designed to serve students who are at risk of not reaching or maintaining academic grade

level. The purpose of the Early Intervention Program is to provide additional instructional

resources to help students who are performing below grade level obtain the necessary academic

skills to reach grade level performance in the shortest possible time. The early intervention

program is part of a broader plan to monitor and assist students that have difficulties in math.

This program falls within the gamut of the schools response to intervention (Pyramid of

Intervention) plan. Eligibility into the program is determined by 4th grade test results. This is an

automatic criteria for eligibility, but teachers can use a grade level checklist to qualify at least

3% of students into the program should they need additional support. The EIP Program at

Loganville Elementary uses a pull-out model for instruction, in which students who are identified

for EIP instruction are placed with a trained EIP instructor for 50 minutes a day for small group

instruction in Math.

5
 The evaluation was conducted to gain more information about these questions so that

changes could be made to improve the already existing program. Collecting data from the CRCT

reports and EIP instructors was the best way to determine if the EIP program met its objective

for providing additional assistance to students not on grade level expectancies.

Audiences of the Evaluation Report

The evaluation will assist Administrators, parents, teaching faculty and the student body

in knowing if the EIP Program has been successful. The primary audiences for this information

are school administrators and school board members. However, parents who are unsure if their

child needs EIP placement could also be serviced as well as the regular classroom teacher in

knowing how his or her student is progressing in mathematical skills.

Overview of Report Contents

This report includes a full description of the evaluation object, the Early Intervention

program at Loganville Elementary School. Next, the evaluation questions are listed. An outline

of the information needed to complete the evaluation is included. The evaluation plan and

procedures are presented giving the viewer a greater understanding of how the evaluation

process works. The results of the presentation are outlined giving a summary of the findings and

full interpretation of the findings. In the last section, conclusions and recommendations are

made for the program.

6
 Focus of the Evaluation

The focus for this evaluation is the Early Intervention Program offered by the State of

Georgia and specifically focuses on Loganville Elementary School’s EIP program. The EIP

program is an educational initiative offered to students in Kindergarten through 5th grade who

need extra assistance meeting grade level standards. The Pull-Out Model of instruction is used at

Loganville Elementary, which entails a certified teacher who pulls out identified students in

groups of 11-14 for 45-50 minute segments. The teacher assists students with obtaining the

necessary academic skills to reach grade level performance.

Objectives/Questions of the Evaluation

The evaluation questions focus on how many of Loganville Elementary students achieve

grade level success after completing the EIP program for one year and how many points students

are improving on the Criterion Referenced Competency Test after completing the EIP program at

Loganville Elementary School for one year.

Information Needed to Complete the Evaluation

Data from Georgia’s Criterion Referenced Competency Test was used to determine the

objectives of this evaluation. Scores were analyzed and compared using testing data from April

2008 and from April 2009. Data was obtained from the CRCT scores of 5th grade students who

qualified for EIP instruction and completed a full year of EIP instruction. Students combined

CRCT Math scores are generated from the following 5 domains: Number and Operations,

Measurement, Geometry, Algebra and Data analysis.

7
 Brief overview of the Evaluation Plan and Procedures

The evaluation of the EIP program was initiated by reading and gathering data about how

the State of Georgia uses the program. Documents relating to the history, criterion for selecting

students, funding, staffing were reviewed. An evaluability assessment was conducted to decide if

the evaluation was feasible. All stakeholders agreed that this program should be evaluated. The

next step was framing and focusing the evaluation by generating the evaluation questions, which

again was adopted by all stakeholders involved. The standards and criteria were established for

each question. Since the evaluation was formative in nature, standards and criteria were limited

to how does the EIP program meet its objective of providing assistance to at risk students at

Loganville elementary. The next step was drawing up an evaluation plan, spelling out what is to

be collected and who was responsible. A time frame was set which met with support by all

stakeholders.

To gather data reports were reviewed and teachers involved in the program were

interviewed. The internal evaluator used reports to gauge the success and impact of the program

on the performance of 5th graders on the state test. Information detailing student gains was

collected from reports generated by Riverside Publishing Company. The reports for April 2008

and April 2009 were compared to determine if there were any achievements or regressions in

scores between the 2 years. A passing score of 800 is used by the State of Georgia to verify

whether students have successfully met standards. Any score under 800 is determined as not

meeting standards while a score of 850 to 915 is an exceeding standards score. The table below

shows the scores.

8
 Presentation of Evaluation Findings

Student April 08 April 09 Improvement Regression

Student 1 780 831 + 51
Student 2 775 834 + 59
Student 3 771 800 + 29
Student 4 784 819 + 35
Student 5 787 821 + 34
Student 6 757 827 + 70
Student 7 787 852 + 65
Student 8 797 844 + 47
Student 9 795 821 + 26
Student 10 762 824 + 62
Student 12 795 813 + 18
Student 12 797 821 + 24
Student 13 781 841 + 60

Interpretation of Findings:
• Scores in yellow: exceeding state standards.
• Scores in green: meeting state standards.
• There were no students that had scores which regressed.
• All 5th grade EIP math students made improvements and showed gains in their scores on the
state test. The improvement column shows the point gains made by students.
• One student exceeded state standards.
• All students met state standards.
• The mean of improved scores is 45. Seven students out of the 13 are above the average
score of 45, while 6 students were below the average.
• The score interpretation guide for the CRCT administered in the spring of 2009, list the
following as a guide as to what students can do if they met standards, which includes all 13
students in the math EIP class.

Students performing at this level demonstrate evidence of conceptual knowledge

of the five content domains. They demonstrate an understanding of the four
arithmetic operations in relation to whole numbers and decimal fractions. They
have an understanding of decimal fractions and fractions. They can add and
subtract fractions with unlike denominators. They are able to use common
fractions and decimal fractions interchangeably. Students can model percents.
They can estimate and compute simple problems involving area and volume.
They can measure capacity with appropriately chosen units. Students have
some understanding of congruence of geometric figures and correspondence of
their vertices, sides, and angles. They can use variables and substitute numbers
for the unknown in simple algebraic expressions. They can represent and
analyze data. Students make few mathematical errors during computation and
problem solving. They understand and apply mathematical process skills to
problem-solving situations. They demonstrate an understanding of mathematical
language, and they translate mathematical representations to solve problems.

http://www.doe.k12.ga.us

9
The one student, who exceeded standards, the interpretation guide maintains the student is
performing at the following levels:

Students performing at this level demonstrate broad and in-depth evidence

of conceptual and abstract knowledge of the five content domains. They have
an advanced understanding of the four arithmetic operations in relation to
whole numbers and decimal fractions. They have an advanced understanding
of decimal fractions and fractions. They can add and subtract fractions with
unlike denominators as well as model multiplication and division of common
fractions. They are able to use fractions (proper and improper) and decimal
fractions interchangeably. Students can model percents and apply percents to
circle graphs. Students can estimate and compute the area of geometric plane
figures, including regular and irregular polygons and circles. They are able to
derive the formulas for the area of a triangle and of a parallelogram. They can
measure capacity with appropriately chosen units and can compare one unit to
another. Students can estimate and compute the volume of simple geometric
solids. Students understand congruence of geometric figures and correspondence
of their vertices, sides, and angles. They can use variables and substitute
numbers for the unknown in algebraic expressions. They can represent and
analyze data as well as compare multiple graphic representations for a single
set of data. Students rarely make mathematical errors during computation and
problem solving. They understand mathematical process skills and integrate
multiple strategies to problem-solving situations. They demonstrate mastery
understanding of mathematical language and effectively translate mathematical
representations to solve problems. They also make connections and can justify
mathematical interpretations clearly.

http://www.doe.k12.ga.us

10
 Conclusions and Recommendations
Criteria and Standards used to Judge Evaluation Object

In determining the effectiveness of the Early Intervention Program at Loganville

Elementary School evaluators used data from the Criterion Referenced Competency Test.

Scores were analyzed and compared using data from April 2008 and data from April 2009. A

passing score of 800 is used by the State of Georgia to verify whether students have successfully

met standards. The score combines all areas of Math instruction including Number and

Operations, Measurement, Geometry, Algebra and Data Analysis. Any score under 800 is

determined, as not meeting standards while a score of 850 to 915 is an exceeding standards

score.

Conclusions

After reviewing the testing data used to determine the effectiveness of the EIP program at

Loganville Elementary School, the evaluators of this program have deemed that the EIP program

was successful. The EIP program met it’s instructional objective of providing additional

instructional support to students with below grade level math competencies to meet or exceed

math standards as measured on the standardized test.

Recommendations

The evaluators recommend that the EIP program at Loganville Elementary School should

continue using the same model. However for future evaluation projects the impact of the pull out

model as administered at Loganville Elementary should be compared and contrasted by other

models used in schools within the district and with other districts. This would provide a clearer

picture to the effectiveness of this program.

11
 Appendix
Chart comparing results between 2008 and 2009 scores.

Presentation of Evaluation Findings

Student April 08 April 09 Improvement Regression
Student 1 780 831 + 51
Student 2 775 834 + 59
Student 3 771 800 + 29
Student 4 784 819 + 35
Student 5 787 821 + 34
Student 6 757 827 + 70
Student 7 787 852 + 65
Student 8 797 844 + 47
Student 9 795 821 + 26
Student 10 762 824 + 62
Student 12 795 813 + 18
Student 12 797 821 + 24
Student 13 781 841 + 60

12