Evaluation of Determining Instructional Purposes

A Proposal Submitted to
Far West Laboratory for Educational and Research Development
by
Terrapin Evaluations, Inc.

March 7, 2010

John Crescitelli
Boise State University
EdTech 505 Introduction
Week 6/7 Assignment Far West Laboratory for Educational and Research Development (FWL)
Submitted has made a preliminary investment in the design and implementation of a
March 7, 2010
training program entitled Determining Instructional Purposes. Before mov-
ing forward with continued investment and possible expansion, FWL is re-
questing RFPs that focus the evaluation of this program. This document is
being submitted by Terrapin Evaluations, Inc. (TEI) in accordance with the
RFP.

The Determining Instructional Purposes Training Program

Prudent, fiscal decision making is catamount to a school system’s success.

Assuring that approved, educational programs are sound and effective re-
quires a skill set that must be maximized for efficiency. In turn, FWL has
created a program called DIP, Determining Instructional Purposes. This
program is being designed for school administrators and graduate students
in order to help administrators become more effective in planning school
programs.

The program centers on three training units, each containing modules and
activities that actively engage learners in authentic administrative scenar-
ios. Each unit focuses on a certain concentration – setting goals, analyz-
ing problems, and deriving objectives. The units can be run concurrently or
in isolation depending on the needs of the school system. Unit books can
Terrapin  
be purchased individually or by the set.
Evaluations, Inc. 
801 Shakedown Street 
Suite 11A    The program can be run with the use of a simple coordinator. Each coordi-
San Rafael, CA  43141  nator requires a short training that examines each unit, but does not re-
Phone: 609‐532‐6125  quire a special skill set. Coordinator tasks are most organizational in na-
Fax: 609‐532‐6128  ture. Coordinator training manuals are available for purchase.
E‐mail: Eval@terrapin.com 
 Request for Proposal—Determining Instructional Purposes Page 2

Before moving forward, the DIP will need to be evaluated on several fronts:
• Are the three units designed and implemented properly to meet the intended learning out-
comes?
• Are the Coordinator’s handbook and training adequate for proper facilitation of the DIP pro-
gram?
• Will the training impact the behaviors of participants when back in the school administrative
environment?

Evaluation Methods

The DIP training program will be evaluated on several fronts. This proposal will categorize the
evaluations as either formative or summative in nature. Formative assessments will focus on two
target populations – the training participants and the training coordinators. The summative assess-
ments will focus on program participants and school system administration. Each of the assessment
methods will be discussed below.

For this evaluation, it is suggested to run the training units in three-day programs during June and
July to accommodate administrative staff and suggested training times. Trainings will be conducted
Monday-Wednesday during the two month period allowing for examination of the assessments prior
to the next training week. It is the hope to TEI that 24 administrators and graduate students will par-
ticipate in each of the three training programs.

Formative Assessments

Assessment 1 – Training participants will be asked to take part in surveys and interviews throughout
the training to assess the effectiveness of the instruction and facilitation. A Likert-type scale will be
incorporated to assess the effectiveness of each training session. The surveys will also contain ar-
eas for participants to make anecdotal comments about the program, identifying areas of strength
and others recommended for correction. Did the participants feel that the instruction was adequate
in meeting learner need? Did the participants feel that the level of instruction was in line with expec-
tations? Interviews will be conducted with all participants, as the number of initial participants is
relatively small.

Assessment 2 – Interviews will be held with training coordinators twice per unit. The interview ques-
tions will focus on two areas: The effectiveness of the training preparation and areas for revision
based on live implementation. It is important to determine if the coordinator preparation is adequate
for the training programs to run effectively. FWL notes that coordinators need only complete the
units prior to becoming a coordinator. It is important to assess the validity of that.

Summative Assessments

Assessment 3 – Both Pre and Post-testing will be administered to all program participants to assess
learning outcomes for each of the three project units. The pre and post tests will consist of educa-
tional scenarios similar to the modules and group activities present in each instructional unit. Partici-

Terrapin Evaluations, Inc. 
 Request for Proposal—Determining Instructional Purposes Page 3

pants will be asked to make administrative decisions based upon the information presented. The
post testing will used to assess how decision making skills have changed and adapted based upon
new understandings. Some scenarios may be identical in both the pre and post testing to further
measure growth. The purpose is to assess if participants met the objectives as outlined in the pro-
gram protocol? Were students able to make adequate progress in the training program?

Assessment 4 – Interviews with administrators to assess overall satisfaction with the program as it
pertains to meeting desired outcomes. This will be conducted three months following the summer
training. This later date will be done to determine the impact of the training in the actual school set-
ting. Administrators will be interviewed to assess the authentic impact of the training on the adminis-
trative staff and the overall impact on systemic change. The goal of this assessment is to determine
if administrators found participating in the training a worthwhile monetary investment.

Task Schedule

An outline of tasks to be completed during this evaluation can be found in the chart on page four. It
highlights an order of processes, those responsible for completion, and proposed dates.

Proposed Budget

A thorough break down of the proposed budget and payment schedule can be found on page five.

Project Personnel

John Crescitelli, M.E.T., president and CEO of Terrapin Evaluations, Inc. will coordinate the imple-
mentation of all assessment protocol. Mr. Crescitelli has been president of TEI for six years. He has
been evaluating educational programs throughout the country for more than two decades. Mr. Cre-
scitelli has helped 14 school districts restructure training and professional development protocols
throughout the northeast. He leads seminars on the national circuit focusing on Administrative
Leadership and Systemic Change.

George Pelc, Senior Evaluation Coordinator, will be in charge of producing all evaluative tools re-
quired for the assessment. Mr. Pelc has extensive knowledge in evaluation design, receiving a Mas-
ters in Instructional Design from the University of Wisconsin at Madison. He has been recognized
nationally by the National Education Association for his work in transforming evaluation design. As
he completed the model of a comprehensive assessment for the Boston Public Schools focusing on
new teacher training protocols.

Betsy Ross, administrative assistant to the President, will act as the contact person for FWL employ-
ees and all training participants. Ms. Ross has worked on more than two dozen projects similar to
this proposal, and displays incredible organizational skills. She will be available at TEI via Facebook,
Twitter, Google Video Conference, and through a secure web site designed for this RFP. The web-
site will contain all pertinent information surrounding timelines, interviews, and surveys and will act
as the portal for all scheduling.

Terrapin Evaluations, Inc. 
 Request for Proposal—Determining Instructional Purposes Page 4

Task Schedule

The following chart represents the proposed organizational outline for the implementation of the DIP
evaluation. It is projected with a starting date of March 15, 2010 and an ending date of January 14,
2011.

Task Responsible Deadline

1. Initial 2-day meeting. This will entail opening dialogues between FWL and TEI. All FWL March 15, 2010
three representatives of TEI will be present. The goal is to determine areas of focus
for evaluation and determine evaluation strategies.

2. Videoconference between Mr. Pelc and FWL representatives. The focus of this TEI April 6, 2010
meeting is to review the questionnaires, surveys, and interviews designed by Mr.
Pelc for the evaluation.
3. Final submission of evaluation design revisions to TEI by FWL for final mark-up. FWL April 21, 2010

4. Meeting with FWL to examine final drafts of questionnaires, surveys, and inter- TEI May 14, 2010
views. Meetings will also include planning sessions for evaluation strategizing.

5. Initial posting of a secure, DIP-specific website and Twitter. These will include TEI May 30, 2010
scheduling operatives and discussion blogs.

6. Full summer training session for all three units in the DIP training program. All FWL June 1– July
scheduling of workshops will be coordinated through the secure web site. 30, 2010
7. Meetings and Interviews with program coordinators to assess coordinator imple- TEI June 04, 11,
mentation and student attitudinal surveys. TEI will examine participant surveys from 18, 25
each week at these designated meetings, and organizing pertinent data for assess- July 9, 16, 23,
ment. 2010
8. Meeting with FWL. At this time, FWL will provide TEI with all Pre and Post test TEI August 18,
data, all interviews and surveys. This meeting will also be used to make determina- 2010
tions about which participant will be used for the long term impact assessment in
November.
9. Letters will be mailed to long-range participants detailing their role in the process. FWL August 20,
They will each be given a secure access password to the DIP assessment site. This 2010
site will how each participates in ending surveys.

10. Meeting with FWL to summarize the pre and post test data, the participant inter- FWL October 23,
views and surveys, and the coordinator interviews. 2010

11. Open secure portal on the DIP web site for participants to begin impact surveys. FWL November 3,
Send emails to all participants notifying them the impact study survey will be avail- 2010
able for 14 days.

12. Close the survey portal on the DIP site and close all data collection to begin re- FWL November 17,
view. 2010

13. Submit final report to FWL . FWL January 14,

2011

Terrapin Evaluations, Inc. 
 Request for Proposal—Determining Instructional Purposes Page 5

Proposed Budget
Personnel Sub Total Total Actual Cost Breakdown
Mr. Crescitelli, Project Di-
rector (45 days @ $325/day) $14,625.00
Mr. Pelc, Evaluation Coor-
dinator (45 days @ $275/day) $12,375.00
Betsy Ross, Communica-
tions Liason $3,425.00
Total $30,425.00

Travel Sub Total Total

Flights
March 15, May 14, Oct. 23 $3,250.00
Hotels
March 15, May 14, Oct. 23 $2,150.00
Estimated mileage $327.00
Total $5,727.00

Communications Sub Total Total

Website and portal hosting $1,200.00
Video conferencing $750.00
Fax and phone services $475.00
Total $2,425.00

Supplies/Materials Sub Total Total Payment Schedule

Paper, binding materials $550.00
Printing services $745.00 Payment 1: March 28, 2010
Stationery supplies $115.00 25% due within 14 days following
Postage $325.00 initial meeting $10,877.37
Total $1,735.00
Payment 2: May 30, 2010
Miscellaneous Sub Total Total 25% due prior to the start of train-
Per diem meals $1,200.00 ing sessions $10,877.38
Total $1,200.00
Payment 3: October 23, 2010
Program Supplies Sub Total Total 25% due at time of summary
Coordinator handbooks $202.50 meeting $10,877.37
Unit handbooks $1,750.00
Other $45.00 Payment 4: January 11, 2011
Total $1,997.50 25% due at time of final report
meeting $10,877.38

Total Expenses Total Total

$43,509.50 $43,509.50

Terrapin Evaluations, Inc.