Evaluation of Professional Development Training

Submitted to: The Administration of Northwoods Magnet School of Technology and Innovation

Submitted by: Sherri Harrelson

Summary The program under evaluation is a one-day intensive training professional development seminar provided to all teachers at a newly created magnet school for technology. The school is currently implementing a one-to-one device per student ratio, and administrators are interested in finding ways to cut costs and decrease consumption of paper and ink. The portion of the training that is being evaluated covered the use of SMART boards in the classroom and a specific method of flipping lessons. The goal is to evaluate how well prepared teachers feel with both technologies, as well as how likely they are to actually use them. Additionally, the administration would like to determine whether the program should be expanded, so that teachers at other schools within the district could participate. The purpose of this evaluation, and subsequently this report, is to demonstrate how effective the training was at meeting the teachers specific needs at adopting the new technologies. Since the training had never before been offered, and over half the staff at this school consisted of new hires, the administration is exceedingly interested in the outcomes of the training. In addition, the desire within the district to develop strategies for teachers to adopt more effective measures of implementing the technology available to them is highlighted by the consideration of expanding the program to other schools. Following the seminar, teachers spent some time setting up their classrooms and establishing their lessons for the current school year. Since this is a year-round school, the students returned much earlier than those in a more traditional setting. In order to truly evaluate teachers attitudes with regards to the implementation of the new technologies, it was decided that they should be given opportunities to actually use the strategies provided to them before asking them to evaluate the efficacy of the training. Therefore, a survey, designed to determine teachers attitudes and ongoing needs, as well as comfort levels with the technologies, was developed and distributed approximately 10 days after the start of the new school year. Through the evaluation, it was discovered that teachers changed the ways in which they would use their SMART boards within their classrooms based on the training they received. Results showed that teachers felt comfortable using basic functions, the SMART notebook feature, pre-made lessons, and Socrative through their SMART board. One area of concern was that many respondents admitted that they still did not feel comfortable designing their own activities to use on their SMART boards. In addition, many respondents either remained neutral or agreed that they did not feel comfortable recording flipped lessons in the style displayed during training, that they did not feel prepared to put videos online for students, that they were unsure of what to do for those students lacking internet access at home, and that they did not fully understand the reasoning behind flipping lessons.

Program Description This training program, which is offered by The Friday Institute and is designed to assist teachers in developing more effective teaching strategies, consists of a one-day intensive seminar designed to address the use of SMART boards within the classroom, as well as a specified strategy for flipping class lessons, or recording them as videos and asking students to watch them as homework. Due to the limited time required for the training, the instructors frequently travel to various schools within a large radius in order to provide training and support for teachers. However, due to the lack of follow-up sessions available, there was some concern among those in attendance that the program would not satisfy the needs within this particular community. The program consists of several interactive components, whereby teachers are offered opportunities to discuss concepts, participate in small groups, and engage with the tool(s) being presented. Although the instructors dont provide necessary equipment, to include laptops, SMART boards, and video cameras, it is expected that the school and personnel receiving the training will be equipped with necessary technological components. Program Objectives: As the school transitions to a magnet program for technology and innovation, it is imperative that the teachers gain the necessary skills of using the technology components effectively within their classrooms. In addition, it has been expressed by administration that the school should look very different than a more traditional school, including both in-class activities and assignments that students are completing for homework. These objectives were designed with those considerations in mind. o Teachers will have the ability to effectively utilize SMART boards to present lessons (e.g. adequately able to use tools, features, pens, etc.). o Teachers will have the ability to design lessons using the SMART notebook system o Teachers will have the ability to use the assessment program, Socrative, to conduct assessments with their students on the SMART board o Teachers will have the ability to record themselves providing short lectures in the manner demonstrated during the training in order to "flip" their lessons by having students view the lesson lecture for homework Program Components The training consisted of splitting the whole group of personnel present for the training into two teams and having them attend two separate sessions, with one being available in the morning and the other being available in the afternoon. One training session focused exclusively on the use of the SMART board and included highly interactive elements. Many individuals brought their computers to

the training, and although each individual in attendance did not have access to a computer, at least one computer was available at each table of 3-4 personnel members. All activities were conducted using the SMART board in the meeting room, and no manuals or other printed materials were provided. The second session focused exclusively on the flip training. It consisted of several video pieces that were displayed using the projector in the meeting room, with several interactive discussion components. While there were no printed materials provided, each group did participate in making a very short flip video using the paper-slide method, and several participants were also able to create a flip video introducing themselves to their new students families. Evaluation Method Participants o The evaluation participants were selected because they were in attendance at the training, and they are also the teachers that will be using the technology component in their classrooms and with their students. Although several other members of the staff were in attendance at the training, they were not asked to participate in the evaluation by completing the survey because they are not directly using the technology with students. o Several stakeholders were present at the training, including the IT facilitator for the school, the director of the IT department for all schools within the county, several other county representatives, and both the principal and the assistant principal for the school. In addition, many of the new IT facilitators for the county were there to observe, but not to participate, in the training. Finally, two instructors from The Friday Institute, which offered the training, were there to provide the training. Procedures The training was the sole focus of this evaluation. Both the instructors have advanced degrees and have worked at The Friday Institute offering training for teachers for over a year. One instructor was a teacher for several years prior to joining The Friday Institute to act as a teacher trainer. The other instructor served as a teacher for one year before returning to school to obtain an advanced degree, and subsequently joining The Friday Institute. This was the first offering of this particular training program in the county, and although the SMART board training has been offered extensively in the past, the flip training is a relatively new offering.

The evaluation began with the evaluator acting as a participant in the training. The evaluator observed activities, took notes, asked questions, and participated in all aspects of the training. In addition, the evaluator sought to define the objectives for the training, while also determining participants viewpoints on the quality and necessity of the training. Finally, the evaluator developed an online survey for participants to respond to questions, both scaled and open-ended, about how well the training met their needs. Data Sources For the purposes of this evaluation, the sole component of interest was participant attitudes, as the teachers must feel comfortable and prepared to use the technologies, and they must also recognize the need for the technology in order for it to be properly and effectively implemented within the classroom. In order to determine participant attitudes and any ongoing issues and/or needs they felt they had, an online survey was developed and administered within the school. The results of this survey were then compiled in order to evaluate how well the training program met specified objectives, as well as any additional training requirements that needed to be addressed with regards to the specified technologies. Results Out of the twenty participants surveyed, eleven responded by completing the online survey. Graphics are provided below that highlights survey results. Out of the 91% of participants that had used SMART boards in the past, 70% agreed that the training had changed the ways in which they would use their SMART boards in the future.

Out of the participants surveyed, 63% indicated that they were comfortable using the SMART notebook feature and 63% also indicated that they were comfortable using premade lessons on the SMART board. Finally, 72% indicated that they were comfortable using Socrative on the SMART board as a means of assessment.

Less than half of the respondents indicated that they were comfortable designing their own activities to use with the SMART board.

With regard to flipping lessons, only 36% of respondents agreed that they felt capable of recording their lessons in the style displayed during the training.

Less than half of the respondents agreed that they understood the reasoning behind flipping lessons.

Only 27% of participants feel comfortably prepared to put videos online for students to access.

Finally, 36% of participants admitted that they were unsure of what they should to help students who do not have internet at home to access the flipped videos, while another 45% remained neutral on the topic.

Discussion The purpose of this evaluation was to determine the training programs abilities to meet objectives associated with improving teachers use of technologies in their classrooms and for their students. As indicated in the results, many teachers now feel comfortable utilizing some of the newly introduced SMART board features, to include the SMART notebook and pre-made lessons that could save time in terms of planning and preparing materials. Furthermore, teachers agreed that they felt comfortable using Socrative on their SMART boards as a means of assessing students which assists the school in cutting back on paper and ink use from printing assessment materials, and also serves to save teachers time in grading and providing feedback to students. The training designed to instruct teachers on techniques of flipping lessons and reaching students outside of school by having them watch lessons at home and then complete related activities in class is a great area of concern, as evidenced in the results. Many teachers do not yet feel comfortable recording their videos and uploading them to the internet, and they are concerned about providing access for students who lack internet access at home. In addition, many teachers admitted that they are unsure of the reasoning behind flipping lessons. This is especially important because if the teachers do not see value in the strategy or understand the reasoning behind it, they are less likely to devote the upfront time necessary to record videos, upload them for students, and develop ways to include those students without internet access. Based on the results of this evaluation, I would recommend additional training for all personnel on methods of flipping lessons, to include providing them with opportunities to flip short lessons of their own and receive feedback from their peers. In addition, teachers need to be instructed on various methods that they can employ in order to assist students without internet access to be able to view the videos. This could include providing teachers with instruction and practice opportunities using USB connected storage devices and directly uploading videos to students computers. Furthermore, I would encourage regularly scheduled surveys of teacher needs pertaining to flipping lessons and other technologies in order to insure that they feel capable using the tools available to them effectively. This will become increasingly important over the next several years as the county continues to roll out new devices, strategies, and web based programs that teachers will be required to use.

Project Cost Evaluator: 15 days X 500/per day 7,500.00 This is the cost associated with having the evaluator attend the training and develop the associated survey. In addition, this covers the time spent analyzing and compiling results, and writing the evaluation report. Travel: 1 day X 50/per day Supplemental: 200 This covers phone costs and related expenses. 50.00 200.00 7,750.00 Total Project Cost

Appendix: Link to online survey provided through Google: http://bit.ly/139Mmgz

10