Dana Fredwell

EdTech 505

Khan Academy
An Evaluation of the Online Program used as a Daily Center in Third Grade Submitted to the administration of Twillman Elementary

Dana Fredwell

Khan Academy

Table of Contents Summary....3 Program Description......4 Evaluation Method..5 Results6 Discussion.8 Project Cost.10 Appendix A..11 Appendix B..12

Dana Fredwell

Khan Academy

Summary This report is an evaluation of Khan Academy used as a daily center in the third grade classroom. By using the Khan Academy program in the classroom students can choose topics and work through them earning badges and points. The purpose of this evaluation is to examine the usefulness of the KA (Khan Academy) in regards to students scores and attitudes about using the technology in the classroom and at home. In order to provide an accurate evaluation of the objectives of the implementation of KA, this report describes the programs objectives along with the classroom teachers objectives. It also explains the evaluation procedures and discusses data gathered from third grade students. Finally it discusses the results and makes recommendations for the further use of KA in the third grade classroom. The third grade classrooms implemented this program to enhance regular classroom instruction by providing support and motivate students to want practice and master math skills. The progress was monitored by KA reports, classroom assessments, and surveys taken by students and third grade teachers. Students and teachers were given an opportunity to provide additional feedback in the survey. Through this process it was determined that KA was partially meeting the objectives. Students determined the program was sometimes too difficult to navigate to the next skill set. The teachers agreed that although the program was indeed supportive and motivating, it might advance too quickly for the third grade students. Most students were unable to use the program at home.

Dana Fredwell

Khan Academy

Program Description Khan Academy is an organization on a mission. They are a not-for-profit with the goal of changing education for the better by providing a free world-class education for anyone anywhere. All of the site's resources are available to anyone. Khan Academy's materials and resources are available completely free of charge. Sal Khan created KA and he prepares each of the video lessons. There are almost 3,300 videos that have been viewed over 160 million times. These lessons were created as resources to enhance students regular instruction. Program Objectives The KA program has been implemented in the third grade classroom to enhance instruction providing support and extra practice. By implementing this program the teachers hope to: Provide support to struggling students Motivate students to practice at home to earn badges Increase skill set to help students become proficient on class, district, and state tests

Program Components The KA program is available online to anyone for free. The program has missions the current mission the third grade class is working on is The World of Math. Students take a pretest that determines the skills students know or need practice with, the test will map skills as, mastered, level two, level one, practiced and needs practice. KA then gives students math skills to work on. Students have the opportunity to test their knowledge by answering questions, practice, or watch a video. When the skill is practiced or attempted students earn energy points with an ultimate goal of mastering the skill to earn badges.

Dana Fredwell

Khan Academy

Evaluation Method
Participants Participants of this program evaluation include 20 students, and three third grade teachers at Twillman Elementary School, a school in the Hazelwood School District. The teachers helped develop the objectives and they were communicated to the students upon the initial implementation of the program. Procedures During the evaluation period the participants used the program for twenty minutes five days a week. Students were given a survey to take after their first use of KA. This was to get students initial thoughts on the program. Students completed the survey online. The three teachers were also given the survey after they used the program; teachers took the same pretest as students. The students were given the same survey at the 4-week point, and the 10week point, to evaluate changes in attitudes about the ease of use, and skills acquired after they had time to delve deeper within the program. The three teachers were interviewed twice, (once in the beginning with the survey, and an interview at the 6-week point) to note changes in students skill set and attitudes about the program. Students also took several assessments during the evaluation period, classroom monthly tests, and the district tests. After the surveys, assessments, and interviews were completed, the results were analyzed in order to determine whether or not the KA program met the intended objectives. Data Sources The data for this evaluation was collected through online surveys and an interview. The surveys focused on ease of navigation, skills learned, and program enjoyment from students. The survey contained six questions; five questions are yes/no with the option of including more information. One question asks students to describe on skill they have learned form KA. The test was given at three different times to evaluate changes in attitudes and knowledge.

Dana Fredwell

Khan Academy

Teachers also took a pretest then completed the survey. Teachers were also interviewed at the six-week point to provide information about the effectiveness, ease, and enjoyment of the program in the classroom setting. The 20 students who participated in the evaluation took classroom assessments weekly and there scores were tracked. The students also participated in the district tests and their scores were compared to their initial district benchmark test given by the district.

Results
Student Survey Results See Appendix A for the results of the three times students took the survey. The first three questions pertained to ease of use and student motivation. Students were asked a yes/no question and could provide additional comments if they wanted to. By the end of the evaluation 90% of students could log in easily with out help. 100% of students could easily navigate the program and find their dashboard. 100% or the students could find help through videos on the site if they needed help with a skill. The next question asked about motivation and the use of the program s badges. 100% of students loved earning badges, they were very excited. The question that asks students about using the program at home was interesting 100% said they would use it at home in the beginning of the evaluation, but this number dwindled down to 30% by the end of the evaluation. Question number five asks students What is one thing you have learned form KA? The responses varied but most of them referred to adding and subtracting with regrouping. A few students said how to multiply one digit numbers. Teacher Survey results Three teachers took this survey also and reported yes to all questions. When asked the one thing they learned they reported the following: I learned how to navigate through the dashboard, I leaned how to add students to my coaches roster, and I learned how students would earn badges

Dana Fredwell

Khan Academy

The interview question asked teachers if they would continue to use Khan in the classroom and why. Teacher one reported that she would definitely use it for her higher achieving students because students could move at a quicker pace. She stated that the topics jumped quickly from simple addition to multiplication and it is difficult for third grade students to transition that quickly. Teacher two reported she would let students use Khan but only to assign topics for extra practice. She discussed the topics moving through levels too quickly for third graders as well. She will use the program when she needs to reinforce a skill or strategy. Teacher three reported she will use Khan but not for independent student use. She reported that she would use the videos to enhance her instruction in the classroom. She also reported that some basic skills are covered but the skills jump from addition to division and multiplication and they were confusing to students because they had not been introduced to that skill in the classroom. The teacher stated this program might be more suitable for middle school students who had been previously introduced to many of the skills. Classroom Assessment Data This test is given weekly and the results were averaged for the month. This test measures many skills needed to be proficient on the end of the year Missouri state assessment.
M-Comp Test Sept Oct Nov

Advanced

0%

1%

1%

Proficient

4%

12%

26%

Basic

30%

47%

48%

Below Basic

66%

50%

25%

Dana Fredwell

Khan Academy

District Assessment Data This monthly test is mandated by the district and tests many skills needed to be proficient on the Missouri state assessment. The data compares the benchmark test from the beginning of the year to Decembers test data.
Evaluate Test August December

Advanced

1%

5%

Proficient

5%

20%

Basic

40%

62%

Below Basic

54%

13%

Discussion The departmentalized third grade math classroom at Twillman Elementary School adopted the Khan Academy Program to help struggling students by giving them support through videos, motivation to learn by earning badges, and ease of use so they would ultimately learn at home to reinforce skills and strategies learned in math class. The results of this evaluation show that although the program is motivating for students it moves rather quickly for third graders. Teachers agree the program is helpful to use as a resource to reinforce skills taught but not to introduce skills independently. The classroom assessments show a 22% jump in proficient students from the initial benchmark test. The most impressive jump was 41% of students were moved out of the below basic category. District assessments show similar results there was a 15% increase of proficient students, and again

Dana Fredwell

Khan Academy

41% moved out of the below basic category. These results may be a combination of instruction, math lessons, and the KA program. The students like the game feel of the program, points and badges are definitely a motivator for students. Students ask to use the program in the classroom. The students were excited when they earned a new badge or conquered a skill. The only draw back was that the skills advanced too quickly, so once they had conquered addition with regrouping it took them right to one and two digit multiplication, which was difficult for students to understand. Although students were excited to use the program at home the cost of the Internet is a cost some parents could not afford. Some students had only one working computer and said they were not allowed to use it, even with a note from the instructor discussing the program. Students also reported they used a tablet or phone and downloaded a Khan App but it did not let you earn energy points or badges. They could only view the videos. One student did use the program at home but again could not get through multiplication of two digit numbers. This is an excellent program and a great free resource to educators. Teachers liked to use the videos to reinforce skills they have taught. Students love the game like interface and the ability to earn badges. This program should be used periodically to reinforce a skill taught in third grade, bit not as an ongoing daily center to help teach new skills. This evaluation concludes that if the program is used daily it would be better suited for older students, possibly fifth grade, who have been introduced to many of the topics and had some practice but need extra help or extra practice. At home use was limited by many factors. The schools demographic is 99% free and reduced lunch so technology is not readily available or accepted as resource third graders should use.

Dana Fredwell

Khan Academy

10

Project Cost

Title Evaluator Dana Fredwell Travel Expenses 90 round trip Research Materials Copies, ink, office supplies misc. (2700 miles @ $0.565) (30 days @ $500/day)

Cost

$15,000

$1,525.00 $200.00

Total Cost

$16,725

This invoice outlines the expenses for this evaluation. Dana Fredwell was the only evaluator receiving a salary. The travel expenses were based on a 90-mile commute to Twillman Elementary School. The program is free of charge and was online so no consumables were needed. The total cost for this evaluation is $16,725.

Dana Fredwell

Khan Academy

11

Appendix A Ease of Use Some of the Comments:

I can log in easily 11/11/13 10/1/13 9/9/13 0% 50% 100%

I needed help I couldnt log in Ms. Fredwell had to help

Some Comments:
I can navigate the dashboard 11/11/13 10/1/13 9/9/13 0% 50% 100%150%

What is the dashboard?

Some Comments:
I can get help easily 11/11/13 10/1/13 9/9/13 0% 50% 100% 150%

The videos are right there! I like the videos It helped me add better

Dana Fredwell

Khan Academy

12

Motivating Students Some Comments:

I like earning badges 11/11/13 10/1/13 9/9/13 80% 90% 100% 110%

Badges are fun I have the meteorite

one
I wish we could earn them

everyday

Some Comments:
Will you use it at home? 11/11/13 10/1/13 9/9/13 0% 50% 100%

It wouldnt work on my moms phone

I dont have a computer

Dana Fredwell

Khan Academy

13

Appendix B