Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY

Computational Thinking and Scratch Jnr.: A Case Study-A closer look at student thinking.

Research Analysis & Critical Review

Cristina Leo
ETEC 500
University of British Columbia

August 19, 2018

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
2

Research Analysis

1. The purpose of this study was to explore young students’ thinking when completing basic

coding tasks using Scratch Jnr. on the iPad. As governments push for the inclusion of

coding into the curriculum, the researcher identified a gap that included empirical

evidence and extensive literature review (knowledge) exploring benefits from

programming to the enhancement of general thinking skills. Therefore, the researcher’s

purpose was to “investigate the types of thinking young students used, and when they

used them within computational tasks, to learn more about whether such tasks create

useful opportunities for students to exercise and possible enhance an array of thinking

capabilities.” The study was guided by the following research questions: “What types of

general thinking skills were evident, and how were they applied, in these students’

computational work? What relationship exists between the exercise of general thinking

skills and these students’ computational work?” (Falloon, 2016)

2. The most significant prior literature used to ground this study included Krathwohl’s

(2002) revision of Bloom’s Taxonomy (Remember, Understand, Apply, Analyze,

Evaluate and Create) and Brennan and Resnick’s (2012) computation thinking (CT)

frameworks. As well, the prior research of Paper and Wing. Pea & Kurland (1984) and

Mayer et al. (1986) work both discuss the lack of evidence determining whether the

thinking skills used in coding are transferable to the enhancement of general thinking

skills. The author’s most compelling argument for further study in this area is based upon

this gap in research.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
3

3. The most significant construct was the computational thinking skills aligned with the

frameworks of Krathwohl (Bloom’s revised taxonomy) and Brennan and Resnick (CT

skills). This was operationally defined as the observable indicators of thinking skills

through verbal responses embedded and aligned with the general thinking and main area

of application. The following thinking skills, Remember, Understand, Apply, Analyze,

Evaluate and Create were of particular importance.

4. The research was qualitative as data was collected by observing children’s dialogue and

communication captured using a video recording app on the iPad’s. This non-intervention

research design did not involve an intervention or experimental manipulation, but rather

aimed to uncover relationships between computational thinking and frameworks in

thinking, in particular what types of general thinking skills were used and how they were

applied. This was a theory based research study utilizing previous theoretical

frameworks.

5. The variables included the thinking skills present from the frameworks of Krathwohl

(Remember, Understand, Apply, Analyse, Evaluate, Create) and Brennan and Resnick.

6. The research design was a qualitative, case study approach to investigating the questions

of what and how. Relying on the theoretical framework of Krathwohl and Brennan and

Resnick, this field study was conducted in two primary year 1 and 2 classrooms in New
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
4

Zealand. Research was done in a non-controlled environment, where students were able

to work uninterrupted, within a normal classroom environment structure and routine.

7. The researchers recorded data using an embedded display and audio capture app installed

on the student’s iPad Air tablets. The app recorded the students’ on screen activity as well

as their verbal discussions during their time coding the geometry task using Scratch Jnr.

Using Studiocode video analysis software allowed for a double blind code of the sample

data, to control for bias and any confounding variables.

8. The sample in this study included two primary year 1 and 2 classrooms from an

elementary school in New Zealand. There were 32 students in total, making up the 16

pairs (14-year 1 students=2 months of schooling, and 18-year two students=14 months of

schooling). None of the selected students had participated in any previous research

conducted. A balance of data from boy/boy, girl/girl, and girl/boy pairs was collected.

9. The researcher stresses that because this study was not about measuring or assessing

progression​ in thinking skills but rather the ​investigation​ into the ​types​ of thinking

present and ​used​ while students completed a computational task (coding in a geometry

lesson) the authors relied on two separate coders to identify specific thinking skills. Using

the camera app as a tool to record data was reliable in accurately verifying language used.

The Kappa results were rated as ‘good’ according to the Landis & Koch (1977) score.
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
5

10. The researchers acknowledge the limitations of the scope of their study and point to the

need for continued research into the study of transferability of thinking during

computational tasks to general thinking. They do acknowledge that through this study,

evidence suggests that benefits from working collaboratively on tasks, students benefit

from coding not only for those interested in vocational aspirations but in the broader

contributions towards student learning. Discussion around the benefits of coding in

schools suggests links to the core competencies, such as communication and

critical-thinking.

11. The sample data was coded using a double blind code using Studiocode video analysis

software. To begin, of the 21 hours of recorded data, the coders selected and analyzed

only 9 ½ hours of data. Selection was based upon the following criteria; balance of

gender pairs (boy/boy, girl/girl, boy/girl), data from both year 1 and 2 paris, and data

from work on all sets. First, research coders looked to identify if any evidence existed of

students using Krathwohl’s revised Bloom Taxonomy and second, how they were linked

to the components of Brennan and Resnick’s framework, they also noted the amount of

times these thinking skills were displayed. Raw field data was converted into fractions of

the day to allow for accurate percentages of the total time spent on each coding task.

From the observed recorded data, evidence (major findings) suggests that students did

display different types of thinking skills and how they were used was often in a

collaborative nature to share information between pairs, to solve the coding activity tasks,

through problem solving.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
6

12. The most important points in the discussion section of the research report speak to the

fact that increased research is needed into the transferability of linking thinking skills

during computational tasks towards generalized thinking skill development as a whole.

The authors concluded that students who participated in the coding of the geometry tasks

did demonstrate thinking aligned with the revised Bloom’s taxonomy by Krathwohl and

the computational thinking framework of Brennan and Resnick. The author notes that

important in this study was the identification of the need to explore the thinking that

occurs during operation (when students are coding) or more specifically understanding

the cognitive dimensions of programming. Also, the study revealed evidence that

students engaged in collaborative, team-based work, that relied on self-management skills

proved to be the foundational skills that led to higher order thinking. Overall, the author

notes that when integrating coding into the curriculum, these additional benefits go

beyond preparing students for a future vocation in programming, but rather developing

skills for student success in learning in general.

Critical Review

Purpose, Contribution, & Significance

Within a global context, governments have recently made changes to the curriculum of

education, emphasising the implementation of computational thinking and coding into the K-12

curriculum. Garry Falloon’s 2016 research article, “An analysis of young student’s thinking
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
7

when completing basic coding tasks using Scratch Jnr. on the iPad” sheds light into the types of

thinking students engage in when completing computational tasks.

The purpose of this study was “not to measure or assess progression in thinking skills

development” but rather to “investigate the ​types​ of thinking young students used, and ​when​ they

used them within computational tasks, to learn more about whether such tasks create useful

opportunities for students to exercise and possibly enhance an array of thinking capabilities.”

(Fallon, 2016)

Falloon recognizes the theoretical work of Papert and Wing regarding computational thinking

skills, but also notes the research of Pea and Kurland (1984b) pointing to the “lack of empirical

studies exploring possible transfer of problem solving in other domains.” From this knowledge

gap, Falloon arrives at his research questions in which he investigates the ​types​ of thinking

students display when solving computational tasks (e.g. coding a geometry task using Scratch

Jnr. on iPad) and ​how​ these thinking skills are applied. The theoretical framework of

Krathwohl’s revised Bloom’s Taxonomy and Brennan and Resnick’s framework for studying

and assessing computational thinking form the basis of his analysis framework (General thinking

and main area of application). He also seeks to identify if a relationship exists between general

thinking skills and computational task work.

Methodology
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
8

Embedded in the theoretical framework of thinking skills of Krathwohl and Brennan and

Resnick, Falloon’s research methods include a qualitative case study, gathering data from two

primary (K-6) year 1 and 2 classrooms (5 and 6 year olds) in a school New Zealand.

Students were asked to work in pairs and create shapes and letters using the Scratch Jnr. program

on the iPads. Student’s voices demonstrating their thinking skills/discussions were recorded as

well as screen captures of their work. Some prior coding experience was conducted before

embarking on this computational task. From the 32 students, data was collected over 5 sessions

ranging from 25 to 40 minutes each. Students worked in pairs due to the limited amount of iPad

devices but also to provide opportunity for discussion and recordable thinking skills with the

embedded display and audio capture app installed on the devices. Data samples included a

balance of boy/boy, girl/girl, and girl/boy pairs, both year 1 and 2 students as well as data

representing the student’s work on all the set tasks for analysis.

In regards to the coding of the data collected, “sample data was double blind coded using

Studiocode video analysis software.” (Falloon, 2016) A total of 9 ½ hours of video of the 11 of

16 pairs were individually imported into the program. Researchers also separately “coded 3 hrs

of the same data to enable an inter-rater agreement calculation”. Using only coding that both

researchers agreed upon was used to calculate rater-agreement. Within case studies, one

methodological issue based on the observations was what was not recorded while using the iPad

app. It is possible that some discussions were missed due to the absence of researcher in the

room. As well, increased opportunity for triangulation would have been useful.
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
9

Weaknesses

One of the weaknesses of this case study is in the sampling of participants. The author notes that

one of the teacher’s had already participated in previous studies conducted by Falloon. This

potential form of bias may have altered the way in which the teacher provided scaffolding during

computational task time using the iPads. The lack of generalization of these findings to extend

into other classrooms can also be seen as a weakness. The coders themselves as instruments can

be seen as potential weaknesses of the research. In particular, the data that was not used due to

lack of consensus, would have benefited from another coder to help with reliability and validity

of observable data. Not mentioned in the study was the philosophy of grading of the assignment.

This could have altered the mentality of the children to take-risks in their thinking. Considering

the Hawthorne effect, it was not mentioned if students were aware that they were participating in

a research study, potentially altering their behaviour. Finally, since this study was conducted in

New Zealand, taking into account the cultural differences, learning styles, etc. may be considered

an extraneous variable.

Strengths

Evident in the final discussion and conclusion of the research, Falloon points to the fact that

additional research is needed to check for generalizability of transfer of thinking skills while

students are engaged in computational tasks. One strength in this research is the combination of

general thinking and main area of application, noting that thinking skills don’t always occur

strictly in isolation. Falloon’s identification of the need to research coding in education is timely

in the event of changes in curriculum. Finally, another benefit of this research was the discovery
Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
10

that many students engaged in risk-taking, developing grit through team-work to problem solve

the computational task. The findings also supported the importance of core competencies, such

as collaboration and self-management skills, also being highlighted by government curriculum.

Summary Recommendations

Overall, this research sought to discover the types of thinking skills students use when

challenged with a computational task and revealed that more studies are needed to fill in the

knowledge gap. Falloon’s work demonstrates that student’s thinking does align with the

framework of Bloom’s revised taxonomy by Krathwohl, and Brennan and Resnick’s framework

of computational thinking when coding, even at an early age. This research reveals that rather

than focusing on transfer theories of thinking skills, which the author states act as a distraction,

emphasis should be placed on studying the thinking skills that occur during student’s minds

while they are coding (present tense), analysing their cognitive dimensions of programming. In

conclusion, the value of this research takes into account the demands to include coding and

computational thinking instruction in the classroom, and the value of this skill development in

overall student learning.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY
11

References

Brennan, K. & Resnick, M. (2012) New frameworks for studying and assessing development of

computational thinking. ​Paper presented at AERA​, Vancouver, BC.

Falloon, G. (2016). An analysis of young students' thinking when completing basic coding tasks

using scratch jnr. on the iPad.​ Journal of Computer Assisted Learning, 32​(6), 576-593.

doi:10.1111/jcal.12155

Krathwohl, D.R. (2002). A revision of Bloom’s taxonomy: An overview. ​Theory Into Practice,

41, ​212-225.

Suter, W. N. (2012).​ Introduction to educational research: A critical thinking approach​ (2nd

ed.).

Thousand Oaks, California: SAGE Publications.