MAINTAINING QUALITY IN BLENDED LEARNING: FROM

CLASSROOM ASSESSMENT TO IMPACT EVALUATION

PART I: DESIGNING AND EVALUATING LEARNING

Suzanne Weinstein, Ph.D.

Director of Instructional Consulting, Assessment, and Research
The Schreyer Institute for Teaching Excellence
The Pennsylvania State University
Part I Session Objectives

As a result of this workshop, participants will be able to…

 …write a student-centered, measureable learning
objective
 …map learning objectives to appropriate assessments
 …use course-level evaluation data for program
assessment
But first, some definitions
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,

Suskie, L. (2009) Assessing student learning: A common sense guide.

San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,
 ensuring that students have sufficient opportunities to
achieve those outcomes,

Suskie, L. (2009) Assessing student learning: A common sense guide.

San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,
 ensuring that students have sufficient opportunities to
achieve those outcomes,
 systematically gathering, analyzing, and interpreting
evidence to determine how well students learning
matches the expectations and

Suskie, L. (2009) Assessing student learning: A common sense guide.

San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student learning,
 ensuring that students have sufficient opportunities to achieve
those outcomes,
 systematically gathering, analyzing, and interpreting evidence
to determine how well students learning matches the
expectations and
 using the resulting information to understand and IMPROVE
student learning.”

Suskie, L. (2009) Assessing student learning: A common sense guide. San

Francisco: Jossey-Bass.
Evaluation

“Evaluation is the analysis and use of data by faculty to

make judgments about student performance. Evaluation
includes the determination of a grade or a decision
regarding pass/fail for an individual assignment or for a
course.”

Goldman and Zakel (2009)

A blended or hybrid course is one in which
some face-to-face class periods are replaced
with online activities

+
Designing learning in a blended course
Designing learning in a (blended) course

1. 3.
2.
Learning Assignment Learning
Objective Activities

Wiggins and McTighe (1998), Understanding by Design http://pixel.fhda.edu/id/six_facets.html

Designing learning in a (blended) course

Face-to -face
or
Online?

1. 3.
2.
Learning Assignment Learning
Objective Activities
Designing learning in a (blended) course

Face-to -face
or
Online?

1. 3.
2.
Learning Assignment Learning
Objective Activities

Integrate
face-to-face
and online
elements
Designing learning in a (blended) course

Learning Assignment
Objective Learning
Activities
Team project:
Find an empirical
At the end of this article on a topic a. Students will
course, students of personal read an article
will be able to interest and b. Instructor will
evaluate an collaboratively model an article
empirical evaluate the critique in class
research article article using a wiki

http://pixel.fhda.edu/id/six_facets.html
 Learning Objective Taxonomy Evaluation Teaching Strategies Face-to-face/Online
Write a specific, Match the objective to How you will evaluate What teaching activity connection
learner-centered, the closest equivalent the students on this you will use to prepare How are the face-to-
measureable objective of Bloom’s taxonomy objective? students to meet face activities
What is the best format objective? connected to the online
for the evaluation? What is the best format activities?
for the teaching
strategies?

At the end of this Evaluating Team project: Find Instructor will Instructor modeling
course, students will an empirical article model an article the critique links to
be able to evaluate on a topic of critique (in class). students practicing
an empirical interest and the critique in the
research article. collaborate to online environment.
evaluate the article
using a wiki.

Adapted from Synthesis Worksheet: http://www.schreyerinstitute.psu.edu/pdf/Synthesis_Worksheet.pdf

Creating Learning Objectives
Identify what the learner should know or
be able to do by the end of the course

A learning objective should…

 …focus on the learner
 …be observable (i.e. measurable)

http://www.schreyerinstitute.psu.edu/Tools/LearningObj/
Use Bloom’s taxonomy (1956) to develop
learning objectives at various levels of
learning

Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge

Adapted from Eileen Herteis, The Gwenna Moss

Teaching & Learning Centre
If you prefer, you can also use Krathwohl
and Anderson version of the taxonomy

Creating
Evaluating
Analyzing
Applying
Understanding
Remembering

Adapted from Eileen Herteis, The Gwenna Moss

Teaching & Learning Centre
Chemistry Learning Objectives
Students will be able to:
 Identify the chemical deficiencies of the soil for a
crop.
 Distinguish between different types of chemicals
based on the chemical deficiencies of the soil for a
crop.
 Develop a solution for the soil deficiencies in a
proposal geared towards a business audience.
Evaluate the following learning objective

1. In this course you will understand the 3 major

communication disorders.
Evaluate the following learning objective

2. In this course we will cover the religion of ancient

Egypt.
Mapping Objectives to Assessment
Strategies (mere examples)
Traditional Electronic

Remember Multiple choice or Online quiz or

matching questions fact-based game
Understand Multiple choice or Essay Online Flow chart Concept
Matching questions summary quiz or map
fact-based
game
Apply Multiple Calculate Demonstrate Online Do an Make a
choice the answer a procedure quiz online podcast
questions to a search that
problem examines a
theory
http://edorigami.wikispaces.com/Bloom%27s+-+Introduction
Mapping Objectives to Assessment
Strategies (mere examples)
Traditional Electronic

Analyze Distinguish Analyze Reverse Conduct Collect and Contrast 2

between 2 data engineer a and report analyze blog posts
theories device on a web media clips
survey
Evaluate Evaluate an Appraise a Judge a Moderate Evaluate a Peer
writing research marketing a video or evaluation
sample article plan discussion debate
forum
Create Generate an Formulate Produce and Generate Create a Build a
expository a research deliver a an e- video website
essay plan presentation Portfolio

http://edorigami.wikispaces.com/Bloom%27s+-+Introduction
One last, but not least, important
consideration: Rubrics

Rubrics for digital assignments

http://edorigami.wikispaces.com/Rubrics+-+Bloom%27s+Digital+Taxonomy
Electronic assessment examples
History course objective
Students will be able to create personal histories
that link to specific historical times.

Family e-Album
Electronic assessment examples
Information Science and Technology
Students will be able to collaboratively develop
summaries of research related to online games
and virtual worlds.

IST 440 Research Project

Give it a try
Go to the following link. It will take you to a google
form. The questions will walk you through the
process of creating the appropriate assignment for
a particular objective.

https://spreadsheets.google.com/viewform?formkey
=dDZSbS1XOHI4ZngtMzZLNno4ZkVnWHc6MQ
Let’s see the results

 Blended Learning Workshop Activity -

Spreadsheet
Use course-level assignments to measure
program goals

Psychology Program Goal

Students will be able to understand and interpret
psychological data, graphs and statistics.

Course Objective
Students will be able to correctly interpret a correlation
coefficient.
Performance by student

Exam 4
100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%
1 2 3 4 5 6 7 8
Performance by learning objective

Exam 4 (Unit11-Unit12)
Percentage of students achieving

100%

80%
greater than 80%

60%

40%

20%

0%
11.3;11.1 11.8 11.8 12.3 12.4 12.5;12.1

Learning Objectives
Resources
 Hybrid Courses, University of Wisconsin at Milwaukee:
http://www4.uwm.edu/ltc/hybrid/

 Schreyer Institute for Teaching Excellence, Penn State:

http://www.schreyerinstitute.psu.edu/

 Bloom’s Digital Taxonomy:

http://edorigami.wikispaces.com/Bloom%27s+-+Introduction

 Bloom’s Digital Taxonomy- Rubrics

http://edorigami.wikispaces.com/Rubrics+-Bloom%27s+Digital+Taxonomy

 Evaluating Media Characteristics

http://www.elearnspace.org/Articles/mediacharacteristics.htm
Suzanne Weinstein
sweinstein@psu.edu
814-863-0689

http://www.schreyerinstitute.psu.edu
http://www.assess.psu.edu

CONTACT INFORMATION