American Journal of Pharmaceutical Education

Flipping the Pharmacy Classroom: Assessing the Impact of Active Learning

--Manuscript Draft-Manuscript Number:
Full Title:

Flipping the Pharmacy Classroom: Assessing the Impact of Active Learning

Article Type:

Instructional Design and Assessment

Keywords:

Active learning, Process-oriented Guided inquiry learning, instructional design, Action

research, Pharmacotherapy

Corresponding Author:

Richard Pierce, Ed.D

Shenandoah University
Winchester, VA UNITED STATES

Corresponding Author Secondary

Information:
Corresponding Author's Institution:

Shenandoah University

Corresponding Author's Secondary

Institution:
First Author:

Richard Pierce, Ed.D

First Author Secondary Information:

Order of Authors:

Richard Pierce, Ed.D

Jeremy Fox

Order of Authors Secondary Information:

Manuscript Region of Origin:

UNITED STATES

Abstract:

Objective. To develop, implement, and evaluate active learning strategies to improve

student performance.
Design. The flipped classroom model was adopted in which a process oriented guided
inquiry learning (POGIL) activity was developed and conducted in class to complement
previously recorded didactic lectures. The activity was constructed to focus on critical
thinking and application of core principles of medication management of end-stage
renal disease (ESRD) patients receiving renal replacement therapy
Assessment. Student learning was evaluated using performance on three
assessments. Students' final exam performance was compared to the prior year
student performance in which no flipped classroom oriented learning activities
occurred. The mean student grades for the students in the flipped classroom model
increased 6.2% over the previous years.
Conclusion. The addition of active-learning activities implemented in the flipped
classroom model accompanied improved student performance in this study. Student
evaluations of the activity suggested the flipped classroom varied from the majority of
the other pharmacy classes and was universally endorsed by students as an engaging
and effective instructional model.

Powered by Editorial Manager and Preprint Manager from Aries Systems Corporation

Cover Letter

Running head: Flipping the Pharmacy Classroom

Flipping the Pharmacy Classroom: Assessing the Impact of Active Learning

Richard J. Pierce Ed.D.
Assistant Professor
Shenandoah University
Winchester Virginia US
Bernard J. Dunn School of Pharmacy
(rpierce@su.edu)
Phone 540-678-4377 Fax 540-665-1283

Jeremy Fox PharmD, Assistant Professor

Assistant Professor
Shenandoah University
Winchester Virginia US
Bernard J. Dunn School of Pharmacy
(jfo2@su.edu)
Phone 888-420-7877 Fax 540-665-1283

*Manuscript

Flipping the Pharmacy Classroom

INTRODUCTON
The challenge for educators in every discipline is to transition from dispensers of facts to
architects of learning activities. Critical in this process is designing experiences that facilitate
students development into active learners rather than passive receptacles of information.
While large group lecture continues to be a staple of medical education, advances in video and
lecture capture technology, the exponential growth of available quality online content, such as
Coursera and The Kahn Academy, and the developments in cognitive science combine to
challenge traditional notions of teaching and learning. The flipped classroom instructional model
was developed by Jonathan Bergmann and Aaron Sams in 2007 to provide instruction to
secondary students who were missing class and therefore missing instruction. 1 In the flipped
classroom what used to be class work, namely the instructor led lecture and student note taking,
is done prior to class while what used to be homework, typically assigned problems, is now done
in the scheduled class. The model has transformed teaching practice, by changing traditional
roles, and increasing the instructor and student interaction during class. In the flipped classroom,
the responsibility and ownership of learning is transferred from the teacher to the students
through interactive activities.
DESIGN
The empirical support for active learning, generally defined as any instructional method
that engages students in the learning process is extensive.2 Graduate health professions continue
to grapple with the form and the extent to which they should include active learning, despite a
growing body of evidence for its efficacy. The Accreditation Council for Pharmacy Educations
(ACPE) requires the inclusion of active learning strategies.3 Gleason, Peeters, Resman-Targoff
et. al provided a comprehensive overview of active learning strategies in pharmacy education as
1

Flipping the Pharmacy Classroom

well as a well-articulated rationale for adopting these strategies.4 While many programs use
lecture capture to extend learning opportunities, there is little empirical evidence concerning the
use of flipped classroom as to improve student performance in Pharmacy education. In this
project, faculty at the Shenandoah Universitys Bernard J. Dunn School of Pharmacy address the
ACPEs requirement for including active learning practices by integrating instructional
approaches that are typically associated with the flipped classroom.
A design experiment was selected as the guiding methodology of this study. The term
"design experiment" was introduced in 1992, in articles by Ann Brown and Allan Collins as a
method to conduct formative research and refine educational designs based on principles derived
from prior research.5 Design experiments are set in complex natural situations in order to avoid
the distortions of laboratory experiments and therefore constitute a means of addressing the
nuances that are hallmarks of educational settings.6 This investigation examined the impact of
active learning strategies in a Renal module within the Integrated Pharmaceutical Care and
Science (ICARE) series on student final exam performance and student perceptions of the
flipped classroom model. The Institutional Review Board of Shenandoah University approved
the project in April 2012.
Renal ICARE PHAR 608, one of nine required Integrated Pharmaceutical Care and
Science (ICARE) courses within the pharmacy curriculum, was taught in block format in spring
of 2011 and 2012. The course presented pharmacy students with the pathophysiology of common
renal diseases, renal replacement strategies, associated complications, and electrolyte and acidbase disorders. Therapeutic management of patients with renal disease, with a specific attention

Flipping the Pharmacy Classroom

placed on the effects of pharmacodynamic and pharmacokinetic properties of medications, was
the focus of the course.
The instructional intervention for this project was grounded in the flipped classroom
instructional model. Students were assessed initially on a pre-test, then directed to view prerecorded lectures on iTunes U prior to the next scheduled class. A process oriented guided
inquiry (POGIL) learning activity was conducted during the following scheduled class. The
activity, a clinically situated case, extended the application of information from the pre-recorded
lectures. The class activity was conducted at the application level of Blooms Taxonomy of
Learning Domains as students were required to combine patient specific data and drug
information in order to design a treatment regimen for the patient. 7 The design of the activity was
structured from simple dosing calculations, to more complex pharmacokinetic calculations based
upon progressively more complex patient characteristics. The instructor-led activity guided
students from one scenario to the next, interjecting relevant contextual information that
connected lecture content and lesson objectives to the clinical settings. Attendance is mandatory
and was 100% for all scheduled classes in 2011 and 2012. During the POGIL activity, students
were randomly called upon to confirm their answers to the series of clinical questions and
calculations during the activity over a two-hour period. Each student was required to complete
five calculations in the activity. The structure of the activity mirrored the knowledge and skills
that are required on the formative assessments, the course examination, and national board
examination.
Student learning was measured across time for spring 2012 students by considering a
series of assessments including a pre-test, a post-test, and performance on the final exam
questions that related to the Renal module in ICARE PHAR 608. Comparisons were conducted
3

Flipping the Pharmacy Classroom

between the final exam performance of the 2011 and 2012 classes. The same instructor
conducted all the lectures in 2011 and 2012. In spring 2011, course lectures were preformed
live, and were also recorded using Camtasia Relay in a voice over slideshow format, and made
available for viewing on iTunes U after class. In spring 2012, the lectures originally recorded in
the spring 2011 were assigned by the instructor to be viewed by students outside of class prior to
the POGIL class activity. In addition to the identical instructor and identical lecture content,
identical final exam assessment questions were used for both spring 2011 and spring 2012. It was
hypothesized that the flipped classroom would improve student test performance and student
perceptions of the Renal module. The research questions included: Did the flipped classroom
impact student performance?, and Did the flipped classroom impact student perceptions of the
Renal module?
EVALUATION AND ASSESSMENT
In the spring of 2012 teaching strategies consistent with the flipped or inverted class
model were employed to disrupt the faculty dominated lecture and replace it with active, studentcentered learning activities. The class contained 75 graduate pharmacy students. Student
performance was assessed in part by a pre-test post-test design. Assessments were administered
electronically using a secure browser in a proctored setting. Questionmark Perception was
selected for the creation, delivery, and analysis of assessments. Data was further analyzed using
the IBM SPSS/PC software version 19. Summed scores computed using yielded descriptive
statistics for the pre-test (mean=33.5, s.d. = 11.6) and post-test (mean =79.2, s.d. = 10.6)
assessments. A paired two sample for means t-test yielded significant differences between pretest and post-test student performance t(74) = -27.7, p = 0.000. Summed scores were computed
and yielded descriptive statistics for the 2011 (mean=78.5, s.d. = 13.7) and 2012 (mean =83.3,
4

Flipping the Pharmacy Classroom

s.d. = 11.3) assessments. A t-test assuming equal variance was conducted to analyze student
performance on the Renal ICARE final exam questions for the 2011 and 2012 assessments and
yielded significant differences t(143) = -2.3, p = 0.024. Table 1 (see Appendix A) summarizes
the students assessment performance.
Student Evaluations
A secondary outcome concerned students perceptions of the flipped classroom
experience. A 10-question survey was administered to ascertain the students perceptions using a
five point Likert scale (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree). The
survey instrument used investigated two domains of student perceptions: 1) attitudes about the
POGIL activity; 2) perceptions about the flipped classroom model. (Appendix A). The survey
was administered electronically using Survey Monkey and was available for a 13 day period
between the post-test and the final exam. The survey responses rate was 65.8% (N=52). Survey
items for agreement (Strongly Agree and Agree) and disagreement (Strongly Disagree and
Disagree) were combined for reporting. Responses to five items addressing student perceptions
of the process oriented guided interactive lecture POGIL yielded a Cronbachs Alpha measure of
reliability equal to 0.82. Responses to five items addressing perceptions about the flipped
classroom model yielded a slightly larger reliability (Alpha = 0.83).
Student perceptions about the process oriented guided interactive lecture
Students perceptions about the POGIL activity (Table 2) were uniformly favorable.
Responses were most favorable regarding the importance of viewing the pre-recorded lectures
prior to class (n=50, 96.0%), participation in viewing the pre-recorded lectures (n=50, 96.0%),
and the extent to which the instructor required student participation in the POGIL activity (n=50,
5

Flipping the Pharmacy Classroom

96.0%). Most students felt the instructor made meaningful connections between the topics in the
pre-recorded lecture and the class activities (n=47, 90.0%). To a lesser extent (n=40, 77.0%),
students perceived viewing pre-recorded lectures was essential to successfully participating in
the POGIL activity.
Student perceptions about the flipped classroom
Students perceptions about the flipped classroom (Table 3) were also uniformly
favorable. Responses were most favorable regarding the students self-efficacy to address the
topics on the final exam (n=42, 81.0%). Students responses regarding the favorability of being
able to view the lectures prior to class (n=40, 77.0%) was equal to responses indicating a desire
for increased faculty student interaction (n=40, 77.0%). The flipped classroom was regarded as
being dissimilar to other classes in pharmacy school (n=39, 75.0%), and a majority of
respondents (n=32, 39.0%) expressed a desire for more instructors to use the flipped classroom
model (n=40, 77.0%).
DISCUSSION
The flipped classroom replaced instructor dominated didactic lectures, where the students
would traditionally take notes. Scheduled class time was devoted exclusively to assessing, and
developing student content knowledge through activities while pre-recorded lectures were
viewed outside class. Promoting active learning through efforts, such as the POGIL, increased
student exposure to the application of knowledge to clinical case scenarios. Fostering critical
thinking and problem solving acted also as a low fidelity simulation for the final exam format
and content. The design of this project also included repeated exposure to the content in an active
setting including multiple formative assessments. Student performance on final exam questions
6

Flipping the Pharmacy Classroom

improved from 2011 to 2012. Further research is needed to delineate the extent to which repeated
exposure to the concepts through the pre-test, the POGIL activity, the post-test, contributed to
improved student performance.
The results of this survey suggested that graduate pharmacy students recognized the
convenience and pedagogical benefits of the flipped classroom. Students expressed a preference
for the flipped classroom approach in which students engaged with content online prior to
interactive, face-to-face contact with their professor. In summary, these results indicated that
graduate students found the flipped classroom preferable to traditional instructor led didactic
lectures. As we redesign instructional activities in the graduate health sciences, it is important to
consider the increasing demands of millennial learners for activity and interactions.8 Students
expressed a consistently high preference for the flipped classroom relative to the traditional
instructor-led didactic lecture.
CONCLUSIONS
Graduate health professions continue to grapple with the format, implementation, and the
extent to which they include active learning, despite admonitions from accrediting bodies. The
flipped classroom model uses technology to extend content delivery beyond the scheduled class,
and thus transforms the interaction among faculty and students when they meet. The interactive
application of health science concepts in this project reframed the learners behaviors as well as
the instructors approach to pedagogy and assessment. Given these findings, the investigators of
this study offer the following recommendations for subsequent research in this area. Additional
iterations of the project in subsequent years may further identify the impact of the instructional
intervention on student performance. Administer the survey instrument used in this study to a
7

Flipping the Pharmacy Classroom

larger sample for two reasons: (a) to allow for an exploratory factor analysis of the sets of items
included in the survey, and (b) to better generalize to a larger population this type of study.
Future research may wish to investigate more fully the factors impacting students perceptions.
Alternative research projects might focus on questions such as, What factors make the flipped
classroom more effective?
REFERENCES
1. Bergmann, J. Sam, A. 2012. Flip Your Classroom. Reach Every Student in Every Class
Every Day. ISTE 9781564843159. 2012.
2. Bonwell CC, Eison AJ. Active Learning: Creating Excitement in the Classroom. ASHEERIC Higher Education Report No. 1. Washington, DC: George Washington University
Press; 1991.
3. Accreditation Council for Pharmacy Education. Accreditation Standards and Guidelines
for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy
Degree. http://www.acpeaccredit.org/pdf/ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf.
Accessed September 10, 2011.
4. An Active-Learning Strategies Primer for Achieving Ability-Based Educational
Outcomes Brenda L. Gleason, Michael J. Peeters, Beth H. Resman-Targoff, Samantha Karr,
Sarah McBane, Kristi Kelley, Tyan Thomas, Tina H. Denetclaw, American Journal of
Pharmaceutical Education. 2011;75(9):186.
5. Collins A, Joseph D, Bielaczyc K. Design Research: Theoretical and Methodological
Issue, Journal of Learning Sciences, 2004;13(1):15-42.
6. Cobb, P., Confrey J., diSessa, A., Lehrer R., Schauble, L. 2003. Design experiments in
educational research. Educational Researcher, 32(1): 913.
7. Bloom BS, ed. Taxonomy of Educational Objectives, Handbook I: The Cognitive
Domain. New York, NY: McKay, 1956.
8. Stephanie F. Gardner Preparing for the Nexters. American Journal of Pharmaceutical
Education 2006;70(4):1.

Table

Figures and Tables

Table 1 Student performance on assessments in a Renal module
Pretest

Post
test

Renal module
final exam
questions

2011 (n= 68)

N/A

N/A

77.4
(18.9)

2012 (n=75)

33.5
(11.6)

79.2
(10.6)

81.6
(17.6)
p=0.024

Table 2 Student perceptions about process oriented guided interactive lecture.

Strongly Agree /

Disagree / Strongly

Agree

Disagree

Viewing the lecture before scheduled class

prepared me for the class activity.
I did not view the lecture before class,
although I was supposed to.

50 (96.0%)

1 (2.0%)

2 (4.0%)

50 (96.0%)

The instructor required student participation

in the in-class activity.
Viewing the pre-recorded lecture was
essential to successfully participating in the
class activity.
The instructor made meaningful connections
between the topics in the pre-recorded lecture
and the class activity.

50 (96.0%)

1 (2.0%)

40 (77.0%)

6 (12.0%)

47 (90.0%)

1 (2.0%)

Table 3 Student perceptions about the flipped classroom instructional model.

Agree / Strongly

Disagree /

Agree

Strongly Disagree

The flipped classroom model was similar to

other classes in the BJDSOP?

3 (6.0%)

39 (75.0%)

I enjoyed being able to view the lecture prior

to schedule class as opposed to live class
lecture.

41 (79.0%)

6 (12.0%)

I wish more instructors used the flipped or

inverted classroom model.

32 (62.0%)

7 (13 .0%)

I am confident about my ability to address

these topics on the final exam.

42 (81.0%)

5 (10.0%)

I want more interaction between students and

faculty in class.

41 (79.0%)

3 (6.0%)