Innovation Corps for Learning

I-Corps-L Pilot Evaluation

Executive Summary
May 2014
By
Gary Lichtenstein, Ed.D.
Cathleen Simons, Ph.D.
Sheri D. Sheppard, Ph.D.
The I-Corps-L pilot was planned from June through December 2013. During that time, NSF
Program Officer Dr. Don Millard recruited Dr. Karl Smith as lead PI for the course, and he in turn
recruited Dr. Ann McKenna and Dr. Chris Swan as teaching team co-PIs. The PIs also recruited
Drs. Russ Korte, Micah Lande, Shawn Jordan, and Bob MacNeal as co-instructors, as well as
Brandy Nagel as course TA. Jerry Engel, I-Corps National Faculty Director, provided consulting
support. ASEE provided logistical support, with Rosio Chavela as the lead contact.
During December 2013, Dr. Millard recruited 11 teams to participate in the pilot (9 teams began
and completed the process). Dr. Millard selected the teams from a pool of currently funded PIs
executing EHR grants. As with I-Corps, each team consisted of an Entrepreneurial Lead (EL),
Principal Investigator (PI), and Mentor (M). Also as in I-Corps, each participating team received a
$50,000 supplement for participating. Unlike I-Corps, the I-Corps-L ELs were usually mid-career
engineering education professionals rather than graduate students, and the Mentors usually did
not have business experience. The pilot was launched with a 3-day Kick-Off in Washington D.C.
beginning January 8, 2014, continued with five weekly webinars in January and February, and
concluded with a 2-day Wrap-Up on February 27-28, also in Washington, D.C.
Evaluation data include surveys at Intake, after the Kick-Off, after the Webinars, and after the
Wrap-Up. Survey response rate was 80% overall, although response to the Wrap-Up survey
dipped to 69%. Surveys were adapted from those used in I-Corps courses (the National Collegiate
Inventors and Innovators AllianceNCIIAgenerously provided I-Corps survey instruments and
comparison data to QED). In addition, QED interviewed each PI, each team at least once and
some up to three times, NSF Program Officers, and other stakeholders (25 interviews total). QED
also used I-Corps and I-Corps-L program documents, LaunchPad Central resources, and
observation notes.
Outcome data revealed some similarities among as well as differences between the I-Corps-L
cohort and typical I-Corps cohorts. Average increase in participants knowledge of the 9
components of the Business Model Canvas, upon which the course was based, was similar.
Proportion of go vs. no-go decisions was also similar. Approximately 83% of PIs from both
groups report that they will use the concepts of the course in their future research.

1
QED, May 2014

I-Corps-L Pilot Executive Summary

Yet only 50% of I-Corps-L ELs and PIs reported that will seek other funding for their innovation
after the course, compared to 81% of those from I-Corps. Also, 44% of I-Corps-L participants
reported that they found a viable commercialization path for their innovation, compared to 80%
of those in I-Corps. In terms of course ratings, 29% of I-Corps-L participants reported that the
course fell below their expectations, compared to 4% in I-Corps. Comparable numbers in each
group reported that the course met their expectations (about 30%), while 43% of I-Corps-L
participants reported that the course exceeded their expectations, compared to 64% for I-Corps.
Overall course rating for I-Corps-L was 3.6/5.0 (between fairly good and very good).
Key successes among the I-Corps cohort include:

Participants highly valued the process of customer discovery. Even during the course,
they began applying it to their teaching and other projects. Respondents reported
that customer discovery was the most valuable part of the course.
Participants were excited about the potential of the Business Model Canvas (BMC).
88% believe the BMC has moderate or extreme potential for designing learning
innovations, and roughly 80% agreed that they will apply the BMC in their teaching
and research.
Projects were modified (some substantially) to improve scalability and sustainability.

Key challenges include:

Preparing the teaching team. PIs acknowledged that they did not have sufficient
training or background to translate BMC components into the education ecosystem
context. Only 45% of respondents agreed that the teaching team effectively applied
the BMC to learning innovations.

Delivering the course. Only 67% of respondents (compared to 95% in I-Corps) agreed
that activities were well-suited to the objectives of the course. I-Corps pedagogy was
challenging to the I-Corps-L participants, only 48% of whom agreed that the
educational climate was conducive to learning.

Incorporating elements of the engineering ecosystem into the I-Corps language and
BMC. Team members were challenged by two BMC components in particular:
revenue streams and cost structure. While they acknowledge that the underlying
concepts are relevant, disconnects between decision-makers and users adds
complexity to the ecosystem that was not addressed by the examples used during the
course. Many participants were challenged in applying the BMC to their specific
innovations, and several commented that they were hampered by the teaching
teams not being familiar with their innovation.

QED recognizes the terrific potential of I-Corps-L as a vehicle for improving scalability and
sustainability of learning innovations. In order for the promise of I-Corps-L to be realized, values
of scalability and sustainability need to be integrated into and aligned with the engineering
education ecosystem. Two key issues include:

Building expectations, supports, and incentives for scaling and sustaining learning
innovations into the grant solicitation, review, and evaluation process. NSF program
officers, as well as I-Corps-L team members and I-Corps-L PIs, agreed that if NSFs
goal is to promote scalability and sustainability of initially funded projects, that
2

QED, May 2014

I-Corps-L Pilot Executive Summary

expectationas well as supports and incentives for integrating those valuesmust be

built into the grant solicitation and review process.

Building expectations, supports, and incentives among PIs and higher education
institutions. PI team members do not typically see it as their role or responsibility to
scale and sustain learning innovations. Current higher education incentive structures
so not typically support for those sorts of activities.

QED believes that, although the I-Corps-L pilot wasnt perfect, everyone involved benefitted from
going through it and they recognize the potential of the approach for improving implementation
projects as well as research projects. We offer the following recommendations:
Successes
1) Continue to offer I-Corps-L courses for engineering education PIs, ELs, and
Mentors. All participants valued the customer discovery process as well as applying
the BMC to their innovations.
2) Continue to adapt I-Corps to the I-Corps-L Community. Uniformly high results on
I-Corps surveys suggest that I-Corps is expertly tailored to the audience and
ecosystem of the start-up environment. Lower ratings on many I-Corps-L survey
items suggest that course content and delivery were not aligned to the expectations
and/or dispositions of the I-Corps-L audience. If customer discovery were conducted
on
I-Corps-L, either through review by an advisory board and/or by putting a
team through I-Corps (or I-Corps-L) with I-Corps-L being the learning innovation
developed, we anticipate that the outstanding results obtained with I-Corps would
begin to be reflected by I-Corps-L teams.
Challenges
3) More training for I-Corps-L instructors. The new faculty training for I-Corps, as
described in the 2013 I-Corps Faculty Development Program Workbook, includes 22
hours of formal training with experienced I-Corp faculty, in addition to participating in
I-Corps sessions, coaching two participant teams, providing feedback during team
presentations, and shadowing the teaching team. The I-Corps-L teaching team
worked very hard in a short amount of time to get up to speed on the many facets of
I-Corps and we commend them. In future cycles, QED recommends that the I-Corps-L
budget and timeline include structured and mentored training by experienced I-Corps
instructors. Also, if prospective I-Corps-L faculty served as adjuncts in an I-Corps
course prior to leading I-Corps-L, their familiarity with the course and the Business
Model Canvas would be strengthened.
4) Build professional diversity into the I-Corps-L teaching teams. Include on the
I-Corps-L teaching team I-Corps instructors who have both business experience and
expertise in higher education contexts.
5) Review appropriateness of I-Corps teaching strategies for the I-Corps-L participants.
I-Corps-L participants careers focus on researching and/or implementing best
practices related to teaching and learning. While some valued the very direct and
aggressive instructional strategies used in I-Corps, many others were put off and/or
offended by them. The teaching team reverted to a more collaborative approach midway through the course. In the future, I-Corps-L developers should anticipate
participants expectations of what constitutes effective learning and consider whether
standard
I-Corps instruction is the best approach for this customer segment.
3
QED, May 2014

I-Corps-L Pilot Executive Summary

6) Increase the value of team presentations by generalizing teachable moments to all

teams. Implement a flipped classroom approach to lectures, so that participants and
faculty can relate their own examples from and experiences within the education
ecosystem to all elements of the BMC.
7) Include authentic and relevant examples. I-Corps-L teams struggled to see how some
elements of the BMC applied to their learning innovations. Authentic and relevant
examples that include the complexity of higher education ecosystems could be
integrated into future I-Corps-L courses. Showing how these examples play out in
each of the 9 BMC component boxes would deepen team members understanding of
the model and their ability to apply the model to their own innovations.
8) Consider adaptations of the BMC tool. Some representations of the BMC have
modified cost structure and revenue streams boxes that might be more relevant to
I-Corps-L teams, such as that offered at www.businessmodelyou.com. To be more
effective for participants, revenue streams, for example, might be framed to reflect
sustainable and scalable benefits in higher-education beyond revenues, such as
student learning, life outcomes, and/or retention, as well as university branding,
and/or publication rates, for example.
9) Conduct customer discovery on I-Corps-L. Consider an advisory board and/or a team
that develops I-Corps-L as an innovation in an I-Corps or I-Corps-L course. (This
recommendation is also included under Successes.)
Aligning Partners & Channels into the Engineering Education Ecosystem
10) Integrate goals of scaling and sustaining innovations into the NSF ecosystem. Modify
the grant award process so that it incorporates expectations for customer discovery,
scalability, and sustainability from the beginning.
11) Incentivize for scaling and sustaining effective learning innovations in the education
ecosystem. Anticipate that those who research and/or implement learning
innovations may have motivations that differ from I-Corps team members. Currently,
programs such as SBIR and STTR may not be seen as relevant to many PIs in
engineering education. Incentive structures in higher education can make scaling and
sustaining an innovation challenging. These incentive structures, however, can be
leveraged, if NSF and other partners address the value propositions that drive
engineering education researchers and program developers. The I-Corps discovery
process is a potentially powerful vehicle by which to identify the means by which to
scale and sustain learning innovations.

Web: www.QualityEvaluationDesigns.com
Email: gary@QualityEvaluationDesigns.com
4
QED, May 2014

I-Corps-L Pilot Executive Summary