Too many decisions in lifelong

learning organizations are

made today using the wrong
approach. It not only leads to
the wrongg decisions beingg
made, but it also actually takes
more time to make the wrong
decisions.

Becoming a Data‐‐Driven Organization
Becoming a Data
LIFELONG LEARNING 2007 GREG MARSELLO

Use a data-driven decision-making approach if

The Wrong Approach
you are making decisions on:
„ The wrong approach usually involves a number of „ Activities to offer
people making the decision, with the decision „ Teachers or presenters
based more on the opinions of those people rather „ Budgeting
than the data.
„ Pricing
„ Canceling courses
„ Promotion
„ Marketing
„ Length of time
„ Course/event format
„ Location
„ Brochure copy or any
other
th programmatic ti or
marketing kinds of
decisions.

The Wrong Way The Wrong Way

„ The wrong way is to make program and marketing
decisions based on:
„ Opinion. Many decisions are based on one or more
person’s opinion, rather than the actual facts and data.
„ Consensus. Group p decisions often tryy to achieve
consensus, more to preserve the group cohesiveness
than to make the right decision.
„ Personality. The personality of a leader may move a
decision in a certain direction. Other personalities, from
those who talk a lot to those with purchasing power to
those with seniority, often have undue influence.
„ Group morale. Some decisions are actually made more
to maintain group morale than to make the right
decision. One person might gain deference on a given
decision because s/he has not “won” another decision,
or certain persons feel left out, or a given decision
makes everyone feel better.

1
Meetings Bad

* Committee generally bad The time spent in meetings, furthermore, takes

* Staff meetings bad time away from actually doing the programming
„ It takes time to… and the marketing. Group decision making takes
„ Plan longer, and often deadlines are missed, causing a
„ Convene decrease in registrations.
g
„ Make sure everyone has arrived
„ Do introductions and pleasantries
„ Distribute information
„ Consider exceptions
„ Handle diversions
„ Ignore interruptions
„ Make decisions.

The Right Way & Person The Right Way & Person

„ The right way is to make data-driven decisions „ The right person to make these decisions is the
based on: person who has the:
„ Customer behavior „ Most information, and/or
„ Customer evaluations „ Best information.
„ Customer demographics
„ Previous history In almost all programming and marketing
„ Other data. circumstances, there is one person who has
access to or experience with previous history and
more data than anyone else. That person should
make the decision.

When Groups Help When Groups Help

The rare circumstances when groups make better In these situations, groups often make better
decisions than individuals. decisions than individuals. But these
circumstances do not apply to most day-to-day,
„ These almost always involve: session-to-session, or event decision making for
most pprogrammatic
g and marketingg decisions.
„ High dollars,
dolla s those decisions in
involving
ol ing $100
$100,000
000 or
o
more
„ High risk, those decisions with little or no previous
history, often involving precedent-setting or new
activities.
„ Little or no data, where there is little or no data
available from which to make a data-driven decision.

2
Redefine Decision-Making Authority Redefine Decision-Making Authority

1. Assign responsibility. For 2. Develop levels. Mentally, devise a dollar scale of

every ongoing and regular decision making. For example, front line staff can
decision involving make decisions up to $100 for such things as
programming and marketing, purchasing, refunds, discounts, and exceptions to
assign
g a person
p to make the rules. Professionals make decisions upp to $10,000
$ ,
decision. This often can be for such things as courses/ events, instructor
written into a person’s job pay, promotion costs, and so on. Executives are
description. It can also be involved in decisions over $10,000.
conveyed verbally or with a
simple email.

Redefine Decision-Making Authority Redefine Decision-Making Authority

3. No precedent setting. Staff understands they 5. Trust. Initially, top executives will need to trust
cannot make decisions which are precedent the decisions made by staff. After a while, most
setting. staff will develop data-driven judgment and
4. Decision guidelines. By your decision making, by decision-making skills, and you can take
your reporting and staff briefings,
briefings by sharing of corrective action with those p people
p not making g
information, and by demanding information, you good decisions.
make it clear that decisions are to be data-
driven, based on customer behavior, participant
evaluations, customer demographics, and other
data.

Use a data-driven decision-making approach if

Redefine Decision-Making Authority
you are making decisions on:
6. Advocate and object. If you are asked to serve „ Activities to offer „ Length of time
on a decision-making committee, advocate for „ Teachers or presenters „ Course/event format
data-driven decision making by one or two „ Budgeting „ Location
individuals, document the inefficiencies of the
committee,, and mentallyy keep
p track of the bad „ Pricing „ Brochure copy or any
decisions made for the next committee „ Canceling courses other
th programmatic ti or
assignment. Advocate for data-driven decisions. „ Promotion marketing kinds of
decisions.
„ Marketing

3
Data to Collect Demographics

„ Demographics: Registration „ Address

„ Promotion Method: Registration/During Activity „ Birth Year
„ Evaluations: During Activity/Spot Check „ Gender
„ Costs: Invoices „ Occupation
„ Preferences: Surveys
„ Job Title

Not best to collect all data at same touch-point. Best that you determine occupation and job title.

Promotion Method

„ Distribution Method
„ Mailing List
„ Snail or Electronic
„ Mailing 1, 2, 3…
„ Word of Mouth
„ Web Site

Evaluations Model Programmer Course Evaluation Form

„ Satisfaction
Course__________________Teacher__________________
„ Teacher/Presenter Programmer Evaluation. At the conclusion of the above course, the
following information should be generated. A copy should be passed
„ New Activities on to your supervisor, a copy should be put in the teacher’s file and a
copy should be put in the course file.

A. Student Evaluations
1. Overall, were you satisfied with the course? ____ YES ____NO
2. Fill in your response to the following questions:
1–very poor, 2–needs improvement, 3–okay, 4–good, 5–excellent
Understood the subject matter. _____
Was well prepared for each session. _____
Made the goals and objectives clear at course. _____
Stimulated discussion and group involvement. _____
Provided individual help when needed. _____

4
 3. Comments. On a separate sheet of paper, list key comments on what 3. Comments. On a separate sheet of paper, list key comments on what
participants liked about the course and improvements they suggested. teachers liked about the course and what improvements they
4. Testimonials. On a separate sheet of paper, list key testimonials along suggested.
with the person’s name. Make sure to hold their evaluation form on file 4. Testimonials. On separate paper, list key testimonials along with the
teacher’s name. Make sure to hold their evaluation form on file
because it has their signature approving the use of their testimonial. because it has their signature approving the use of their testimonial.
B. Teacher Evaluation C. Performance Analysis
1. Overall, were you satisfied with the course? ____ YES ____NO 1. Price of the Course _____
2. Total Registrations _____
2. Fill in your response to the following questions: 3. Total Withdrawals _____
1–very poor, 2–needs improvement, 3–okay, 4–good, 5–excellent 4. Total Income (after withdrawals) _____
Was the organization’s staff helpful? _____ 5. Total Cost of Promotion _____
Was your room prepared for your course? _____
*If there was additional promotion beyond the catalog listing, attach a
Did the learning environment meet your needs? _____ copy to this evaluation. *To figure the promotional cost of a course in
Did the description clearly outline your course? _____ a catalog, divide the total cost of the catalog (desktop, printing, mail,
Did your programmer give you feedback? _____ etc.) by the number of courses.

Costs

„ Promotion
6. Total Cost of Production (instructor fee, materials, space)
_____ „ Production
7. Total Direct Costs (5 + 6) _____
8. Operating Margin (4 – 5 + 6) _____ „ Administration
D. Half-Life: The number of weeks/days prior to the course when 50% of
registrations were generated
d _____
Income $____
$ 100%
Promotion $____ 10-15%
E. Participant Analysis Report. On separate paper, breakdown the
students’ demographics: Age, Sex, Geography, Education Level. Production $____ 45-50%
F. Programmer Evaluation Direct Costs $____ 60%
1. Overall, were you satisfied with the course?
2. Rate the teacher on the Likert Scale (1–5)
_____
_____
Operating Margin $____ 40%
3. What action steps would you take to improve this course? Administration $____ 35%
Net $____ 5%

Preferences The Language You Should Be Speaking

„ Whatever You Need to Know „ Repeat Rate „ Staff Productivity

„ More Surveys, Not Less „ Lifetime Value „ Average Participants
„ Operating Margin „ Average Fee
„ Cancellation Rate „ Satisfaction Level
„ Brochure:Participant „ Development Costs
Ratio or Response „ Go/No Go Point
Rate „ Promotion
Cost:Income Ratio

5
Making It Happen

„ LERN Program Planner Tool

Push
„ LERN Contract Training Tool Down
Data
„ LERN Segmenting Tool 80/20
Analysis

„ Web-Based Software

7 Primary
Market
Segments

Dig Deeper Total sales by market segment

„ Analysis by market segment involves taking each „ What is my top market segment by sales?
of your market segments or primary customer „ What are my four top market segments by sales?
audiences and analyzing sales. „ My four top market segments are what percent of
total sales?
„ M nextt three
My th ttop market
k t segments
t are what?
h t?
„ What market segments are growing faster than
others?
„ Are any market segments stable or declining in
sales? Do I keep them?

Average sales per customer Percentage growth

„ What is my top market segment by average sales
per person?
„ What are my poorest market segments by average
sales per person?
„ Do I keep targeting my poorest market segments?
„ What market segments are growing as a
percentage of total sales?
„ What market segments are declining as a
percentage of total sales?
„ What is my strategy for my segments that are
growing?
„ What is my strategy for those segments not
growing?

6
Market share/market potential By product

„ As market share, how are my market segments „ How are sales of each of
doing? my product lines by
„ As market potential, how are my market segments market segment?
doing? „ Are there new
„ What do I keep as my primary market segments? products/programs or
„ What do I drop as my primary market segments? product lines I need to
develop for a particular
„ What do I add as my primary market segments? market segment?

Top customers Be Data-Driven!

„ Who, by name, are my top 1,000 customers for

this year, by dollar sales?
„
„
Who, by name, are my top 100 customers?
Who, by name, are my top 10 customers?
Thank You!
„ I th
Is there anything
thi I wantt tto write,
it say or d
do ffor my
top customers? marsello@lern.org