Anda di halaman 1dari 10

Eleven Reasons Why Training and

Development Fails . . . and what you can do about it


By

Jack J. Phillips & Patti P. Phillips

During their more than 10 years as consultants to some of the worlds largest organizations,
Jack and Patricia Phillips have developed a unique vantage point within the training and
development community. They have assisted hundreds of organizations with measurement and
evaluation to bring accountability to their workforce processes. As a result, theyve examined major
training and development programs in all types of settings, conducted impact studies using a
comprehensive measurement and evaluation process and reviewed the success (or lack thereof) of
studies conducted by their clients.
Of the more than 400 impact studies the Phillipses have conducted or reviewed over the
years, some studies have shown positive results while others yielded a negative return on investment.
Along the way, they have observed repeat patterns of issues that inhibit or enhance results. Even
when a program is successful, issues may stand in the way of more impressive results. Collectively
through their clients impact studies, along with comprehensive evaluation, they have identified 11
reasons why training and development fails and provide a prescription for change.

Lack of Alignment with use of a consistent four-level concept (See

[1] Business Needs. A training


programs payoff comes from
the business measures that
drive it. Simply put, if a
Linking Needs Assessment chart). If we
accept this evaluation framework, four
corresponding levels of objectives and needs
assessment exist as well. Without the
training program is not business connection at Level 4, the
aligned or connected to a Attempting to program will have difficulty in
business measure, no credibly driving any business results
improvement can be
solve job One major telecom firm in the
linked to the program. performance USA faced this problem directly as
Too often, training is issues with they reviewed its corporate
implemented for the universitys major programs. A first
wrong reasons a trend,
training will not step to check for business alignment
desire or perceived need work when other was to connect core courses to some
that may not be factors such as business measure or need based on
connected to a business perceptions of the corporate
measure.
reward systems, university staff. When the staff could
Initial training needs job design and not readily make the connection, they
may be linked to the motivation are the determined the linkage did not exist.
objectives and The company needed a more detailed
evaluation through the
real issues. up-front analysis.

SEPTEMBER 2002 TRAINING 1 Reproduced with Permission


Linking Assessment with Evaluation
Needs Program
Assessment Objectives Evaluation
Business Impact Business
4 4
Needs Objectives Impact

Job Performance Application Application


3 3
Needs Objectives

Skills/Knowledge Learning Learning


2 Needs Objectives 2

Preferences Satisfaction Reaction 1


1
Objectives

[2]
Failure to Recognize Non- and motivation are the real issues. To
Training Solutions If the overcome this problem, training staffs must
wrong solution is implemented, focus on methods to analyze performance
little or no payoff will result. rather than conduct traditional training needs
Too often, training is perceived as a solution assessments a major shift in performance
for a variety of performance problems when improvement that has been developing for
training may not be an issue at all. A recent many years. Up-front analysis should be
evaluation of a leading U.S. banks major elevated from needs assessment, which is
training program illustrated this problem. In based on skills and knowledge deficiencies,
its training program, the bank attempted to to a process that begins with business needs
prepare the commercial loan officers and works through the learning needs.
(relationship managers) to sell products other
than commercial loans, such as the banks

[3]
Lack of Specific Direction
capital market products and cash
and Focus. Training and
management services. But the training
development should be a
produced little change in the managers
focused process that allows
behavior. An impact study subsequently
stakeholders to concentrate on desired
revealed that the culprit was the
results. Training and development
compensation arrangement. When probed
objectives should be developed at higher
for a reason for the poor results, the bankers
Kirkpatrick levels than traditional learning
clearly indicated that unless their
objectives (See Examples of Objectives
compensation system changed to account for
chart). These objectives correspond with six
the new product lines, their behavior would
measures that lead to a balanced approach to
not change. They will continue to sell only
evaluating trainings success (See A
the products on which their commissions
Balanced Approach chart). Most training
were based.
programs should contain objectives at
Attempting to solve job performance
multiple levels, ideally including those at
issues with training will not work when
levels 3 and 4.
factors such as reward systems, job design,

SEPTEMBER 2002 TRAINING 2 Reproduced with Permission


Examples of Objectives
Objective Level
1. Decrease error rates on reports by 20%. Business Impact (4)
2. Increase the use of counseling skills in 90% of situations where work habits Application (3)
are unacceptable.
3. Achieve a post-test score increase of 30% over pre-test. Learning (2)
4. Initiate at least three cost reduction projects. Application (3)
5. Decrease the amount of time required to complete a project. Business Impact (4)
6. Achieve a 2:1 benefit to cost ratio one year after program implementation. ROI (5)
7. Receive a job relevance rating from participants of at least 4.5 out of 5. Reaction (1)
8. Increase the external customer satisfaction index by 25% in 3 months. Business Impact (4)
9. Handle customer complaints with the 5-step process in 95% of complaint Application (3)
situations.
10. Achieve a job simulation test score average of 75. Learning (2)
11. Conduct a meeting with direct reports to set performance improvement Application (3)
goals.
12. At least 50% of participants use all customer interaction skills with every Application (3)
customer.

When developed properly, these program when we cannot define the


objectives provide important direction and behavior expected from participants and the
focus for a variety of stakeholders at subsequent business impact driven by the
different time frames. For designers and program?
developers, the objectives provide needed While not all programs should undergo
insight to focus on application and impact, such detailed up-front analysis, it is a critical
not just learning. The facilitators need issue that needs more attention.
detailed objectives to prepare individuals for

[4]
the learning experiences ultimate outcome: The Solution is Too Expensive.
job performance change. Of course, a training and
Participants need the direction provided development programs ROI
by level 3 and 4 objectives to clearly see might ultimately fail to recoup its
how the training programs outcome will high costs. Its important to note, however, that
actually help the organization. Sponsors of a negative ROI is not always a sign of failure.
training and development, the key clients Many programs might add enough perceived
who pay for the program and support it, value through intangibles and significant short-
require such objectives to connect training term behavior change to overcome negative
with important business-unit measures. ROI. If positive ROI is expected, however, then
Finally, evaluators use this type of direction negative ROI shows failure and is unacceptable.
to know what data to collect to determine One large banks executive leadership
whether the program has been successful. development program, for example, offered an
Recognizing the importance of multiple impressive design from a learning perspective
objective levels, including business impact, and included project assignments for
a vice president of corporate training and participants, mentors, and learning coaches.
development at a major package delivery Unfortunately, the program proved too
company recently posed an important expensive for the monetary value that it added,
question to the organization: How can we even after multiple years of providing benefits.
expect our management team to support a When the full cost of conducting the four-week

SEPTEMBER 2002 TRAINING 3 Reproduced with Permission


A Balanced Approach to Measuring Trainings Success
Measure 1: Participant Reaction To, Satisfaction With, and Planned Action Connected to the Training

Measure 2: Participant Learning Improvement (Changes in Knowledge and Skills)

Measure 3: Application of Knowledge Learned and New Skills on the Job

Isolate the
Effects of
Training & Development

Measure 4: Business Impact Directly Linked to the Training

Measure 5: Return on Investment Comparing Monetary Benefits to Costs

Measure 6: Intangible Benefits Linked to the Program

The ROI Process

The process used to capture the success of training and development programs develops six types of data about a specific program (See A Balanced
Approach chart). These six types of data provide a balanced, credible approach to identifying the success of a training and development program. The first four
types of data are consistent with the traditional Kirkpatrick levels. The process also requires a specific method or technique to isolate the impact of the program.
This critical step answers the question, How do you know it was the training? This comprehensive process provides the complete profile of training success.
At the heart of the process is a step-by-step model that shows how the data are collected, processed, and analyzed (See The ROI Process Model chart). The
process starts by developing evaluation plans to collect data and make decisions regarding how the data are processed and analyzed. During the program, reaction
and planned actions are captured from participants. Learning is captured as specific improvements in skills, knowledge and perceptions are measured.
After the program is implemented, application and implementation data are collected which show the use of the skills and the application of what was
learned in the training program. Next, the corresponding business impact, which is directly linked to the training and development program or solution, is
measured. Together, these blocks in the process model comprise the key elements of data collection.
The next set of blocks in the process model comprises the ROI analysis. The first task is to isolate the effects of learning from other influences. This
process uses one or more methods to separate trainings influence form other factors that had an impact on the business measure. Next, business impact data are
converted to monetary value and annualized to present an annual value for the training. One-year values are used for short-term solutions; longer periods are used
for more extensive, long-range implementation.
The fully loaded costs are captured to reflect both direct and indirect costs of the training solution. Monetary benefits and the costs are combined in the
calculation of the actual ROI. Intangible benefits are identified throughout the process and are tabulated after an attempt is made to convert them to monetary
value. If an intangible item cannot be credibly converted to monetary value, it is left as an intangible measure and becomes the sixth type of data. This
comprehensive measurement process is now being used by thousands of organizations in all types of settings, including public sector and nonprofit organizations.

SEPTEMBER 2002 TRAINING 4 Reproduced with Permission


THE ROI PROCESS
Calculating the Return on Investment of
a Business Performance Solution
Tabulate
Costs
Of Solution

Evaluation
Planning Data Collection Data Analysis Reporting

Level 1: Reaction, Level 3:


Satisfaction, and Application/
Planned Actions Implementation
Develop Develop Collect Collect Convert Calculate
Generate
Objectives Evaluation Data During Data After Isolate Data To The Return
Impact
Of Plans And Solution Solution The Effects Monetary On
Study
Solution(s) Baseline Data Implementation Implementation Value Investment

Level 5: ROI
Level 2: Level 4:
Learning Business Impact

Identify
Intangible
Measures

Intangible Benefits

training sessions in major cities around the This issue raises the question of what
world were tallied along with the costs of costs should be included in the analysis.
consultants, personal learning coaches for each Actual costs are traditionally included in an
executive, and costs for the design, impact study, although some of them are
development, and facilitation teams, the total indirect and might not be visible or
reached almost $100,000 per participant. The contained in a particular cost statement (See
program was unable to deliver what Recommended Cost Categories). Too
management expected. often there is a tendency to use only direct
costs or even to minimize them to a certain
Recommended Costs for Training extent.
Programs The good news is that many effective
Assessment Costs (Prorated) learning solutions can be implemented with
Development Costs (Prorated) inexpensive processes and still drive
Program Materials business results. For example, a sexual
Instructor/Facilitator Costs harassment prevention workshop conducted
Facilities Costs at a hospital network cost each participant
Travel/Lodging/Meals (supervisor and managerial level) $424. The
Participant Salaries and Benefits ROI was 1,052 percent. It is possible.
Administrative/Overhead Costs (Prorated)
Evaluation Costs

SEPTEMBER 2002 TRAINING 5 Reproduced with Permission


[5]
Regarding Training as an under fire. But, we often fail to focus on the
Event or a Series of Events. participants role in the process.
A positive business impact A recent impact study involving a
must come from an individual technology firm based outside the United
participants behavior change, and such States revealed that several leadership
change does not come easily. When training programs designed for employees at various
is considered a single event, such as organizational levels failed to yield the
attending a two-day workshop for example, expected results. A major barrier was
the odds of changing behavior are slim. noticed: Each group of participants
Without behavior change, training fails to continually identified lack of support from
generate business results. immediate managers as the problem.
One major physician malpractice Ironically, each level blamed the next. The
insurance provider offered various training
seminars to help physicians adjust their Barriers to Transfer of Training to the
approach and behavior regarding certain Job
medical procedures. Traditionally, the Immediate manager does not support the training.
The culture in the work group does not support
programs were offered in four-hour or full- the training.
day programs, with no pre-work and no No opportunity to use the skills.
follow-up reinforcement. Not surprisingly, No time to use the skills.
the seminars changed few, if any, behaviors. Skills could not be applied to the job.
Sometimes it may be helpful to consider The systems and processes did not support the use
of the skills.
behavior change as bodybuilding. An Didnt have the resources available to use the
occasional visit to the gym will have little skills.
impact on the body. Yet, a continuous Changed job and the skills no longer apply.
process of working out, along with the Skills are not appropriate in our work unit.
proper motivation and support, will make it Didnt see a need to apply what was learned.
happen. Could not change old habits.
Reward systems dont support new skills.

[6]
Participants are Not Held
Accountable for Results. For CEO commented that somewhere along the
training programs to be chain of authority a person must accept
successful, participants must responsibility and make things happen.
individually drive performance change. Participants can succeed with training if
When pressed for reasons for not changing they are properly motivated to do so and are
behavior, participants are quick to blame held accountable for their results, even with
others, usually the boss. But that may not be an unsupportive manager. Traditionally, the
the real issue. Of individuals most likely to participants role in a training program has
be held responsible for results including been limited to attendance learning the
managers, trainers, developers, and senior skills and knowledge being offered. At
executives the overlooked participant times, they may even be required to apply
deserves more attention. Participants often the newly acquired skills on the job.
dont see changing their behavior as their But participants should not only apply
responsibility. Historically, when results are what is learned, they also should ensure that
few, the training and development staff, doing so will reflect business results. While
along with immediate managers, comes this creates additional expectations, the
participants role is elevated from learning

SEPTEMBER 2002 TRAINING 6 Reproduced with Permission


[8]
to actually achieving results and reporting Lack of Management
them to the training and development staff. Reinforcement and Support.
This shift is accomplished by developing Without management
expectations into learning solutions, encouragement and support,
providing handouts that detail specific participants will rarely implement new skills
expectations and defining the roles of and knowledge in the workplace. The
employees in various handbooks, employee managers role, therefore, is critical in the
manuals, and orientation sessions. learning process. Most studies have shown
Participants must understand that the that the two most powerful opportunities for
programs success rests largely with them, managerial input occur during the
and disappointing results may be their interaction with the learner prior to the
responsibility. training solution an after the training has
been completed.

[7]
Failure to Prepare the Its clear that managers usually dont
Environment for Transfer. realize their influence. This disconnect is
Regardless of what participants most frequently identified in follow-up
learn from a training program, surveys conducted as part of an impact
without transferring it to the job, study. More action must be taken to ensure
performance will not change and the training mangers understand their impact and how
program will fail. This training-transfer they can make changes.
problem has been an important issue in At one major computer manufacturer,
training and development for decades. participants were asked specific questions
Unfortunately, studies continue to show that regarding the actions and performance of
between 60 and 90 percent of what is their managers following training. By using
learned isnt applied on the job. The reason multiple-choice responses, the survey
this occurs is complex, involving many essentially listed the same questions but
different barriers (See Barriers of Transfer reworded the choices from the perspective
chart), to which little attention is given until of the group being surveyed. The results
its too late.
The results: Its clear that managers usually dont realize their
Barriers kill the
success of an influence. More action must be taken to ensure
otherwise managers understand their impact and how they can
successful make changes.
program.
Barriers must be understood at the showed a tremendous disconnect. Some 40
beginning of the process as part of percent of managers said they encouraged
needs assessment and analysis. Identified and coached their employees with the
early, inhibitors can be addressed in the training, while the participants indicated that
solutions design, development, delivery, 0 percent actually provided encouragement
and implementation. Efforts to minimize, if and coaching.
not eliminate, the barriers before the The problem often exists in managers
learning sol is implemented will pay off perceptions about reinforcement and
significantly. support. Some managers feel that since they
created an empowered environment for
employees, they should not have to probe

SEPTEMBER 2002 TRAINING 7 Reproduced with Permission


further into each learners application of this technique often, but only one-third of
new skills or knowledge. Learning new our studies contain this type of arrangement.
skills is a different situation. A new process For the remaining studies, another technique
implemented in the workplace, particularly must be used to pinpoint trainings impact.
one involving a significant departure from Recently, various techniques have evolved
pervious approaches, requires the immediate to estimate the connection between the
managers support. A simple inquiry about training and business improvement (See
the training programs success and how it Techniques to Isolate).
will be implemented into the work unit is
often sufficient.
Organizations offering support have Techniques to Isolate the Effects of the
tackled this process by developing Training Program
management reinforcement modules for a
particular program, defining managers Use of control groups
support roles, conducting workshops to Trend line analysis
show managers their specific roles, holding Forecasting methods
managers accountable through their own job Participants estimate of impact
descriptions and responsibilities, and (percent)
rewarding managers for doing it right. Supervisors estimate of impact
(percent)
Managements estimate of impact

[9]
Failure to Isolate the Effects (percent)
of Training. Too often, Use of experts
training programs are Subordinates report of other factors
conducted, business measures Calculating/Estimating the impact
are monitored, and improvements are of other factors
credited to the training process alone. The Customer input
assumption is the training program
improved the business. Actually, other
influences and processes may have The good news is that at least one of
influenced the business measure. The these techniques identified in the chart will
challenge is to isolate the improvement work in every setting, and the issue can be
directly related to training. addressed in every impact study. To show
Failure to attempt to isolate trainings trainings real value, designers, developers,
contribution might cause some training pros and evaluators must accept the challenge to
to be discarded as irrelevant. Such pros may tackle this issue.
actually bolster the bottom

[10]
line, but if there is no attempt to isolate their Lack of Commitment
impact, executives and sponsors are puzzled and Involvement From
about the actual connection to business Executives. Without top
improvement. executive commitment
No doubt this is perhaps the most and involvement, training and development
challenging issue. The classical approach is will be ineffective and major pros will fall
to compare a group that has received the short of expectations. Commitment is
training to a group that has not, and let the critical, which equates to resources being
difference in the two groups represent allocated to the training and development
trainings impact. We have attempted to use function and its specific pros. Involvement

SEPTEMBER 2002 TRAINING 8 Reproduced with Permission


Communication: Common Target Audiences
Reason for Communication Primary Target Audience Level of Data
Secure approval for program Client, top executives 4, 5
Gain support for the program Immediate managers, team 3, 4, 5
leaders
Build credibility for the training staff Top executives 4, 5
Enhance reinforcement of the program Immediate managers 3, 4
Enhance results of future programs Participants 1, 2, 3, 4
Show complete results of the program Key client team 1, 2, 3, 4, 5
Stimulate interest in training programs Top executives 4, 5
Demonstrate accountability for client All employees 4, 5
expenditures
Market future training programs Prospective clients 3, 4, 5

includes the actual presence and actions of adjustments should be made to delivery, and
individual executives in the process. clients need feedback on a programs
Business literature is laced with success. Without such feedback, a program
examples of top executives taking active may not reach expectations.
roles. Andy Grove, chairman of Intel, sees The challenge is to provide a stream of
training and development as one of his key information, as data are collected, to a
responsibilities. Jack Welch, former variety of audiences (See Common Target
chairman of GE, devoted a prescribed Audiences chart). Sharing evaluation data
number of days per month at the GE from Kirkpatricks levels 1-5 can help refine
management development center in New the training process. Reaction data and
York. Bit Gates, Microsoft chairman, learning data can improve learning design
conducts a portion of the orientation for new and facilitation. Application data should be
employees as part of a rotating assignment provided to those individuals implementing
with senior executives. the pros so that adjustments can be made.
Active roles by senior managers are And business impact data must be shared
critical and can be accomplished in many with clients and others so that the entire
ways ranging from minimal participation group can understand the value.
to increased involvement in which specific Most importantly, the results may be
days are allocated to teaching. When used to make adjustments in the design,
executives take a very visible role, others development, and delivery of the program.
will do the same. This attitude filters The routine communication of data serves as
throughout the organization and makes a big a process improvement in making a
difference. successful program more successful.

Conclusion

[11]
Failure to Provide
Chances are, these 11 issues sound
Feedback and Use
familiar. Organizations must address them
Information About
all if training is to live up to expectations
Results. All stakeholders
and generate appropriate returns on
need feedback. Employees require feedback
investment. With increased pressure to
on their progress, developers and designers
show the payoff of the investment in
need feedback on program design,
learning and education, failure cannot be
facilitators need feedback to see if
tolerated. Failure can be prevented. Using

SEPTEMBER 2002 TRAINING 9 Reproduced with Permission


training and development results means worlds largest organizations. He has written
more than simply collecting data. The over 12 books on the subject. Phillips can
results-based training process must be be reached at jack@roiinstitute.net.
examined if training and development is to PATRICIA P. PHILLIPS is CEO of ROI
be successful and respect in an organization. Institute, an international consulting
company focused on the implementation of
the ROI Process. She has provided
The Authors
consulting services and support for the ROI
JACK J. PHILLIPS is with ROI process for several years and has served as
Institute. Phillips developed and pioneered co-author on the topic in several
the utilization of the ROI process and has publications. She can be reached at
provided consulting services to some of the patti@roiinstitute.net.

SEPTEMBER 2002 TRAINING 10 Reproduced with Permission