Effective Reaction Evaluation in Evaluating Training Programs

Purposes and Dimension Classification
by Sung Heum Lee, PhD and James A. Pershing, PhD
he evaluation scheme that many corporate training programs use is Kirkpatrick's four-levels of evaluation: reaction, learning, behavior, and results (Kirkpatrick, 1994). However, surveys of the evaluation of corporate training programs show limited application of the levels other than at the reaction level (Alliger & Janak, 1989; American Society for Training and Development, 1996; Brinkerhoff, 1989; Dixon, 1990; Industry Report, 1996; Parker, 1986; Plant & Ryan, 1994). Training participants' reaction is the most commonly used criterion for determining the effectiveness of corporate training programs. Most corporate trainers evaluate their training programs by using a sim-

T

pIe end-of-course reaction form, often referred to as a "happy sheet" (Plant & Ryan, 1994), a "smile or whoopie sheet" (Robinson & Robinson, 1989), an "end-of-event questionnaire" (Bramley, 1996), or a "reactionnaire" (Newby, 1992). Based on a recent survey of corporate training programs using Kirkpatrick's four-levels of evaluation (American Society for Training and Development, 1996), only 4.3% of the organizations surveyed measured results, 13.7% measured behavior change, 27.9% measured learning, and 88.9% reported using participant reactionnaires. These findings indicate that the majority of the organizations evaluated the reactions and opinions of their training participants

immediately upon completion of training programs.

Reaction Evaluation of Training Program
Purposes The main purpose of reaction evaluation is to enhance the quality of training programs, which in turn leads to improved performance. The ultimate objective is to make training programs more efficient and effective for organizational performance improvement. Reaction evaluations are a type of formative evaluation when the results are used for program modification and the redesign of content, course materials, and presentations (Antheil & Casper, 1986; Robinson & Robinson, 1989). Generally, they collect information that

32

Performance Improvement.

SEPTEMBER1999

This is crucial information. The evaluation of a training program. expansion. or quality To make decisions about a program's future or adoption Program impact Post-tests. Summative Evaluation To determine program's worth. not to trainee. Keller. instructor. 1986. & ever. Therefore. modification. the goal of reacrevision. or adoption tion evaluation is to deterFigure 1. training programs is to evaluate a training program. Formative Evaluation Purpose Use Focus Tools Time Audience User Major Characteristics To determine program's worth. merit. Basic Differences Between Formative and Summative Evaluation. Both forand applicable presents a formidable problem for corpomative and summative evaluations are essential because rate training evaluation practitioners (Mattoon. 1991) and to make decisions Dimensions to Evaluate concerning program continuation. 1990. He indicates for reaction evaluations and for the rigorous design and that ideal reaction evaluations provide the maximum development of reactionnaires. they may not use what they have learned and will probably advise others not to attend the training program. howabout a program's worth or merit (Worthen. 1997). training materithe instructor alone. Volume 38 • Number 8 33 . effectiveness. Sanders. Worthen & In some organizations the primary purpose of reaction evalSanders. presents a few sample reaction forms but does not suggest This difference calls for the careful selection of dimensions guidelines for selecting reaction dimensions. Reaction questionnaires amount of information and require the minimum amount of should be designed to supply valid and reliable information time to complete. 1992). Summative evaluations provide program uation focuses on the course instructor or facilitator decisionmakers and potential customers with judgments (Phillips. expansion. value. questionnaires. timing and use determine whether an evaluation is forma1991). supported by internal evaluators in unique cases Convincing information decisionmaking for observations. or Fitzpatrick. Judgment for program continuation. Kirkpatrick (1994. 1987). The kinds of questions to be addressed in to program evaluators. In Diagnostic for program modification. be linked with the Performance Improvement. 1997). During training program Program designer or team Primarily internal evaluators. Identifying and ferent uses of the evaluation results. termination. but in how. 1987). mine the value. or quality To improve training program and correct errors Program process Tests.is specific enough to help make revisions and improvements in the training program. Consequence such cases. instructional strategies. Worthen & Sanders. is a judgment about the quality. a reaction evaluation will. Figure 1 summarizes the and make constructive feedback about what they do rather basic differences between formative and summative evaluathan who they are. however none of the citations preterminated. or redesign termination. of course. interviews After training program Stakeholder or potential consumers External evaluators. and even the training facilities. If participants are not satisfied with the training experience. or adoption (Worthen. when. interviews observations. supported by external evaluators Timing and control for program improvement Reaction evaluations can also be summative in nature. or efficiency of a training program (Smith & Brandenburg.1996) Bloomer. not instructors or facilitators (Cangelosi. tive or summative. impact of the training activity. The responsibility of an evaluator of tion. and by whom it is used (Beer & In writing about level 1 evaluation. value. there are few differselecting reaction evaluation dimensions that are valid ences in how trainers collect and analyze the data. Although these two distinctive roles call for difsent a comprehensive set of dimensions. The evaluation of training programs can play either a formative purpose to improve the program or a summative purThere are a number of different dimensions for training pose to decide whether a program should be continued or reaction evaluations. effectiveness. 1991). The main difference is not in the information. questionnaires. decisions are necessary during the developmental stages of Guidelines for reaction dimensions can help practitioners a training program to improve it and-when it has been stadesign useful reaction evaluations for program modificabilized-to judge its final worth or determine its future. The tion. the reaction evaluation of a training program should relate to an instructor's Any aspect of a training program can be evaluated: the instruction and the impact of the training program. merit. 1996. value. Reaction evaluations provide program designers with insights about participants' degree of satisfaction with a program's design and implementation. Instructors are far more likely to accept als. termination. and improvement (Grove & Ostroff.

level. Robinson and Robinson (1989) indicate that reaction evaluations should include some questions that are specific to the particular program being evaluated. 1989. Instructional materials include published and unpublished print materials such as textbooks. gaps in content. 1993). trainers. comprehensive dimensions for reaction evaluations can be summarized as follows: • Program objective(s)/content • Program materials • Delivery methods/technologies • Instructor/facilitator. allowing for more precise information about a program's content and process. discrepancies are bound to occur. 1993. and relevance to the job or to intended changes. and the learner's belief as to the overall effectiveness of the event. SEPTEMBER1999 . and reaction evaluations can identify these weaknesses. Other areas incorporate logistical concerns. program relevance to job/work area. However. the use of media. program value. content. Keller. manuals. program materials. Program designers select procedures. During the design and development stages of training program materials. Some reaction forms might be very simple. Typically. Considerations include how well the training materials matched the real world of the trainee. etc. Faerman & Ban. communication medium. efficiency. or decisionmakers (Payne. or textbooks-performed for participants. such as the size and comfort of the room and the tests or other performance measures (Wart. Based on the results of an extensive literature review on reaction evaluations. The results of material evaluation can be used to revise the training materials and to make the materials as effective as possible (Dick & Carey. individualized instruction packages. and recommendation for program improvement (Basarab & Root. helping to improve future programs. 1992. Cayer. In addition. 1993). duration. methods. and value (Hellebrandt & Russell. handouts. & Cook. Answers can verify the consistency of the materials with the program objectives. Phillips (1996) enumerates the most common dimensions of reaction evaluations as being program content. such as the quality of classroom environment. resources. 1994). trainer. (1995) suggest some guidelines for selecting dimensions of reaction evaluation. participatory materials. and methods that are relevant to the training objectives. quality of materials. training environment/facilities. The most important concept associated with program content is that of a performance objective. adaptability. and general comments. Sanderson (1995) advocates dimensions such as the participants' opinion of the precourse briefing. or approaches. reactionnaires inquire about participants' reactions to and interest in the usefulness ofthe program content. instructor. instructional activities • Program time/length • Training environment • Planned action/transfer expectation • Logistics/administration • Overall evaluation • Recommendations for program improvement Program Objective(s)/Content. There are areas to assess during a reaction evaluation. laboratory manuals. The selection of training objective(s)/content depends on the purposes of the training program and is largely a judgment procedure (Tracey. and timeliness of the content presented can be judged by the participants' reactions. The reaction questions should consider how well training materials-tutorial guides. Forsyth et al. and any conflicts in concepts and terminologies used. level of difficulty. effectiveness of the instruotorls). program coordinator/facilitator. while others might be detailed and require a considerable amount of time to complete. Designing a training program starts with these factors. technological materials. They also suggest that the reaction evaluation of a training program should not only focus on the program itself. such as objectives. whether the content was organized into manageable amounts. the trainer. developers should make every effort to avoid unnecessary duplication of content. and planned improvements. manipulable materials. Schouborg.information requirements of program designers. whether the 34 Performance Improvement. trainee. administrative details. various delivery methodologies. objectives. quality of the program materials. and delivery methods. The areas of feedback used on reactionnaires should be directly tied to the nature and scope of the training program and the purposes of the evaluation. ease of use. Sample reaction questions for objective(s)/content dimensions are as follows: • Did the program content meet the stated objectives? • Were the program topics effectively sequenced? • Was the program content up to date? • Was the course content at an appropriate level of difficulty? • Was the course content practical? Program Materials. The purpose of evaluating the training materials is to determine their effectiveness. Robinson & Robinson. 1996). but ask more general questions about whether the training participants feel that they will be able to transfer what they have learned to the work environment and whether the organization is ready to support new skills. the course materials. class handouts. 1993). 1996. facilities/accommodations. The content of a training program should be identified with recognition of some significant variables. the appropriateness. and helpfulness. Program materials are the objects the trainer and instructor use in the training environment. overall evaluation. 1992). A performance objective is a detailed description of what trainees will be able to do when they complete a training program. content. techniques.

Sample questions about the instructor/facilitator dimension for reaction evaluation are as follows: • Did the instructor present content clearly? • Was the instructor responsive to participants' questions? • Was the instructor well-prepared? Instructional Activities. 1995). there are 60 standards covering preparation. and course evaluation. As a manager of the training situation. and constraints before selecting methods or technologies. gaining participation. Seels & Glasgow. the training situation. to group-based activities. objectives. the instructor/facilitator is one of the key components of an effective training program. 1992). as well as each aspect of the total instructional activity. explained concepts. programmed instruction. to multimedia-mediated instruction. Reaction questions should also cover how the instructors interpreted and used the training materials and whether they presented materials in a way that was stimulating. Based on the performance standards for instructors (Powers. laboratory. visual. Performance standards for instructors are the backbone of instructor excellence. platform skills. materials. & Stevens. After using delivery methods such as audio. demonstration. They must identify trainee characteristics. to ensure content understanding and performance change. and helpful. Sample questions for the evaluation dimension of delivery methods/technologies are as follows: • Were the audio learning aids helpful? • Were the presentation technologies used in class effective? • Were the visual aids helpful? InstructorlFacilitator. skills. The selection of instructional activities for a training program has significant implications for course management strategies. Choices of delivery methods/technologies are based on selection criteria such as whether the delivery methods are appropriate for the trainee. and technology-based instruction (Davies. 1996). Physical separation from the workplace distinguishes classroom instruction from on-the-job training. 1992). Sample questions for program materials are as follows: • Were the materials consistent with the training objectives? • Were the program materials of high quality? • Was the level of difficulty of the materials appropriate? • Was the content of the handouts easy to understand? Delivery Methods/Technologies. An instructional activity is a set of structured experiences designed to help trainees achieve one or more training objectives. The designers of training programs strive to be effective in creating each element of classroom instruction. Training action begins with this person. Jolliffe. content and sequencing..sequence was from simple to complex and from concrete to abstract. Volume 38 • Number 8 35 . and tutorials. and the quality of any performance tests or examinations. or multimedia for a training program. & Stevens. Jolliffe. training aids. Heinich et al. Evaluation questions revolve around the instructor's ability to interact with the learners and his or her ability to deliver the training content in a meaningful way. 1995). Developers should consider several factors in selecting delivery methods/technologies that will help trainees reach objectives. 1981. designers adhere to many instructional principles derived from learning and instructional theories (Yelon. 1994). 1999. 1990). cooperative learning group. simulation. Various instructional activities can take place in a classroom. To evaluate the appropriateness and helpfulness of instructional activities. particularly for the use of class time (Dick & Carey. and equipment selected for a training program (Tracey. written assignments. self-instruction. Consideration should be given to whether the instructor encouraged active participation through the use of examples and illustrations. Several of these standards can be assessed using reactionnaires. and attitude and be successful in using the strategies. instructional staff variables can be one of the more important factors in attempts to account for variance in program outcomes and to distinguish a program's success. 1992). the evaluator can evaluate the appropriateness and helpfulness of the delivery methods in helping learners understand the content of a training program. the evaluation might ask questions such as the following: • Were the course exercises relevant to the program objectives? Performance Improvement. The instructors must possess the required technical and pedagogical knowledge. training objectives. tutorial. The instructional designer determines method/technology options to achieve the objectives of a training program. Group teaching distinguishes classroom instruction from individualized instruction. 1992). Another important consideration is the degree of trainee involvement in the training activity. and enthusiastically answered questions (Forsyth. participant evaluation. Classroom instruction has two distinct attributes: the teaching of groups of trainees and the physical separation of the classroom from the workplace (Yelon. and instructional constraints. and whether the training materials were presented in a way that was both interesting and stimulating (Forsyth. Determining whether the delivery methods will help trainees reach the stated objectives is an important issue in selecting appropriate delivery methods/technologies for different types of objectives (Dean. role play. interesting. The designer can choose from delivery methods such as lecture. In this sense. questioning techniques. problemsolving. To be effective in using instructional activities to enhance job performance. case study. ranging from listening to the instructor. discovery. discussion. drill and practice. Reaction evaluation of program materials should also include gathering data regarding the relevance of reading materials. game.

Positive transfer is highly contingent on factors in the trainee's work environment. developers can use program time/length as an evaluation dimension. facilitator or coordinator. and the location of the placers) where learning occurs (Tessmer & Harris. patterns of work. travel arrangements. or car. McVey. how well pretraining enrollments are executed. room temperature. division of duties and responsibilities. lighting. Broad and Newstrom (1992) report that there is a positive relationship between favorable organizational climate and management support of training and the participants' ability to apply classroom learning to the work environment. From this point of view. the evaluators of a training program can assess the length of sessions and/or entire training program and use the results for schedule changes and considerations of overall program length. 1996). Time-on-task and the efficient use of time are important in planning a training session. To ensure quality programs for performance improvement. office. To improve future training programs. it may be more likely that they will implement what they learn (Sanderson.• Were the group discussions helpful to participants in exchanging ideas with each other? • Was the homework helpful in understanding the course content? Program Time/Length. social and special events. effectiveness of the instructor. Sample questions for the training environment dimension are as follows: • Was the training environment appropriate for the learning? • Were the environmental conditions-comfort. and program procedures and policies (Miringoff. and extracurricular activities associated with the program. reaction evaluation should include questions on planned actions and anticipated organizational barriers. and how well the program is managed once underway. 1993. The logistics and administrative sides of program planning are important. The quality of large training programs depends on how well the objectives and content of the program are marketed. 1988). heating. the program evaluator could ask participants about their plans and expectations for applying the content of the program when they return to their jobs. When evaluators are considering questions in this area. seating arrangements. Environmental psychologists recognize the environment as a persistent and powerful influence on human learning and behavior. Too little time or too much time can negatively affect training effectiveness. Using this dimension. they need to ask specific questions regarding learning space. Peterson & Bickman. This dimension of reaction evaluation is used to measure the participants' overall reactions about the usefulness of the course content. 1988). living room. To find and remove the barriers for planned action and transfer of training content. These questions function as a type of administrative audit that assesses administrative aspects such as personnel practices. 1980. SEPTEMBER 1999 . sleeping accommodation. To help the participants implement the results of the program on the job. Sample questions for this dimension of reaction evaluations are as follows: • Was the scheduling for this course efficiently administered? • Was the process of registration for this course easy? • Was the assistance with extracurricular activities helpful? Overall Evaluation. the conditions. acoustics. The place could be a classroom. kinds of activities in which people are engaged. and special events such as registration procedures. reaction evaluations can include questions about operations. arrangements. Questions related to this fact would focus on understanding and awareness of ergonomics as applied to the logistics and physical adequacy of the training environment (Faerman & Ban. Training participants' perceptions of classroom environments can have a significant influence on both cognitive and affective learning outcomes (Haertel & Walberg. 1995). accessibility. Sample questions regarding the planned action/transfer expectations dimension for reactionnaires are as follows: • Was the training content relevant to your job? • Do you expect the organization to support your use of the skills learned in this program? • What factors will encourage job transfer of the training content? • What factors will inhibit job transfer of the training content? Logistics/Administration. and overall flexibility in terms of training event demands. adequacy of the learning environment. An understanding of logistical and administrative support undergirds the effective team-building effort that is necessary in conducting successful programs. 1992). The facilities of the learning environment include the furnishings. Sample questions for the program timellength dimension are as follows: • Was the amount of time in the program sufficient? • Was the length of the program appropriate for program objective(s)? • Was there enough time for practice of course content? Training Environment. If participants have to report to their managers about their training experiences and their intended transfer actions. planned action/expectation for job trans- 36 Performance Improvement. study facility-meet your needs? • Was there enough workspace for class activities? Planned Actionlfransfer Expectation. visibility-conducive to learning? • Did the arrangements-food. 1988). noise. computer lab. measuring participants' perceptions regarding the likelihood of their being able to transfer training content to the work environment may be particularly important (Baldwin & Ford.

recognized as "customer satisfaction. but also when they should be assessed. but it takes longer to It is a common mistake for a training department to create summarize the results. ing more effective in meeting customer requirements. Sample questions for recommending one reaction evaluation for all its training programs. • What would you suggest to improve the training program? Reaction evaluation. (1996). including media/technologies To rate the ability. and usefulness of written material and other aids To judge the appropriateness and effectiveness of delivery methods. G. 1996). or feelings about a specific training program are complex. training location. as specified in the participants to express their own thoughts without being purpose of reaction evaluation. 11\ Each participant's reactions. preparation.and/or out-of-class activities To assess the length of session and/or entire training program for schedule change and considerations of program length To evaluate the adequacy of the physical training environment. & Janak. including classroom. E. and timeliness of the program content To determine the effectiveness. Sample questions of this dimension for reaction evaluation are as follows: • Was the overall instructional environment conducive to learning? • Was there enough time to cover the program content? • Did the training program meet your intended needs? • Would you recommend this training program to others? Dimension Program Objectives/ Content Program Materials Delivery Methods/ Technologies InstructorlFacilitator Instructional Activities Program Time/ Length Training Environment Purpose To evaluate the program objectives with participants' expectations and the appropriateness. the training context. and effectiveness of the trainer or facilitator in leading the program To evaluate the appropriateness and helpfulness of in. structure. efficiency. At least program improvement are as follows: a portion of each evaluation should be specific to the program it is designed to evaluate (Robinson & Robinson. With appropriate dimensions. "Kirkpatrick's levels of trainee's perceived success at achieving some of the goals of training criteria: Thirty years later.fer. it is best to use a series of open-ended questions that allow the aspects of these reactions are of interest. level. attitudes. This approach can produce very helpful information for program improvement and decisionmaking for future training. Dimensions of Reaction Evaluation. 1989). (1989). Questions Transfer Expectation applying the content of the training program on the job in this dimension collect useful information for conTo evaluate the smoothness and effectiveness of the scheduling. When the Recommendations for To receive suggestions/recommendations for improving similar or future evaluator wants more sponProgram Improvement training programs taneous feedback about participants' attitudes toward Figure 2. They are the results References of many factors. the training program. Logistics! tinuous quality improveregistration. reaction evaluation can be a more useful and valuable tool in the evaluation of training programs and perConclusion formance improvement programs in general." is being used in the best-practice companies to make train• Please make any comments for changes that would improve the program." Personnel Psychology.. dining room. Therefore. 1992). other trainees. such as training content and methods. lodging. and other logistical and administrative matters Administration ment of the training program and provide an open forum To determine overall participant satisfaction and feelings about the training Overall Evaluation for the participants to share program their opinions.A. some being transitory in American Society for Training and Development. forced into a set of choices (Keller. evaluators must decide not only what Performance Improvement.M. training (Patrick. nature. and leisure facilities Recommendations for ProPlanned Action! To evaluate the participants' plans/expectations and anticipated barriers for gram Improvement. Volume 38 • Number 8 37 . ventions for improving human performance. 331-342. and quality of materials for the training program. throughout the training program. The idea of selecting dimensions for the reaction evaluation of A total of 11 dimensions and their purposes are summarized training programs also can be applied to evaluate other interin Figure 2. Participant reactions may vary 42(2). and the Alliger.

Molenda. Payne. 129-137. D. & Root. (1991).R. 294-312. L. (1986).. & Carey. New York: HarperCollins. MA: Kluwer Academic. 38 Performance Improvement. & Russell. (1992). Kirkpatrick.L. K.. 335-345. J. (1994). Evaluating New York: Longman. Davies. G.N. Heinich. J." In D. J. San Francisco: Berrett-Koehler. Russell.). Houston." In R. Craig (Ed. M..L. FL: Krieger. D. VA: Author. C.. Keller.. Hinrichs (Eds. J. Hellebrandt. handbook. (1995). Brinkerhoff. 45-61. S. --. "Assessing social-psychological classroom environments in program evaluation. 8(4).M. (1997).D. (1992). Dick. 29-55. I. D.W.T. (1992). London: Academic. I. J.performance (2nd ed. NJ: Prentice-Hall. J. (1992). Developing human resources.A. (1988).). RO.C. San Diego: Pfeiffer. N. Instructional McGraw-Hill. 32(6). (1988). B. A." Innovative Higher Education. Reading. Jolliffe. J. Training evaluation Diego: Pfeiffer. Training: Research and practice. The systematic instruction (4th ed." Performance &Instruction. "Using evaluation to transform training. "Trainee satisfaction and training impact: Issues in training evaluation. Grove. Baldwin.).J. (1996). "Evaluation. Newby. "Program personnel: The missing ingredient in describing the program environment. (1994).C. (1994).. Wexley & J. "Industry report: Who's learning what?" (1996).. & Newstrom. Brooks.G." Personnel Psychology. "Confirmative evaluation of instructional materials and learners.M. A. (1996).). Upper Saddle River. MA: Addison-Wesley. 299-314. (1992). "Levels of evaluation. lecturers and trainers.K. Boston. Cangelosi. 11(1). (1999). "The relationship between trainee responses on participant reaction forms and post-test scores. 33(10). Handbook of research for educational communications and technology. 5-20. Miringoff. (1996). Accountability in human resource management: Techniques for evaluating the human resource function and measuring its bottom-line contribution..J. & Ford. (1993).). (1981). J. (1996). Phillips. I. 41(1). (1980).S. New York: Dean." New Directions for Program Evaluation. Faerman. Malabar. & Walberg. D.F. "Comprehensive evaluation model: A tool for the evaluation of nontraditional educational programs.L. San technique. New York: Macmillan. Designing instruction for adult learners." New Directions for Program Evaluation. (1991). J. Forsyth. M." Public Productivity &. Boston. Designing educational project and program evaluations: A practical overview based on research and experience. ---. Washington. G.D. 23(2). (1993). New York: Simon & Schuster Macmillan. & Ostroff. London: McGraw-Hill. & Smaldino." Journal of Industrial Teacher Education. Management in human service organizations. & Casper.. "Transfer of training: A review and directions for future research. 22-27. (1986). D. & Stevens.S. Antheil. Evaluating diversity training: 17 readyto-use tools. J. Beer.L.A. Bramley. "Sumrnative evaluation in training and development. (1989)." In K. 53-65. Evaluating training effectiveness: Benchmarking your training activity against best practice (2nd ed..K. W. L. & Ban. 1045-1104. & Bloomer. T.K.. Patrick. Peterson. Mattoon. MA: Kluwer Academic.L. 83-92. R. 44. (1988). Jonassen [Ed.J. classroom instruction. "Ergonomics and the learning environment. TX: Gulf. M. 40. & Bickman. Training.J.Management Review.E. (1986). 63-105." Human Resources Development Quarterly. Evaluating training programs: The four levels. 16(3). SEPTEMBER 1999 . Alexandria. H.H. DC: The Bureau of National Affairs.. J. v." New Directions for Program Evaluation. McVey.A. TX: Air Force Material Command. (1996).The 1996 American Society for Training and Development report on trends that affect corporate learning &. Haertel.. Basarab. The ASTD training and development handbook (4th ed. Evaluating courses: Practical strategies for teachers. Broad. S. G. London: Kogan Page. C. D." Educational Evaluation and Policy Analysis. (1996). 55-64. The training evaluation process: A practical approach to evaluating corporate training programs. P. New York: McGraw-Hill. Evaluating training and educational programs: A review of the literature. Instructional media and technologies for learning (6th ed.. design of Parker. (1990).). 40. A. "Program evaluation.). Transfer of training: Action-packed strategies to ensure high payoff from training investments. 1(2). Handbook of training evaluation and mea- Dixon.D. 5-185-5-220.H.).

e. Smith Research Center.surement methods: Proven models and methods for evaluating any HRD program (3rd ed. Exercises in instructional design. training evaluation. OH: Merrill. This article was accepted for publication before James A. training evaluation. B.J.).).I§. He holds a PhD in Instructional Systems Technology from Indiana University. 383-411." In H. (1993). Room 101. email: pershin@indianaedu. He may be reached at the Office of Education and Training Resources. Designing training and development systems (3rd ed. "Objectives and evaluation. computer-based several topics in professional Educational Performance Communications Improvement. Sanderson.e. G.D. for and and theory of instructional and performance technology. and the business impact of training and development. Bloomington. B. Robinson. 4(2). (1994). Smith. M. Plant. Worthen. B. needs analysis. & Cook. He may be reached at Wendell W.R (1990). Z. Amherst.). Handbook of training and development for the public sector: A comprehensive resource. B. Seels. PhD." In S. Sanders.. Flex: A flexible tool for continuously improving your evaluation of training effectiveness.J. His research focuses on the fields of performance analysis. Program evaluation: Alternative approaches and practical guidelines (2nd ed." Performance Improvement Quarterly. & Fitzpatrick. Instructor excellence: Mastering delivery of training. San Francisco: Jossey-Bass. the Worthen. 27-30. Walberg & G.D. (812) 855-8545. Powers.. (1992).R (1992). He has presented such as the Association Society for Training & Technology. associations the American instruction/training. and the International Federation of Training and Development Organisations over the last three years. J. D. "Program evaluation.G. (1992).. J." In H.E. Tessmer. Wright Education Building #2230. Stolovitch & E. Keeps (Eds. PhD. (1993). & Harris. S. Columbus. Pershing. Handbook of human performance technology: A comprehensive guide for analyzing and solving performance problems in organizations. 35-58. & Sanders. (1990). & Brandenburg. is an Associate Professor in the Department of Instructional Systems Technology and Director of Education and Training Resources at Indiana University He teaches courses and conducts research in the areas of performance technology. Handbook of training and development (znd ed. San Francisco: Jossey-Bass. (1991). J. MA: HRD. Educational evaluation: Alternative approaches and practical guidelines. Yelon. Got a Performance Qu. (1997). Training for impact: How to link training to business needs and measure the results. "Who is evaluating training?" Journal of European Industrial Training. W. S.R. TX: Gulf. the International Society for Development.lspl. Sung Heum Lee. J. 2805 EastTenth Street. IN 47405-1006. Cayer. New York: Longman. & Glasgow.. San Francisco: Jossey-Bass.). Worthen. New York: Longman. email suhlee@indiana.pn? • Check Out the ISPI Bulletin Boards at WWW. New York: AMACOM. Analyzing the instructional setting: Environmental analysis. is a Research Associate with Education and Training Resources at Indiana University. New York: Pergamon. Houston.edu. D. Bloomington. London: Kogan Page. or fax: (812) 339-8792.R (1987). Volume 38 • Number 8 39 • . "Classroom instruction. (1989). Haertel (Eds. N.R..V.L.L. G. RA. Wart..R.org Performance Improvement. RJ. or fax: James A. 201 North Rose Avenue.). J. (1995). 18(5). The international encyclopedia of educational evaluation. 42-47. IN 47408. D. Truelove (Ed. B. M. Schouborg. (1992). 113-14. & Robinson. "Summative evaluation. & Ryan. Pershing was selected to replace Martha Dean as the new editor of Performance Improvement..). M. San Francisco: Jossey-Bass. Oxford: Blackwell.). Tracey.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.