Forum

Integrating Essential Components of Quality Improvement into a New Paradigm for Continuing Education

THOMAS J. VAN HOOF, MD, EDD; THOMAS P. MEEHAN, MD, MPH

Continuing education (CE) that strives to improve patient care in a complex health care system requires a different paradigm than CE that seeks to improve clinician knowledge and competence in an educational setting. A new paradigm for CE is necessary in order to change clinician behavior and to improve patient outcomes in an increasingly patient-centered, quality-oriented care context. The authors assert that a new paradigm should focus attention on an expanded and prioritized list of educational outcomes, starting with those that directly affect patients. Other important components of the paradigm should provide educational leaders with guidance about what interventions work, reasons why interventions work, and what contextual factors may inuence the impact of interventions. Once fully developed, a new paradigm will be helpful to educators in designing and implementing more effective CE, an essential component of quality improvement efforts, and in supporting policy trends and in promoting CE scholarship. The purpose of this article is to rekindle interest in CE theory and to suggest key components of a new paradigm. Key Words: continuing education (CE), continuing medical education (CME), effectiveness, health professions education, leadership, quality improvement (QI), systems

Introduction Educational leaders with oversight of continuing education have an obligation to improve patient care in addition to promote clinician learning.1 The continuing education of clinicians is a major component of health care quality improvement (QI),2 which is a systematic form of ongoing effort to make performance better.3(p5) National trends in reaccreditation, maintenance of specialty certication, and re-licensure increasingly are to require continuing education to be used primarily as a vehicle to change clinician behavior and to improve patient outcomes.2 Other important trends, such as the emphasis on patient-centered care,4 are also focusing attention on important patient outcomes that require the attention of continuing education.5 Given this important
Disclosure: The authors report none. Dr. Van Hoof: Associate Professor, University of Connecticut School of Nursing, Storrs, and Associate Professor, Department of Community Medicine and Health Care, University of Connecticut School of Medicine; Dr. Meehan: Chief Medical Ofcer, Qualidigm, Rocky Hill, and Associate Clinical Professor, Department of Medicine, Yale University School of Medicine, and Assistant Clinical Professor, Department of Medicine, University of Connecticut School of Medicine. Correspondence: Thomas J. Van Hoof, University of Connecticut, 231 Glenbrook Road, Unit 2026, Storrs, CT 06269-2026; e-mail: tom.vanhoof@ uconn.edu.
C 2011 The Alliance for Continuing Medical Education, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education. r Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.20130

shift in focus in the eld of continuing education, educational leaders require a new way of thinking about continuing education. We believe that a new continuing education (CE) paradigm should focus on a prioritized and expanded list of educational outcomes5 and include three key QI concepts: what interventions work; reasons why interventions work; and what contextual factors may inuence the impact of interventions.6 The purpose of this article is to rekindle interest in developing a new CE paradigm and to recommend important components of the paradigm, reecting important ideas from education and QI. The goal of any continuing education activity should be an important change or outcome, and this goal needs to be kept explicitly in mind through the stages of planning and implementation if one expects the outcome to occur. In subsequent sections, we review literature relevant to designing effective continuing education programs that have improvements in patient care as their goal, and we offer a framework that reects the importance of educational outcomes and QI concepts. As we will show, the literature on CE effectiveness complements this new paradigm and demonstrates its relevance to educational practice. Though the continuing medical education (CME) literature provides much of the research in this area, unless referring specically to CME, we use the phrase continuing education (CE) to be inclusive of all health care disciplines. Paradigm Shift in Continuing Education An outdated but commonly held paradigm, most closely associated with CME, largely explains the inclination among

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 31(3):207214, 2011

Van Hoof and Meehan

Six Important Educational Outcomes in Decreasing Order of Priority5 Areas of strong emphasis of new CE paradigm* guided by Quality Improvement Intervention Theory 5,6 Clinician Knowledge/Competence Clinician Satisfaction Clinician Participation Community Health Status Patient Health Status Clinician Performance

Areas of strong emphasis of outdated CME paradigm710

*Under a new CE paradigm, an educational program would look for opportunities for improvement at the community health level. For any community health opportunity found, the program would look for other performance gaps at lower levels of outco mes. Ultimately, the program would design an educational strategy to address any/all performance gaps found.5

FIGURE 1. Outdated and New Paradigms of Continuing Education in Relation to Important Educational Outcomes

some educators toward passive activities (eg, dissemination of information through lectures or readings) when thinking about CE (FIGURE 1).710 The old paradigm assumes that physicians who have completed their medical school and residency training are prepared for a lifetime of clinical practice. This perspective assumes that physicians, because of their rigorous professional preparation, require only knowledge updates in order to change clinical behavior and to improve patient outcomes.7,8,10 While logical, this outdated paradigm is inconsistent with numerous empirical studies reected in systematic reviews and meta-analyses of CME and CE.1,8,1015 Generally, increased knowledge and other elements of competence are necessary but insufcient to change clinician behavior and improve patient outcomes.13 The paradigms aw is not the emphasis it places on inadequacies in clinician preparationimprovements are necessary2 but rather its inability to account for the complexity of the learning process or the health care system. A new CE paradigm is needed that focuses attention on (1) higher-level educational outcomes (where to focus); (2) the selection and sequencing of educational activities (what to do); (3) the reasons why educational interventions work (rationales behind critical educational design choices); and (4) the contextual factors that help or hinder a CE effort. These components make explicit the importance of planning CE with attention to educational outcomes of utmost importance to patient care5 and build on QI concepts, which offer structure to a developing paradigm.6 These recommended components provide a simple but comprehensive set of constructs to improve patient-related educational outcomes of CE6 and reect a decades-long tradition of efforts to link quality patient care with continuing education.16 Consistent with QI concepts, a new paradigm should describe what interventions are necessary to produce results and account for the complexity inherent in learning and in systems.6 From the outset, the paradigm should focus the attention of learners on practice change, which requires much more effort than just learning new information or skills. The paradigm should also make clear to participants that in addition to the interventions themselves, CE needs to consider
208

why interventions work and what contextual factors may inuence the impact of a well-designed plan. A new paradigm would help participants become more informed consumers of CE and more sophisticated participants in complex systems. Interventions and Educational Outcomes in Continuing Education The term intervention refers to a set of actions to improve the quality of services to patients or populations.6(p16) An example of an intervention is audit and feedback of patient care data, which is a summary of clinical performance over a specied period of time.9 The summary may include comparison and benchmark data and recommendations for improving care. The interventions associated with a CE activity can be effective in changing clinician behavior and in improving patient outcomes if they account for the predisposing, enabling, and reinforcing factors of learners working in complex systems (TABLE 1).5,10,11,17 An educational program must generate or capitalize on teachable moments among its target audience regarding performance gaps, reecting the following priorities among educational outcomes: community (population) health, patient (subpopulation and individual) health, clinician performance (ie, clinician behavior in practice), and clinician learning (ie, clinician knowledge and competence in an educational setting).5 An educational activity that does not have its basis in performance gaps at the community health or patient health level is not an appropriate utilization of scarce educational resources, and educational activities that support clinician learning and competence should always be in the direct service of higher levels of outcomes. Data predisposing clinicians to an educational program can come from many sources, such as audit and feedback of patient care data or new evidence-based clinical guidelines that challenge current practice.25 Once clinicians are aware of a performance gap, an educational program must help them develop competence and improve performance by enabling knowledge, skills, attitudes, and behaviors reecting opportunities for improvement.5 The unique set of competence and performance needs existing within the target audience

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

Essential Components of a New Paradigm for CE

of clinicians will drive the curriculum and its instruction. Finally, an educational program must be reinforcing of competencies to sustain improvements and to overcome system barriers to implementation.5 A variety of relatively effective interventions are available for accomplishing the predisposing, enabling, and reinforcing components of CE.

Since at least 1998, an expanded list of educational activities has been available under the category of CE.15 While some interventions tend to be more effective than others in changing clinician behavior and in improving patient outcomes, even a less effective intervention (eg, noninteractive conference or educational materials) can be effective

TABLE 1. Educational and Quality Improvement Interventions Organized by Predisposing, Enabling, and Reinforcing Factors

Factor and description5,8,11

Intervention and description

Predisposing Create a teachable moment by communicating information about some important opportunity for improvement

Audit and feedback of patient care data7,9,11,1315,1821 A summary of clinical performance over a specied period of time; peer and benchmark comparisons, as well as recommendations for clinical action, may enhance the feedback; may be more effective when baseline performance is low. Local opinion leaders7,9,11,12,14,15,18,21,22 An educationally inuential local colleague, recognized by peers, who is able to inuence others performance by exemplary practice, role modeling, consultation, or teaching. Local consensus process14,22 Process of seeking input and buy-in from participating clinicians about the relevance and importance of an opportunity for improvement; participants may also help design the intervention strategy with input from an expert in quality improvement. Social marketing14,22 Interviews, focus groups, or surveys of participating clinicians to identify barriers to change and/or input on the design of an intervention strategy. Conferences7,8,9,1115,18,22,23 Brief, noninteractive meetings (eg, conferences, lectures, seminars, etc) or didactic programs held outside of the practice setting that convey information; may include question and answer periods; best if used in conjunction with another predisposing intervention. Educational materials1,7,9,11,14,15,1822,24 Noninteractive printed materials, audiotapes, videotapes, computer-generated materials, including clinical practice guidelines; if materials are solicited and focused, this may improve their effectiveness; best if used in conjunction with another predisposing intervention. Educational outreach visits9,1115,1822 Use of a trained person to visit a clinician in her/his practice setting to discuss clinical information relevant to her/his patient care performance; may be enhanced if it includes audit and feedback of patient care data and/or use of a local opinion leader. Interactive workshops1,8,1012,14,18,21 Interactive meetings, typically involving smaller groups of participants, using learner-centered strategies such as role playing or practice rehearsal, case or peer discussion, and/or hands-on or skills practice. Patient mediated interventions7,9,11,12,14,15,22 Any intervention aimed at providing information (patient education materials or counseling) or reminders (patient reminders) directly to patients that may predispose or prompt the patient to seek care or follow-through on some other health care service

Enabling Develop competence among clinicians or facilitate the desired change in practice relating to the opportunity for improvement

(Continued on next page)

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

209

Van Hoof and Meehan

TABLE 1. Continued Intervention and description

Factor and description5,8,11

Reinforcing Assist in recall of competence or reinvigorate change in performance relating to the opportunity for improvement

Clinician reminders7,911,14,15,18,2022 A manual (chart sticker) or computer-generated message (electronic patient registry or electronic health record) that prompts a clinician to perform some clinical action or recommend some service to a patient; replaces memory and provides useful, timely, and relevant information at/about the point of care. Audit and feedback of patient care data7,9,11,1315,1821 See description above under predisposing factor. Interactive workshops1,8,1012,14,18,21 See description above under enabling factor. Local opinion leaders7,9,11,12,14,15,18,21,22 See description above under predisposing factor.

Some overlap exists between interventions. For example, educational outreach visits often include audit and feedback of patient care data. No intervention strategy necessarily excludes another, although some interventions are more complementary of one another. Not included here are other noneducational administrative and QI interventions, such as nancial incentives and standing orders, which may also be part of an intervention strategy.

if combined with one or more effective strategies.18 Since no single intervention is effective in all circumstances18 and since multifaceted strategies are typically more effective than single ones,7,911,14,15,18,2022 a new CE paradigm should promote selection of a set of CE interventions that satisfy the predisposing, enabling, and reinforcing factors.5 Other important considerations in choosing a set of interventions include the details of the local context (eg, data and resources available) and the specic needs of the target audience (eg, weaknesses identied through assessment).26 Interventions that address the predisposing factor include initial audit and feedback of patient care data, use of local opinion leaders and local consensus processes, social marketing, educational conferences, and educational materials. Interventions that address the enabling factor include educational outreach visits, interactive workshops, and patient-mediated interventions. Interventions that address the reinforcing factor include clinician reminders, follow-up audit and feedback of patient care data, interactive workshops, and continued use of local opinion leaders. Interventions are important in continuing education, as their selection, order, and associated resources constitute the curriculum of the educational program, with its goals relating to higher-level educational outcomes. Ideally, educational leaders should involve participants in developing and implementing the curriculum, as this will increase their buy-in for the activity and their engagement in it as well.9 As part of discussion in a CE activity, leaders should make explicit which interventions align with predisposing, enabling, and reinforcing purposes so that participants will understand the learning and practice change process and its connection to the interventions and desired educational outcomes. Sharing such information about an educational activity with partici210

pants is referred to as visible pedagogy,27 which can help participants become more engaged and more informed consumers of CE. Reasons Why Interventions Work in Continuing Education While interventions are instructive about what to do and outcomes are instructive about where to focus, leaders need to understand why the interventions of an educational activity are effective so that such understanding may inform educational design decisions (TABLE 2).6 Important reasons underlying why include: the relevance of the CE activity to the clinical practice of the target audience; the intensiveness of the activity as a change strategy; the logic of the activity as a process of change; the engagement of participants in the activity; and the commitment of clinicians to behavior change as a result of the educational activity. For example, one would expect that an educational activity that reects that practice needs of participating clinicians would be more effective than one that does not because a needs-based activity would be more relevant. Relevant is why the activity would be more effective. The same is true for CE activities that are intensive, logical, engaging, and inspiring of commitment. Strategies that take advantage of reasons why interventions are more effective include: conducting a needs assessment to determine relevance; using a multifaceted strategy to ensure intensiveness; sequencing an activity to reect the logical steps to accomplish change; planning an activity that provides interaction or deep thinking within each participant; and nally, engendering commitment to change from participating clinicians. Such strategies nd theoretic and evidence-based support in a variety of literatures beyond

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

Essential Components of a New Paradigm for CE

TABLE 2. Reasons Why Interventions Work and Strategies to Leverage Them in Continuing Education

Reason

Strategy denition/description

Relevance

Needs assessment8,9,11,12,14,15,23,25,26,2833 Needs assessment is the process of collecting and analyzing data about patients, the care provided to them, and the clinicians and context relevant to such care. Data about clinical needs, patient outcomes, performance gaps, practice needs, readiness for change, and barriers to change are good examples of needs assessment data. Having data from more than one source is important, as is conducting a root cause analysis to understand opportunities for improvement. Multifaceted strategy7,912,14,15,1823,3436 A multifaceted strategy is the act of combining two more educational or quality improvement interventions for the purposes of addressing the same opportunity for improvement. The source of a multifaceted intervention could be one or more parties. Coordination is important to a multifaceted strategy. Multifaceted strategies may address multiple barriers to change, and multifaceted strategies compete better than single interventions with clinicians previous experience and training in order to bring about change. Multifaceted interventions are inherently more intensive. Sequencing6,8,9,15,23,26,32,37,38 Sequencing is the logic underlying an educational activity or series of activities for the purposes of helping participants learn and change in support of providing better patient care. Activities should build logically and incrementally on one another and increase the breadth and depth of learning over time. Administrative or other reasons (ie, convenience of scheduling) for ordering activities do not constitute sequencing as enhancing learning is not their primary goal. Interaction813,18,21,23,26,2830,32,3942 Interaction refers to the process within each learner of deliberating carefully (especially involving reection and analysis) about new concepts, ideas, and skills (particularly relating to an opportunity for improvement) and how they relate to her/his own assumptions, biases, and previous experience. Interaction in the social sense can be helpful to learning but learning can happen without social interaction. Leamnsons denition is helpful on this point: [Learning is] stabilizing, through repeated use, certain appropriate and desirable synapses in the brain.30 Commitment to change23,4347 Commitment to change is explicitly asking participants to change specic behaviors as a result of the educational activity. The idea for the behavior change may come from the participant, peer, presenter, or any other party; however, the request for the commitment must come from the program, and it must be specic and measurable. Commitment to change engages both cognition and emotion in the learner; this combination may explain its effectiveness.

Intensiveness

Logic

Engagement

Commitment to change

health care, including learning theory,40 adult learning,29,31,32 biology of learning,30 curriculum and instruction,38 diffusion of innovations,42 and systems theory.36,48,49 A previous set of articles describes how to use these strategies to plan an educational activity.25,37,39,43 Other strategies certainly exist as well, but these are the ones most commonly described in the CME and CE literatures. Reasons why interventions are effective and strategies for leveraging them are important to continuing education. As part of a visible pedagogy,27 every discussion with CE participants should include the rationale of why, by way of evidence and theory, one has planned a particular intervention or combination of interventions the way planners have. With time, CE participants will begin to appreciate why planning is so important, and they may be more receptive to participating in planning efforts. For example, if CE participants know that by

participating in a needs assessment they would receive a more relevant educational program, they may be more forthcoming with information about their practice needs and barriers to change. Contextual Factors That May Inuence the Impact of Interventions in Continuing Education In contrast to interventions (ie, what to do), outcomes (ie, where to focus), and reasons why interventions work (ie, the rationale for design decisions), educational leaders must be aware of contextual factors that may help or hinder the effectiveness of an educational activity (TABLE 3).6 The extent to which planners of an educational program can identify and address these factors through needs assessment and formative evaluation is important.6 Contextual factors of CE
211

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

Van Hoof and Meehan

TABLE 3. Categories and Examples of Contextual Factors in Continuing Education

Clinician/peer group characteristics Learners motivated and ready for change6,15,19,24 Baseline performance low14,24 Local standards of peer group consistent with evidence or change desired22 Patient population characteristics Seriousness/importance of the clinical topic or outcome11,13,14 Patient expectations consistent with evidence or change desired22 Organization/system characteristics Correct skill mix available in workforce22 Resources (eg, facilities and equipment) available22 Financial and other economic incentives align with improvement efforts6,19,24 Educational activity characteristics Small audience10,14 Single discipline10 Live media1,24,28 Multimedia1,24,28 Intensive activity (multiple sessions, contact hours, or exposures)1,10,14,24,26,28 Simple clinical behavior to be addressed13,14 Multiple instructional techniques used14,24,28 Consistent attendance14

as patient adherence to recommendations, may be different from their peers or from experts in their eld. Discussing such factors may increase receptivity to the importance of interventions that require system changes (eg, clinician and patient reminders) that are important to the sustainability of many improvements. Implications for Practice, Policy, and Scholarship While the elds of continuing education and quality improvement have matured signicantly in the past four decades, it is humbling to note that the concept of a new CE paradigm as partially described here has its conceptual roots in the 1960s, if not earlier.16 What is relatively new about the components and recommendations described in this article is the importance of combining ideas from QI and education to plan activities that reect the complexity of both the learning process and the practice environment. What is also new is the value of making explicit to CE participants, through a visible pedagogy,27 this complexity and in sharing with them the educational decisions reecting it. Application of QI concepts, paired with a focus on an expanded and prioritized list of educational outcomes (eg, community health status and patient health status), offers some building blocks toward a new CE paradigm that is more likely to be successful in changing clinician behavior and in improving patient outcomes because it is designed with these purposes in mind. Regardless of a new paradigm, this article recommends that educational leaders follow an explicit and transparent process in planning and implementing CE activities. Focusing on opportunities for improvement in patient outcomes, leaders should identify gaps in performance at each level of educational outcomes.5 Based on this gap analysis, leaders should utilize theory and evidence to design an educational activity that predisposes participants to change, enables them to make changes in practice, and reinforces improvements in practice. Leaders should consider important reasons why interventions work and the contextual factors of practice as part of planning, implementing, and evaluating educational activities. Disciplined adherence to this process will predict more effective CE activities and participants who may be more amenable to additional theoretical and evidence-based changes. Policy and scholarship can also support these recommendations. As accrediting bodies for education demand that graduate medical education and CME activities be aligned with changes in clinician performance and improvements in patient outcomes, and as specialty boards expect diplomates to engage in performance improvement activities as part of maintenance of certication, these recommendations will increasingly be seen as solutions to accreditation and specialty board expectations. The Accreditation Council for Continuing Medical Educations advocacy for performance improvement CME (PI CME),50 and the American Board of Internal Medicines performance improvement modules (PIMs),51 are two good examples of such policy. The

Presented in the form of facilitation.

include characteristics of the clinician/peer group, the patient population, the organization/system, and the educational activity. Most factors can be either facilitators or barriers to an educational activity depending on their direction or degree of inuence. For example, an inappropriately brief educational activity may be a barrier to change, particularly if it generates resistance to change among clinicians by declaring an opportunity for improvement without providing credible data in support of the claim. Meanwhile, a longer, longitudinal activity may be a facilitator of change, as it can bring people along a continuum of change starting with relevance and ending with commitment and practice support. While not all contextual factors are modiable or even identiable, educators should consider common, important ones in planning an educational activity. As with interventions, educational outcomes, and reasons why interventions work, educational leaders should discuss contextual factors with CE participants as part of educational activities. Factors include both system and individual characteristics, and as such, they set the stage for a rich discussion about the power of context and the value of different perspectives. CE participants may be surprised to learn that some of their personal assumptions about practice, such
212

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

Essential Components of a New Paradigm for CE

References

Lessons for Practice

Quality patient care, rather than clinician knowledge/competence, should be the primary guiding force for continuing education. Educational leaders should view continuing education as intervening in complex systems with the goal of improving community or patient health. Data to understand how a system is performing and changing as a result of efforts in continuing education to improve it are critically important. Clinicians participating in educational activities should receive an explanation of the rationale for choices made in developing an educational activity. Planning and evaluation efforts should anticipate and identify important contextual factors of practice and continuing education.

scholarship of CE should address underlying theory in order to advance these components and, with time, a new paradigm. While evidence of effectiveness, the primary concern of such manuscripts, is important, without a comprehensive theory that reects the complexity of learning and practice, it is more difcult to learn from CE studies. Historically, some educators have offered thoughtful theories about CME,52 but such theories have neither evolved nor received sufcient attention. Editors should expect, and CE scholars should devote, more attention to theory as a means of advancing the eld and of promoting learning from CE activities.

Conclusion The continuing education paradigm in the US health care system should undergo change. Quality improvement, in conjunction with education, offers compelling ideas, which place the needs of patients in the forefront and which respect the complexity of learning and systems. Educational leaders in health care can support this shift by developing programs that are population-focused, patient-centered, data-driven, and multifaceted, and that are mindful of important reasons why interventions work and common contextual factors of practice. A leader does not have to be an expert in QI or education to leverage the benets of these perspectives in planning and implementing effective continuing education activities, which are really attempts to change established professionals working in complex systems.

1. Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of Continuing Medical Education. Evidence Report/Technology Assessment No. 149. AHRQ Publication No. 07-E006. Rockville, MD: Agency for Healthcare Research and Quality; 2007. 2. Committee on the Health Professions Education Summit, Board on Health Care Services, Institute of Medicine. Health Professions Education: A Bridge to Quality. Washington, DC: National Academies Press; 2003. 3. Agency for Healthcare Research and Quality. http://grants.nih.gov/ grants/guide/rfa-les/RFA-HS-11-002.html. Accessed February 25, 2011. 4. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 5. Moore DE, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Cont Educ Health Prof. 2009;29(1):115. 6. Ovretveit J. A framework for quality improvement translation: Understanding the conditionality of interventions. Joint Comm J Qual Patient Saf. 2004;Global Supplement:1524. 7. Amin Z. Theory and practice in continuing medical education. Ann Acad Med Singapore. 2000;29(4):498502. 8. Davis D, Thomson OBrien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867874. 9. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288(9):1057 1060. 10. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Cont Educ Health Prof. 2007;27(1):615. 11. Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA. 1992; 268(9):11111117. 12. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274(9):700705. 13. Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;CD003030.pub2. 14. Oxman AD, Thomson MA, Davis DA, Haynes B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995; 153(10):423431. 15. Davis DA. Does CME work? An analysis of the effect of educational activities on physician performance or health care outcomes. Int J Psychiatry Med. 1998;28(1):2139. 16. Brown CR, Uhl HSM. Mandatory continuing education: sense or nonsense? JAMA. 1970;213(10):16601668. 17. Green LW, Kreuter MW. Health promotion planning: An educational and environmental approach. Mountain View, CA: Mayeld; 1991. 18. Bloom BS. Effects of continuing medical education on improving clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380385. 19. Smith WR. Evidence for the effectiveness of techniques to change physician behavior. Chest. 2000;118(2):8S17S. 20. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001; 39(8, Supplement 2):245. 21. Satterlee WG, Eggers RG, Grimes DA. Effective medical education: insights from the Cochrane Library. Obstet Gynecol Surv. 2008; 63(5):329333. 22. Grimshaw JM, Eccles MP, Walker AE. Changing physicians behavior: what works and thoughts on getting more things to work. J Cont Educ Health Prof. 2002:22:237243.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp

213

Van Hoof and Meehan

23. Van Hoof TJ, Monson RJ, Majdalany GT, Giannotti TE, Meehan TP. A case study of medical grand rounds: are effective methods being used? Acad Med. 2009;84(8):11441151. 24. Davis D, Galbraith R. Continuing medical education effect on practice performance. Chest. 2009;135(3):42S48S. 25. Van Hoof TJ. Conducting a needs assessment. Conn Med. 2008;72(8):477480. 26. Robertson MK, Umble KE, Cervero RM. Impact studies in continuing education for health professions: Update. J Cont Educ Health Prof. 2003;23:146156. 27. Iedema R, Degeling P, Braithwaite J, Kam Yin Chan, D. Medical education and curriculum reform: putting reform proposals in context. Medical Education Online. 2004;9(17):115. Retrieved August 20, 2007, from www.med-ed-online.org. 28. Mazmanian PE, Davis DA, Galbraith R. Continuing medical education effect on clinical outcomes. Chest. 2009;135(3):49S55S. 29. Knowles MS, Elwood FH, Swanson RA. The Adult Learner: The Denitive Classic in Adult Education and Human Resource Development. 5th ed. Houston, TX: Gulf Publishing; 1998. 30. Leamnson R. Thinking About Teaching and Learning: Developing Habits of Learning With First Year College and University Students. Sterling, VA: Stylus; 1999. 31. Long HB. Adult Learning: Research and Practice. New York, NY: Cambridge University Press; 1983. 32. Brookeld SD. Understanding and Facilitating Adult Learning: A Comprehensive Analysis of Principles and Effective Practices. San Francisco, CA: Jossey-Bass; 1986. 33. Wu AW, Lipshutz AKM, Pronovost PJ. Effectiveness and efciency of root cause analysis in medicine. JAMA. 2008;299(6):685687. 34. Van Hoof TJ. Designing a multifaceted intervention strategy. Conn Med. 2008;72(9):535537. 35. Hulscher MEJL, Wensing M, van der Weijden T, Grol R. Interventions to implement prevention in primary care. Cochrane Database Syst Rev. 2006;1:CD000362. 36. Senge PM. The Fifth Discipline: The Art and Practice of the Learning Organization. New York, NY: Doubleday; 1990. 37. Van Hoof TJ. Sequencing an educational activity. Conn Med. 2008;72(10):589591. 38. Tyler RW. Basic Principles of Curriculum and Instruction. Chicago, IL: University of Chicago Press; 1949. 39. Van Hoof TJ. Promoting interaction within educational activities. Conn Med. 2009;73(1):2931. 40. Dewey J. Experience and Education. New York, NY: Simon & Schuster; 1938. 41. Cervero RM. Place matters in physician learning and practice. J Cont Educ Health Prof. 2003;23:S10S18. 42. Rogers EM. Diffusion of Innovations. Rev. Ed. New York, NY: Free Press; 2003. 43. Van Hoof TJ. Using commitment to change within an educational activity. Conn Med. 2009;73(3):161163. 44. Mazmanian PE, Daffron SR, Johnson RE, Davis DA, Kantrowitz MP. Information about barriers to planned change: a randomized controlled trial involving continuing medical education lectures and commitment to change. Acad Med. 1998;73:882886. 45. Mazmanian PE, Johnson RE, Zhang A, Boothby J, Yeatts EJ. Effects of a signature on rates of change: a randomized controlled trial involving continuing education and the commitment-to-change model. Acad Med. 2001;76:642646. 46. Wakeeld J, Herbert CP, Maclure M, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23:8193. 47. Dolcourt JL. Commitment to change: a strategy for promoting educational effectiveness. J Cont Educ Health Prof. 2000;20:156 163. 48. Deming WE. The New Economics: Industry, Government, Education. 2nd ed. Cambridge, MA: MIT Press; 2000. 49. Nolan TW. Understanding medical systems. Ann Intern Med. 1998;128(4):293298. 50. Accreditation Council for Continuing Medical Education. http:// education.accme.org/tags/performance-improvement-cme. Accessed February 11, 2011. 51. American Board of Internal Medicine. www.abim.org/moc/earningpoints.aspx. Accessed February 11, 2011. 52. Fox RD, Mazmanian PE, Putnam RW. A theory of learning and change. In: Fox RD, Mazmanian PE, Putnam RW, eds. Changing and Learning in the Lives of Physicians. New York, NY: Praeger; 1989.

214

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS31(3), 2011 DOI: 10.1002/chp