Evaluation of the Franchise Conference Professional Development Workshops

Submitted by: Latanya Blakley

A short-term evaluation of the professional development workshops integrated into the companys Annual Franchise Conference for 2010.

EDTECH 505-4173 DECEMBER 9, 2010

Table of Contents
Learning Reflection2 Executive Summary....3 Purpose of the Evaluation.4 Background Information5 Evaluation Design Description.9 Results....12 Discussion of the Results....16 Conclusions and Recommendations.18 Appendix.20

Learning Reflection
Evaluation is a part of our daily lives. We informally evaluate products and programs prior to making decisions about them. We also evaluate those same things after weve engaged them to determine if it was worth our time, money, effort and so on. Therefore, when we shift our focus of evaluation to the implementation of products, programs and instruction that typically affect more than one individual and utilizes significantly higher investments of time and money, it is understandable that deliberate and systematic processes should be employed. This class helped to reinforce that evaluation is an important part of educational technology and many other disciplines. While several aspects of an evaluation were somewhat familiar to me, there were a few areas that I really learned a good deal of new concepts. First of all, the evaluators program description as explained in the text was an eye-opener in terms of the way to initiate the evaluation process. In terms of incorporating this into my existing schema, I likened it to a needs analysis in the instructional design process. I learned to rely on the EPD as essential to being knowledgeable about the program, understanding the objectives, the audience and the questions that the client wants answered. It is a major launching point for the evaluator and now that I understand its function, I cant imagine completing a comprehensive program evaluation without starting with this step. I also understand that data collection and analysis plays a key role in supporting and substantiating the conclusions and recommendations. Coupled with a strong base of evaluation questions answered by the analysis and recommendations, those aspects lend a great deal of credibility to the evaluator. In my experience, many in-house evaluations, especially those of instructional programs tend to stop at evaluations that only gauge the attendees reaction. Knowledge of evaluations can be useful when writing proposals to implement technology or instructional solutions. Educational technologists must research and determine the appropriate technology needs to best support instructional methods. In order to do that with some degree of certainty, a short-term evaluation such as the one in this report may need to be conducted before making a decision. One would also be expected to request funding for technology resources and many times this also means a justification for allocating resources. Therefore, educational technologists have a responsibility to measure the effectiveness, efficiency and if possible, the impact of technology resources, instruction and programs to know whether the resources utilized have or will yield the desired results or met the specified standards. Overall, I learned that evaluation is a both a science and an art. Though data collection and analysis is methodical, it can be challenging. After this process, I understand data analysis beyond what I learned in my Undergraduate studies, but I will benefit from additional specialized training on statistics and analysis. I have a strong desire to strengthen my skills in this areas, especially learning to measure correlations, standard deviation and performt tests.
2

Executive Summary
The National Franchise Conference provides an annual professional development opportunity for the franchisees of our organization. The purpose of this event is to positively provoke and educate the attending franchisees. The event is funded by a pool of royalty payments from franchisees, so they come with high expectations for learning about how to improve their business position and execute better instruction. Recently, a new conference committee acquired the responsibility for planning the conference and selecting the content for the professional development workshops. This committee was charged with first evaluating the effectiveness of the 2010 conference workshops before moving forward with any planning activities and thus any funding. The evaluation program description uncovered a few deficiencies in the workshop planning process including the fact that standards had not been developed prior to the workshop development. As a result, the evaluation was executed as goal-free with the intent of obtaining a picture of how the program currently operates and how it could be improved as planning ensues to meet the needs of the audience. The 2010 conference attendees numbered over 1300 and immediately following the conference an online survey was deployed to the over 700 franchisees who attended the professional development workshops. Later, a brief small scale group interview was also done to supplement the results of the survey. There were both qualitative and quantitative aspects to this survey which had a high respondent rate of approximately 33%. However, the questions on the survey lacked the necessary depth and range and served to provide one-dimensional ratings of the workshops and a host of uncategorized comments. While the results of the quantitative analysis did not produce very significant or distinct results, the qualitative analysis revealed the details needed by the committee to improve the process of selecting, developing, executing and evaluating the professional development workshops for the 2011 and 2012 conferences.

Purpose of the Evaluation

This evaluation will assess the usefulness of professional development workshops offered at our 2010 annual franchise conference. As the franchisee royalties are the primary source of funding for this event, it is vital to ensure that their needs are being met. However, the evaluation itself is being funded by the corporation with oversight being provided by the Conference Planning Committee (here forward known as the committee). The primary purpose of the evaluation is to examine what happened when attendees participated in the 2010 conference workshops as well as report the attendees reactions. The secondary purpose of this evaluation is to inform future planning decisions of the 2011 and 2012 conferences being made by the committee. The following questions were asked of the committee during the evaluators program description meeting: 1. Overall, what would the committee like to accomplish with this evaluation? 2. Are the any historical data that can be used in the evaluation? 3. Who is ultimately accountable for the outcome of this program? The committee shared the program information that is found in the background section of this evaluation report. In addition to understanding the outcome of the 2010 evaluation, the committee will use this evaluation to make decisions about the planning of the 2011 and 2012 conference workshops.

Central Questions
The central questions this evaluation will attempt to answer are: According to the franchisees, how useful were the workshops? What were the reactions of the franchisees to the workshops? What improvements should the committee consider for the future conference workshops?

There are also several ancillary questions, subdivided under the top three questions that will also be answered in the scope of this evaluation. These are found under the evaluation design description section.

Evaluation Results Audience

The committee and the corporate university department leaders are ultimately responsible for the planning and accountable for the outcome of this program. The two key audiences that will be most impacted by the results of this evaluation are the committee and, most notably, the attendees of the 2011 and 2012 conferences. Challenges and limitations of this evaluation include: There were no formal instructional objectives determined prior to the planning of the 2010 conference. Of the six members from the 2010 conference committee, five of them are in other roles or no longer with the company because of a recent reorganization. Since the inception of the conference,
4

a formal evaluation of this caliber has yet to be completed, though there were surveys done in at least the last five years. Due to time constraints, the evaluator could not obtain access to the previous years data for a comparative analysis.

Background Information
Program Origin
Each year over a thousand people from the Canada, Mexico and the U.S. gather in select cities for the companys largest franchisee conference. Hundreds of thousands of dollars are invested into the development of this event for the purpose of educating and motivating franchisees. The North American Franchise Conference has been an annual professional development event for franchisees and their families since its inauguration in year 2000. The first conference was held in a small hotel in downtown Chicago and was a fairly intimate event with a few main stage guest speakers and breakout sessions. At that time the conference lasted approximately two days and had only about 300+ attendees. Over the next 9 years, the conference grew to reflect the companys growth. The 2010 conference spanned 3 full days, hosted 1301 guests, including over 700 franchisees, and was held at a large conference hotel in Manhattan.

Conference Structure/Standards
The 2010 conference structure included the following key elements: a welcome reception, 1-2 keynote speakers, 1-2 franchisee main stage presenters, 2 student speakers, 8-10 workshops, an extended conference experience including city tours and visits to local franchises, addresses from the both the North American and Worldwide Presidents, 2 awards ceremonies, software demonstrations and numerous other networking opportunities. Franchise attendees were allowed to register for four workshops, two academic and two center management workshops. The minimum number of seats in a workshop was 150 and the maximum was 300. Company associates were allowed to register after franchisee registration was complete, but attended the same workshops. Registration is very strictly controlled. Once an individual has registered for the four workshops, no changes could be made. This is important to control the amount of constant or last minute changes that would potentially be made by registrants. Strict guidelines were also necessary to support the logistics of sending confirmations and printing badges with all the guest event details for such a large group.

The Committee
The conference committee in 2010 was comprised of 6 company associates. Planning for the conference usually takes a little over a year from venue booking through the day before the event. The new committee, associates from the companys corporate university, believes that conference content and production should be developed with the following goals in mind:
5

Educate franchisees on ways to improve their business and increase profits Share inspirational stories of highly successful franchisees Present findings of student case studies and research-based VSGs (voluntary study groups) Increase participation and attendance at the conference each year

The company recently announced a new expansion initiative that was discussed with the committee to determine if there was a potential impact to the conference content. The committee agreed that the new performance indicators should be considered when developing the future conference workshop content.

The Workshops
The workshops being evaluated can be grouped into two main categories, academic and business management workshops. An academic workshop can best be characterized by content that is student-centered and focused on helping the franchisee improve instruction. A center management workshop can best be characterized by content that is business-centered and focused on helping the franchisee grow his or her business. Presentations in either category can also be the result of a Voluntary Study Group (VSG). These presentations tend to be research-based and include case-studies of franchisee practices and student instruction methodologies. Each of the workshops in the program is no longer than one hour in length. Workshops are delivered in person via a classroom style lecture format using digital media such as Microsoft PowerPoint. The presenters include successful franchisees, company associates and outside consultants with current corporate affiliations. In past years, the committee has selected the workshops for the conference by meeting with potential presenters who have been nominated by leaders within the company. The concept is presented and the committee typically reads them and chooses the workshops based on the diversity of the content and prestige of the franchisees involved. The presenters must submit a synopsis of their workshop for inclusion in the conference registration website. The following are descriptions of the 10 workshops conducted at the 2010 conference. Please note that some names and terms have been omitted to protect the privacy of the company and its franchisees. Academic Workshops Workshop A: Learning from Students with Special Needs Students with special needs often provide Instructors extraordinary learning opportunities. The Instructors in this St. Louis-based VSG have worked with many children with special needs those with specific diagnoses, such as a learning disability and ADHD, autism, and developmental and emotional delay and recognize that each student and situation is unique, but that what we learn from these students has broad application. The Instructors will share their journeys with three students with special needs. They will discuss lesson
6

planning, parental involvement, and specific accommodations, which have helped these students to reach their potential. Workshop B: Simple Steps for Retaining Students Learn three of the keys to student retention. The Phoenix Branch Retention VSG has spent its time focusing on the orientation and enrollment process, effective ongoing communication with students and parents, and developing Center assistants. Come and see how focusing on these three keys as well as other fundamentals has helped this group of Instructors to retain students longer and to bring them to higher levels of achievement, while at the same time improving their retention rates, increasing ASHR percentages, and raising their IAR rankings. Workshop C: Title Withheld Members of the renowned Gateway VSG will share their experience and techniques for successfully guiding students to reach the highest levels while in elementary school. The workshop will focus on the dynamics of working with students in three key areas: Instruction tips to develop students XXXX skills Communication that emphasizes the benefits of advanced study Motivation strategies based on goal sharing and self-evaluation

These talented Instructors will share their experiences and knowledge related to the building of skills and confidence in their students. Join this workshop to learn more about how to instruct excellent elementary-aged children with focused hints and how to help them to succeed with the right amount of motivation. Workshop D: I Can: Assistant and Student Development for the Reading Program Esteemed veteran Instructors began the I Can VSG with the goal of clarifying and refining what their assistants assess when working with Reading students and grading their worksheets. As the VSG continued, they turned toward their lesson planning, seeking new ways to streamline lesson planning based on their VSG findings. Join this workshop to learn firsthand how these excellent Instructors are continuing to perfect their lesson planning, instruction techniques, and assistant training through their ongoing collaboration. Workshop E: Title Withheld XXXX said, Each instructor may have his or her preconceived beliefs, and each may reach a point where they feel something is good enough. However, the truth is, there is nothing good enough, but always something better. This presentation will explore how four Los Angeles Branch Instructors who felt they had reached good enough discovered through collaboration that by pursuing the potential of their own students, their ability as Instructors blossomed, too. These
7

four Instructors will share with you how learning from their excellent students and from each other has impacted how they communicate with, observe, and lesson plan for all of their students.

Center Management Workshops Workshop F: Maximizing Your Employees Potential to Maximize Your Potential Are your employees giving you and your students what you need and expect from them? Have you tried to provide sufficient training, but things still dont seem to be working out? If you are at all like the accomplished veteran Chicago Instructors who took part in this Voluntary Study Group (VSG), you may be experiencing some of the same staff challenges they have begun to address effectively: lack of initiative or confidence, ineffective interaction with students, lack of customer service skills, frequent employee turnover all of which lead to problems within the classroom, as well as have an impact on your own ability to determine compensation and manage payroll. These Instructors are eager to share how they utilized employee surveys and worked together to overcome these challenges, while also increasing their self-awareness and development as leaders. By sharing the path they have taken and the resulting accomplishments, these Instructors hope to motivate you to begin your leadership self-development anew, strengthen your staff development process, and work towards building a strong, successful team. Workshop G: Growing Your Business from the Grassroots Any small business owner will tell you that the best way to build awareness is through grassroots marketing. But what does this mean and how do you do it? This presentation offers examples of well-executed, community-focused initiatives, and outlines the tangible benefits of non-traditional marketing that not only drives leads, but also delivers value back to the community. Join this workshop to learn how Instructors in the U.S. and Canada have enhanced the reputation of the brand and their Centers, and how you can, too! Workshop H: Measuring Instructor Achievement with New Center Performance Analysis Reports Join this workshop to discover the new tools to help Instructors and Associates evaluate Center performance. The Center Performance Analysis (CPA) is a newly developed web-based report examining several key enrollment, retention, and student achievement indices. The CPA graphically depicts statistics found on the existing CMFA Report, as well as many new informative measurements, such as Student Inflow/Outflow, Student Turnover, Student Attainment within 12Months, and more. Discover how these new online reports can aid in your analysis of Center operations, including the ability to compare ones achievement with company goals and other benchmarks. With this new way of examining
8

Center performance, Instructors can become more empowered to improve instructional quality and Center operation. Workshop I: Operating an Efficient and Profitable Franchise Business Take a good, hard look at the way youre conducting your Center. Consultants from Coleman, Chappel & Company, one of the Top 50 public accounting and business consulting firms in the U.S., will offer expert advice on how to operate a successful franchise business. This workshop will provide tips on how franchise owners can operate their businesses more efficiently and profitably, including how to overcome the unique challenges franchise owners face throughout the life cycle of their business operations and how to boost profits. Workshop J: Understanding and Effectively Communicating With Prospective Parents Just who is our prospective parent? Why do they matter to us? And what does it take to communicate more effectively the extraordinary benefits of XXXX to them, ensuring they not only enroll their child, but remain committed for a significant length of time? Join us for an informative session that is sure to change the way you think about parents and which will give you the tools you need to reach and retain families.

Audience Characteristics
The physical audiences for the conference workshops include associates, franchisees and their family members. However, the clients who are most affected by this program include the franchisees of this international supplemental education company. They are adult learners with at least a bachelors degree, although several have postbaccalaureate degrees. Over 60% of the learners speak English as a second language. Also about 70% of have owned their franchise for 3 or more years.

Description of Evaluation Design

The committee acknowledged not being aware of any objectives, explicitly written or otherwise stated, that were determined in advance of the 2010 conference workshop selection and development. Therefore, the evaluation design is based on an amalgamation of the Goal-free Model with some components of the Decision-making Model. Both of these models will yield a summative evaluation of the conference workshops. As stated in the purpose section, the answers to the following questions will provide the information necessary to determine the useful of the workshops in the manner that they took place. A portion of the questions are also included to gather what franchisees would prefer to see in conference workshops.
9

1. 2. 3. 4.

Overall, how did attendees rate the 10 workshops? Which three workshops received the top ratings? Why? Which three workshops received the lowest ratings? Why? How do the results for center management workshops compare to academic workshops? 5. What are the attitudes of franchisees about the content for the conference workshops? (ordinal data ranking) 6. What are the attitudes of franchisees about the format of the conference workshops? o What aspects of the workshops did franchisees like most? o What improvements did franchisees suggest?

Data Collection
The main set of data was collected using an online survey instrument (Appendix A) with both a quantitative and qualitative element. A series of questions with a Likert scale was employed to gather a general rating of the workshops. Questions 1-4 in the list above, which were derivatives of the evaluators program description, were the impetus for the use of the Likert scale in the evaluation design. The ratings were used to address the first two central questions from the purpose section about usefulness and franchisee reaction. The survey also utilized open-ended survey questions to elicit more specific feedback related to each workshop that the respondent attended. It was necessary to gather this qualitative data to answer the third central question about considerations of future content. Questions 5-7 provided the framework for incorporation of an open-ended question format for each of the workshops on the survey. Approximately one week following the conference, the survey was distributed to the 707 franchises who attended the conference. Due to the nature of the business, it is typical that each attendee be given an opportunity to respond to the survey and the management was not willing to forgo that format at the risk of upsetting franchisees. Therefore, the data collection effort initially included the entire attendee population with the expectation that about 25% would respond. These respondents would constitute a 25% purposive sample in the evaluation design. The survey was made available for about three weeks before it was closed for reconciliation and preliminary data analysis. After the primary data collection activities, the evaluator was able to briefly meet with two separate groups of franchisees to conduct group interviews. These were unplanned opportunities coordinated with the approval of the committee that allowed for discussion about the workshop selection process. The first group consisted of 7 franchisees and the second group had 8 franchisees all of whom attended the conference and went to the workshops. A series of questions were asked to the groups to capture supplemental detail about why they chose the workshops for which they were registered.
10

Did you register for all four workshop slots?

Did you attend the workshops for which you were registered? What was the main reason you registered for the workshops you chose?

Data Analysis
In the quantitative analysis, the evaluator used measures of central tendency and variability to categorize most of the data sets. In order to make clearer comparisons, some data was converted to percentages where necessary. Qualitative data which consisted of many different comments was reviewed and coded to produce comparable nominal data such as positive, negative, neutral as shown in Tables 5a and 5b. The comments were also categorized as a suggestion or nonsuggestion. The suggestions were then coded to determine what type of suggestion and if there were similarities they were lumped together and discussed in the results section. Part of the analysis also included an interpretation of the comments to determine the aspects of the workshops most enjoyed by the respondents. This was done by tallying the coded comments in the prevalent categories.

11

Results
Survey
The online survey instrument generated responses from 239 respondents out of 707 attendees in the total population, yielding a 33.8% response rate. This section of the report will show how the data provides the answers to the questions and establish to what extent the attendees thought the workshops were useful. 1. Overall, how did attendees rate the 10 workshops? The mean of Likert scale ratings for each workshop was calculated. Because each workshop had a different number of attendees and respondents, n varies in the data set. The range of the scores (3.31 2.51) is 0.8.
Workshop n= Mean A 80 2.93 B 143 2.97 C 98 2.74 D 137 3.31 E 53 2.81 F 145 3.28 G 79 3.09 H 68 2.71 I 126 2.51 J 92 2.88 Table 1. Mean of Likert Scale Ratings per Workshop

2. Which three workshops received the top ratings? Why? 3. Which three workshops received the lowest ratings? Why? The top three workshops are D,F,G and the bottom three are C,H, I.
Workshop n= Mean D 137 3.31 F 145 3.28 G 79 3.09 B 143 2.97 A 80 2.93 J 92 2.88 E 53 2.81 C 98 2.74 H 68 2.71 I 126 2.51 Table 2. Highest & Lowest Ratings 12

4. How do the results for center management workshops compare to academic workshops? The Academic workshops received a slightly higher average rating than the Center Management workshops. However, it is noted that two of the top three workshops were from the Center Management category.
Academic A B C D E Center Mgmt. n= 80 143 98 137 53 Mean 2.93 2.97 2.74 3.31 2.81 2.95

Mean F 3.28 G 3.09 H 2.71 I 2.51 J 2.88 2.89 Table 3. Workshop Category Comparison

n= 145 79 68 126 92

5. What are the attitudes of franchisees about the content for the conference workshops? 6. What are the attitudes of franchisees about the format of the conference workshops? o What aspects of the workshops did franchisees like most? (next section) o What improvements did franchisees suggest? (next section) Respondents were aware that the numeric rating for each score corresponded to a description. Over 70% of respondents felt the workshops were either good or excellent. Appendix A illustrates the results of each individual workshop. Below is the combined result of those responses. (Colors correspond to the chart colors in Appendix B.)
All Workshops Response Number Percent 1 Poor 65 6% 2 Fair 215 21% 3 Good 445 44% 4 Excellent 296 29% 1021 100% Table 4. Overall Attitudes of Respondents 13

Following each rating question, respondents were also asked to share open-ended comments about the overall content, format and delivery of the workshops. This table, which represents the total of all comments entered, shows the number of responses coded with a positive, negative or neutral comment. Each comment was also examined for suggestions made to improve the content of the workshops (Appendix C). A summarization of the 69 comments is provided in the next section.
Content Positive Negative Neutral Total Suggestion 2 8 5 15 7 14 4 5 23 7 3 15 4 22 14 25 1 4 30 3 0 3 1 4 1 19 5 4 28 5 7 2 4 13 1 2 8 5 15 6 9 26 6 41 17 10 16 1 27 8 91 88 39 218 69 42% 40% 18% Table 5a. Respondent Attitudes Reflected in Open-Ended Answers Workshop Positive Negative Neutral Total A 13% 53% 33% 100% B 61% 17% 22% 100% C 14% 68% 18% 100% D 83% 3% 13% 100% E 0% 75% 25% 100% F 68% 18% 14% 100% G 54% 15% 31% 100% H 13% 53% 33% 100% I 22% 63% 15% 100% J 37% 59% 4% 100% Table 5b. Percentage of Attitudes Reflected in Open-Ended Answers

A B C D E F G H I J

7. What topics/content do franchisees want to see for upcoming conferences? There were 153 comments sharing suggestions for upcoming topics. For the purposes of the evaluation, a sample equal to the response rate, or 33% (51), was randomly taken of the comments.

14

Interview
The online survey instrument generated responses from 239 respondents out of 707 attendees in the total population, yielding a 33.8% response rate. This section of the report will show how the data provides the answers to the questions and establish to what extent the attendees thought the workshops were useful. 1. Did you register for all four workshop slots?
14 93% 1 7% 15 100% Table 6. Interviewees with Complete Registrations Question 1 Yes No

2. Did you attend the workshops for which you were registered?
15 100% 0 0% 15 100% Table 7. Interviewees with Correct Attendance Question 2 Yes No

3. What was the main reason you registered for the workshops you chose?
Question 3 The synopsis The presenters Influence from others Nothing else available 7 47% 5 33% 2 13% 1 7% 15 100% Table 8. Reasons for Registration

In the next section, these evaluation results will be interpreted.

15

Discussion of the Results

This evaluation was employed to answer three central questions posed by the committee responsible for planning the future conferences and designing workshops: According to the franchisees, how useful were the workshops? What were the reactions of the franchisees to the workshops? What content should the committee consider for the future conference workshops?

Results of the data analysis show that on a scale of 1-4, with 4 being the highest rating, the mean rating that respondents gave to the workshops overall was 2.92. This number was calculated by taking the average of the averages in Table 1. The ratings had a variance of 0.8 which means that the ratings varied moderately. As the overall average is closer to a 4 than a 1, and is closest to a rating of 3, we can interpret that the respondents collectively felt the workshops were good as established by our rating description. Analysis of data in Table 2 reveals that the top 3 workshops are D, F and G. The topics of these workshops are reading instruction, employee training/leadership and grassroots marketing respectively. Two out of three of these workshops are on Center Management and two of them are based on VSG research. The same analysis reveals that the least liked workshops are C, H and I. The topics of these workshops are instruction tips for elementary school students, the new center performance analysis and operating efficiently and profitably. Two of these workshops are also Center Management and only one is based on VSG research. The respondents who attended the top three workshops constitute 35% of the sample population, compared with 29% of those who attended the bottom three workshops. Upon examination of the qualitative data in Tables 5a and 5b, or the comments, the top three workshops D, F and G share 56% of the comments coded as positive. There seems to be a strong correlation between the ratings for D and F to the number of positive comments received. The comments themselves (Appendix C) also show that respondents appreciated the practical nature of the workshops and tools demonstrated from the top three workshops overall. In terms of accomplishments, the workshops yielded 42% positive comments from respondents versus 40% negative. These numbers could each be higher, but with the same ratio, because some of the neutral comments were coded as such because they included both a positive and negative aspect, which was deemed neutral overall. Also over 70% of the respondents rated the workshops good or excellent as shown in Table 4. Appendix A will also show that in each individual rating table, the percentage of good and excellent outweighs the percentages of fair and poor ratings without exception.

16

There key areas that stand out as aspects of the workshops preferred most by the respondents (Appendix C) can be summarized as follows: Workshops that offer specific strategies or recommendations Helpful and practical tools that have been tested and are available for use Strategies and ideas that can be immediately implemented Information that can impact instruction or help students more directly Hearing from their peers rather than from an outside speaker Having more informative and accurate workshop descriptions that reflect the target audience

The last point is also supported by the results of the small group interview where 47% of franchisees said their main reason for choosing their workshops is because of the synopsis they read on the website.

17

Conclusions and Recommendations

Immediate Conclusions
1. There appears to be no clear distinction between the effectiveness of Center Management workshops and Academic workshops. Therefore, the committee should continue to offer a balance of these workshop categories. 2. Develop program objectives/standards Prior to beginning the planning and selection of professional development workshops, the committee should meet and determine specific and measurable objectives for this aspect of the conference. Having benchmarks will help guide the activities of the committee and set standards that the attendee surveys can be measured against. 3. Use a criteria checklist when choosing workshops to ensure there are objective standards used that meet the needs of the audience. The preferred content mentioned in the survey should be considered: Workshops that offer specific strategies or recommendations Helpful and practical tools that have been tested and are available for use Strategies and ideas that can be immediately implemented Information that can impact instruction or help students more directly Hearing from their peers rather than from an outside speaker Having more informative and accurate workshop descriptions that reflect the target audience According to the committee, the new company initiatives should be included in the future conference content. The criteria can include these initiatives to ensure that new content meets the needs of the company and franchisees. 4. The workshop synopses appear to have a large bearing on the expectations that attendees have for the workshops. Designate a strong writer and someone from the marketing team to make sure the synopses include information such as the target audience and an accurate description of the workshop content and format, so that attendees can choose the best workshops suited to their experience level and dont feel misled because of unmet expectations. 5. Consider involving an advisory committee comprised of franchisees in the planning of the conference professional development workshops. The committee can and should control the extent to which they have input, but franchisees might feel better that their interests are represented in the planning of event funded by their royalties.

18

6. Hire a professional survey writer to redesign the survey used by franchisees to evaluate the workshops and write a survey to request content suggestions for the upcoming conference workshops. The new survey should include demographic data and be much more specific in terms of categorizing comments. The demographic data can be used to cross tabulate with survey ratings. It is important to determine the experience of franchisees and whether or not any correlation exists with their level of satisfaction regarding workshop content. Include location, number of students, number of previously attended conferences. The standards and criteria established from points 1 and 2 should also be included in the new survey and used to confirm the standards are being met.

Long-Range Planning
This evaluation should be used as a benchmark for future evaluations. The committee should discuss ways to improve these statistics and work to track this data over time if these workshops will continue to be produced for the conference. Objectives-based point of view choose a couple of workshops with clear cut learning expectations write actual learning objectives for them. In the future, the committee can employ a focus group or small group interview to gain insight into how well the learning objectives were met which could aid in assessing effectiveness. May need to randomly sample from franchisees either on-site at the conference or a short-time after. For example, the committee could collect a list of participants and then send them a questionnaire immediately after the workshop followed by an interview schedule.

19

Appendix A Individual Ratings with Percentages

Workshop A Response Number Percent 1 Poor 7 9% 2 Fair 15 19% 3 Good 35 44% 4 Excellent 23 29% 80 100% Workshop D Response Number Percent 1 Poor 1 1% 2 Fair 19 14% 3 Good 54 39% 4 Excellent 63 46% 137 100% Workshop G Response Number Percent 1 Poor 3 4% 2 Fair 15 19% 3 Good 33 42% 4 Excellent 28 35% 79 100% Workshop J Response Number Percent 1 Poor 6 7% 2 Fair 25 27% 3 Good 35 38% 4 Excellent 26 28% 92 100% Workshop B Response Number Percent 1 Poor 6 4% 2 Fair 28 20% 3 Good 73 51% 4 Excellent 36 25% 143 100% Workshop E Response Number Percent 1 Poor 4 8% 2 Fair 14 26% 3 Good 23 43% 4 Excellent 12 23% 53 100% Workshop H Response Number Percent 1 Poor 5 7% 2 Fair 22 32% 3 Good 29 43% 4 Excellent 12 18% 68 100% Workshop C Response Number Percent 1 Poor 8 8% 2 Fair 24 24% 3 Good 51 52% 4 Excellent 15 15% 98 100% Workshop F Response Number Percent 1 Poor 1 1% 2 Fair 18 12% 3 Good 66 46% 4 Excellent 60 41% 145 100% Workshop I Response Number Percent 1 Poor 24 19% 2 Fair 35 28% 3 Good 46 37% 4 Excellent 21 17% 126 100%

20

Appendix B Survey Questions and Graphic Results

21

22

23

24

25

Appendix C Open-ended Comments

Workshop A: Learning from Students with Special Needs Each special need student is different and I am still wondering how to better help my special need student. I was hoping to get information on definition of different disorders and how to handle it. Did not attend I did not attend This really was three case studies; would have preferred to see a presentation on specific strategies for students with specific types of needs. The supplements showcased are not XXXX's -- why can't our corporation adapt to the needs of our communities by growing our supplemental tool arsenal?... Well done. Needed more specifics. I would better know how different between normal students and special need students to XXXX study plan and goal. not attend There were no specific recommendations from the VSG. I expected to get more information on symptoms and characteristics of specific learning differences and a way to deal with them. NA More in depth tips of identifying and working with these students would have been helpful. XXXX needs a forum where instructors can go to talk & get help for these students...there are so many and the need is so great. Could have been better presented. It was kind of obvious this isn't something they do regularly. There was not enough deep content in this workshop. I left with a sincere appreciation of what these Instructors are doing for their students but without anything substantial to take back to my Centre. I was expecting specifics of working with these students. e.g. what approach to take, what works, tips for various needs etc. Wish we can have more Q&A time. Love the discussions. Workshop B: Simple Steps to Retaining Students Good but a bit basic for my needs Not much additional info. Just seemed to be recycled info. I received a lot of good ideas. This was very well done. Nice format, the speakers were very good. Did not feel that we got any new information, XXX's presentation was very good, however, the ideas of conferences etc were the same old stuff. Good enough just reinforces what I do Excellent Liked the ideas of conferencing with parents of new students after 3, 6, 9, and 12 months. Also liked talking about assistants and the role they play in retention. Lots of good ideas to try out over time. Nothing new. I liked this - similar to other IMMs but some new ideas also. I liked XXXX - gave some specific suggestions. Would have liked even more. This VSG did a great job of providing actionable tips to increasing ALS. Excellent 26

Great new ideas to take back and use. After talking with other Instructors, we came to consensus that these were the steps we are currently taking to retain students and communicate with them. These were also the steps continuously discussed throughout the initial training before we became Instructors. It would be nice to see actual samples of the newsletters, how a parent conference is held, and some of the logistics of it. I think Instructors have many fancy ideas, but executing the plan and coming up with the materials is always the hardest. I wanted something more concrete. This workshop should have been entitled, "What everyone should already be doing". I was very disappointed with the superficial nature of this workshop. These were retention strategies - they were basics. I was however, pleased that these "simple steps" have lead to increased retention and subsequently, increased XXXX. Interesting and useful to hear about approaches used by successful instructors. It will be more meaningful if all the presentations and material are available to us. Gave me some ideas that I will incorporate in my center. It is helpful for the new instructors. To balance listening to the instructors reading from their pages, it would've been nice to have more visual aids or handouts for reference.

Workshop C: XXXX Very disappointing. "I liked this the least. Mrs. XXXX (sorry not to spell correctly) has a wealth of information but delivery is poor. Everyone states that observation is important. I prefer more how to than results presentation. I wish I had more content. More specific details. Loved it Very, very good. The blurb should have made it clear that the workshop was Gateway related. Because I thought it would be more curriculum based I was disappointed. Didn't learn much new but gave me new impetus to XXXX as quickly as possible. too cold Was not what I expected- expected to hear more about how to equip my students . Mrs. XXXX spoke entirely too fast- could not keep up nor follow. key concepts / XXXX were breezed through too quickly need more advance notice of session content not really geared to instructors with more than 1 year experience XXXX was the only presentation that I could understand Poor/ If I knew it was a gateway meeting, I would have selected something else. I need to learn more about how I can help students better by showing them examples from previous sets that explain new concepts. I wish there was a cumulative reference print-out. Very informative and motivating The presentation was difficult to understand as the slides were moved thru quickly. However, I will view it again on XXXX once it is available. There was a great deal of use info. Expected more solutions used to problems students encountered.... What they are talking about does not fit this topic. Although the content was very good, the material was presented too quickly for satisfaction. 27

The presentation was presented in a very boring manner and didn't really give me any concrete tips for helping the average student to program completion. Disappointed. Didn't seem to really give any concrete, hands-on ideas. Seemed mostly to overview the XXXX instead of how to adequately and/or uniquely plan for them.

Workshop D: I Can: Assistant & Student Development I hope XXXX can share the guides and material with us. Wish there had been a Q&A. I really thought this was a helpful workshop and I look forward to trying out their materials. Found the 'contract' approach something I wouldn't use, liked the booklets with the oral checks, too much paper for each reading student in my opinion Wish I had seen this. I will be anxious to watch DVD. New tools to help communication with staff. Great work! I liked the presentation, I feel XXXX has to put the booklets together and make it available for instructors as a non-chargeable item. Reinforces all that I do The tools they have developed were just the thing we have been looking for! Excellent. Eye-opening. The reading department should take the material from this VSG and apply it to all the XXXX of the reading program. XXXX should also provide the student and assistant reading guide for all the reading XXXX. WOW! Lots of info and am very thankful they are willing to share their findings with us. The best workshop I attended. Although the process seems a bit cumbersome/redundant, I love the XXXX-by-XXXX customization. Again, it would be nice to see XXXX adopt some of these tools presented by franchisees as the "standard" available to all franchisees. Practical and informative. Helpful to go away with tools to use in the Centre. Well done. Wow -- I love those booklets. Seems like this was a fantastic VSG. Can't wait to see that presentation on-line. Extremely interesting. I would like to hear more from this group. Very useful and practical. We need this ASAP. This is a missing link in the reading program. "We need more summary the other XXXX, too. They provide only XXXX. We need the chart in irregular past verbs." Great- would have liked to walk away with the Reading XXXX summary for XXXXs. Interesting replacement for the orange book and good staff training Don't need more paperwork. Would like to see video in real time with the normal "chaos" of feedback, speaking to the students, etc. It always seems that these video clips show the ideal situation; no other students or interruptions while the instructor is focusing on ONE student. I hope the little booklets will eventually become a non-chargeable order. This is a great tool and is extremely useful for bringing staff to a higher XXXX of understanding of the reading program concepts. Love this tool. Please help the VSG complete the project and get them online for us to use. I would love to see these available through XXXX in the future. These Instructors from the VSG presented concrete materials that they used in their centers which make this presentation more interesting and useful. Especially - these are Instructors that are experienced and have large centers. 28

These instructors have obviously presented before and they did a good job. I am hoping there will be a posting of the "I Can" printouts and I look forward to developing more. Thank you for sharing your hard work, XXXX and team. This was the very best. I am very grateful to the instructors who put so much work into their VSG, were so creative in generating the Student and Assistant Reading guide booklet, and most of all, for being so generous in sharing their insights. I am touched, impressed and inspired by this group of excellent instructors. The presentation was good, but will only be helpful if all the reading booklets are made available to us. I want to thank these instructors for all their hard work Very helpful and it was delivered by platinum instructors...

Workshop E: Learning from XXXXs Everyone states that observation is important. I prefer more how to than results presentation. "I doubt even though young students are excellent in academic, how many students handle the end of our XXXX? I chose one of students in my center, parent reject to work the completion XXXX." Didnt include reading completers More of the same from general assembly. Workshop F: Maximizing Your Employees Potential I learned many practical hints I can use right away. Addressed many of my concerns and was very motivational. Well thought out, well presented, good materials, hope soft copies of their ideas will be made available Looking forward to implementing a lot of these. Awesome new materials! There were nuggets of information that we could take away. I am waiting for the posting of the presentation. If the information is available to all on the forum it would be easy and customizable. Being a part of every VSG may not be viable for everyone. I am currently in a VSG in my branch. Adding one more will distract me from that one. Did not feel I got a whole lot from this presentation as well. Insightful. Every presenter had much to offer. A little too general. It would be better if they address some of the more specific concerns. But I do understand the time constraint. I hope that I can receive some record from the presentation. Well done- but needed to get to the point sooner. This was the most valuable part of the conference for me. I got MANY concrete ideas to improve my center at this workshop. XXX is such a wonderful asset. The instructor-presenters did a fantastic job. Best workshop this year. I was expecting something a little more specific. It would be good to know what a VSG has decided is a best ratio staff:student or how specialized XXXX staff arrangements compare with "Jack/Jill of all XXXXs"- in terms of efficiency. Wonderful presentation, I wish, we could get started with such VSG in all branches. I walked away with practical steps and have already implemented two in my center! I would have loved to have more time with Dr. B! This was great. 29

Lots of great work done by VSG. I am looking forward to contacting them to obtain templates, etc. produced to tackle this huge component of running a XXXX franchise. I think these should be posted on XXXX under some forum of 'best practices'. "A head hanging for me really since I should have already been doing most of what they talked about and either haven't or haven't done it consistently. I have already written the long overdue job descriptions although I still need to tweak them." great ideas Came away with a few good ideas. Very well done. Great ideas presented. Very clear and to the point w/ great ideas. Some good tools and strategies. I am hoping the criterion for employee evaluation papers will be posted on line. I would like copies of the employee, student checklists I will be giving the employee survey to my employees and hopefully will develop my own concrete policies and procedures. Still wished they could share the PD's they developed in their VSG. Forty-five minutes of the hour was spent talking about the leader's VSG. The only helpful portion was XXX's Q&A at the end. I was especially disappointed because I took the pre-conference survey and expected to have some of those topics covered. However, handouts really would have been great here.

Workshop G: Grassroots Marketing Realized that many ideas need to follow and keep trying every year. Very varied and lot of really good options. Good. I got some ideas that I may implement. Excellent. I am biased on this one because I was one of the presenters. Unfortunately I was not able attend another workshop on Day 2 because of the time conflict. I wish I'd gone to this one.... Great ideas. I particularly liked the idea of a Parent Education Night. That's on my list for later this summer. Nothing new - really a redo of a monthly IMM. Did not attend Good ideas... why did we not hear from all participants in the VSG? I really like that a variety of instructors were used. XXXX had a smaller number of students than many center instructors that present. Still she had a good turnout for her family night and she feels it really helped her. Sometimes we need to learn from the big, excellent centers and sometimes it is helpful to see what will work for a smaller center. Good ideas and presentations. Didn't really hear anything new. Workshop H: Center Performance Analysis Report Although many of us thought it was our IPA opportunity, it was still interesting, just not what I expected. XXXX should get the XXXX equivalent of an Oscar--should we call it a XXXX? Same for XXXX for that video in day one. Thought it was about IPA I would like to see this replace XXXX with full capability. This way staff time will be spent in discussing solutions as opposed to a lengthy, you did not do this got to change. The information 30

presentation need not take a negative tone that something is deficient hence improving with this proposal. Instead it can give time for the instructor to digest and approach the meeting in a 'How can we take care of this' mode. Presentation explained the new reports, but presenters should not have avoided issue of outgoing / incoming transfers. There is a student ID. Why can't the student be tracked across different centers? For the first time in many years, I fully understand a report before I have to analyze it! The lighthearted presentation kept my interest. The content of the workshop was very interesting but I felt the presentation was a bit too silly for me. It was useful to see how the data will be provided and how to interpret it. XXXX and XXXX provided a light-hearted presentation to a topic that could have been dull and stale. =) The presenters did a great job, but this just didn't answer the questions I needed answered. For example, when a student comes to me -- knowing up-front they will only be able to enroll for 3-4 months, but really want to get started doing something... Why am I penalized for working with them? I enroll them because it's the right thing to do, but my ALS is trashed by it. Far too basic Having my own report as a reference would have been ideal Another session for new instructors with silly role play No new material There was no report. It was an overview of CPA. Most of people who attended the workshop had an interest how to figure out CPA report points. This information was already presented by the branch to each of us individually. I felt the majority of the audience was misled to believe the presentation to be about IPA. The questions left me unsure on several areas. Can this be explained on a smaller venue such as IMM?

Workshop I: Operating an Efficient & Profitable Franchise Disappointing - way too basic! Did not find any new info for me. Gave me some good ideas to move ahead with, simple, direct and to the point This was a very good workshop. Main speaker very effective. Should have been called Running a Business 101... Good explanation of basic accounting and financial analysis. Would like to have presentation on business valuation (equity) next time. It was an accounting lesson -- where was the part about becoming more profitable -- not just tracking our costs and preparing a budget etc. Should not be part of workshops could be part of general assembly This presentation was not what it was advertised to be. Most disappointing element of the conference for me. The presenter gave tips and strategies for how a new business owner should handle budget management, not how an existing business owner (the VAST MAJORITY of the audience) could adapt the business to become more efficient/profitable than it is today. Make sure your workshop titles accurately reflect the presentations being given. I wish I'd gone to this one too... It was very basic. It would be nice to have different XXXXs for the workshops so that long term instructors can get more out of it. Nothing new 31

32

Very good info - this should be part of training and IMMs. We need this kind of additional support in order to run a business. XXXX treats us like educators instead of business owners. Very good. Having attended training in the past year, I don't feel I was the target for this presentation. Far too basic. This should have been flagged as most appropriate for new instructors. Far too basic "This was basic info that should be presented at training and not at the conference." This was essentially a review from training Did nothing to provide help with operating any kind of Centre Basic accounting 101was useless to even the newest instructor let alone those with 30 years business experience 10 of which running million dollar budgets. Printing balance sheets and P&L statements and sticking them in a book hardly rates up there with state of the art financial guidance. I sure hope the advice and counsel provide XXXX on their money matters is of significantly higher quality This was very good. Entertaining and informative This was too rudimentary for me...I was looking for guidance on analysis of the BS/PL or tips on what areas should be watched to manage expenses better, etc. I probably should have attended H instead. He was excellent, answered key questions, and covered what's involved. VERY CLEAR. Did not learn anything new Good basic business understanding. We need more of this type of session. It would be more helpful to drill it down even further rather than making it so broad. This is more so a presentation on how to keep a good bookkeeping system. What are some suggested percentages of our monthly/annual budget should be spent on what areas? What are some successful marketing techniques that Instructors have used to PR their center? How do you run a profitable franchise? These were questions I had prior to the presentation yet still remained unanswered. While I thought he was entertaining, I didn't think this would be Accounting 101. I have a tax advisor, use Quick Books, know how to produce P&L's & Balance Sheets, etc. I would have liked to hear about what XXXX thinks is an optimal staff to student (subject) ratio. Also, what is an optimal payroll percentage vs. net profit, etc. Also, what is the formula for determining what your center is worth, should you decide to sell it. Too basic. Not enough XXXX specific recommendations just general business tips. Coming from a business background before being an instructor - I truly believe XXXX is doing a disservice to new franchisees in not giving them enough information about running a business. The business plan helps but it is a drop in the ocean compared to reality. The speaker was good but this was an accounting workshop. This had nothing about efficiency and profitability. It was an hour recap of my grade 10 accounting course. I already have balance sheets and P&L statements - I was looking for ways of increasing the bottom line on these, not just a workshop on actually doing them. The speaker, however, was very good. Waste of time. I had hoped this would talk more about student:staff ratio, reasonable rent to pay in relation to tuition, 12 month commitment or not, tuition payment plans, time management on the part of the instructor, what I can or should not delegate... XXXX has a great printout available online and I am studying it but want to learn more. Seemed pretty basic.

Not bad but too basic for me. I wonder if you can give instructors the XXXX of lecture e.g. beginner/intermediate/advanced This was just a lesson in accounting jargon. The presentation should deal with cost cutting/control, mistake avoidance, ways to boost profit, etc. The title is deceiving...please use better descriptions next time and it will help if you include what type of instructor could benefit from it; i.e. brand new, veteran, etc. Helped me realize how important financial statements are. Topic was misleading. I thought it was going to show you ways to efficiently run a profitable business; instead it was more like an accounting class. It was OK. Good, basic info. No really interesting tips though.

Workshop J: Understanding and Effectively Communicating with Prospective XXXX Parents This session alone made the conference worth it, even if there had been nothing else. XXXX rocks. I'm now focusing more on the "Involved Mom." Very well done! Enjoyed this and obtained a lot of info from workshop. Exciting prospects with the new ad company!!! It was all about XXXX and social media. I expected something more comprehensive -- perhaps info from exit surveys that we could turn around and use with prospective parents? Did not get a lot from this... I thought it was for the communication with perspective parents but somehow it covered more marketing which we already attended in other workshop. I feel the name for the workshop was very misleading. Should not be a work shop should also be part of general info. I feel I missed out on opportunities to learn from other instructors I thought I was going to learn techniques to make me better at understanding and communicating with parents not listen to the new marketing direction. Interesting information. Well presented I am looking for great things from this new agency. I hope they really understand the need to cut down on the length of material we give to families. I really believe it rarely gets read. Super plans on revamping web site and targeting our advertising. Can't wait to see it roll out. Informative It was mostly about understanding, not communicating. Excited about going digital Very excited to see new information that is coming soon. I do not like the vendor to take one of our conference sessions for their advertisement. I would like to learn more from instructors. It's a good start, I was happy to see the staff wanting some feedback. Hitherto, XXXX in NA has barely scratched the surface of the non-immigrant market. This serious attempt by a commercial advertising company to target that market will begin a campaign to make some inroad in that direction. Let's hope there will be positive and significant results! Excellent. This workshop actually excited me and I look forward to the new websites and the ability to communicate with parents more effectively. My expectation was different from the presentation. The title of this presentation is misleading. I expected to learn how to effectively communicate with parents, not what's in store for us from marketing perspective. However, I am glad I took this session. This was informative in another way and I am excited to see the good thing coming to us. 33

Wasn't what I expected - thought it would provide more concrete info to give parents as well as answering questions during enrollment session Did not care for data regarding educated moms, etc....expected handouts on sample "here is how (centers)communicate with prospective parents" I was turned off with the idea of just marketing to mothers earning $75000 and more. My town doesn't have many of these mothers. We are looking for more specific communication skill improvement. "This workshop was informative, but it was mistitled. It was all about the new advertising campaign. I expected some additional advice about communicating with prospective parents who actually come in to the center to see me." Was disappointed with information. The title was a bit misleading...the actual instructor communication w/parents was not addressed fully

34