Running head: EVALUATION PROCESS

Evaluation Process
Dan O'Brien
CUR 528
November 14, 2016
Professor Michael Burnett

EVALUATION PROCESS

2
Evaluation Process

Program Description
The Fullerton College Machine Technology Program is one of four community college
machinists training programs in the Orange County area. Along with the community college
programs there are a few private schools and urban workshops offering training in this area.
Neighboring Los Angeles County has eight programs along with a large private school operated
by the National Tooling and Machining Association. The program at Fullerton offers traditional
machining skills, computer numerical controlled (CNC) set up and operation. CNC
programming classes are offered in several different software formats. The program has grown
from a student population in 2005 of forty enrolled to over one hundred and forty students in
2016. Program staff has grown from one fulltime instructor and two adjuncts to two full time
instructors and seven adjunct faculty members. The program currently offers sixteen different
courses each semester in twenty two sections. We offer six certificates for various levels of skill
obtainment along with an Associate in Science degree majoring in manufacturing.
New course and program proposals are currently in the curriculum process and will add
seven new courses and three new certificates. A new metrology program will be in place
offering a certificate in advanced inspection and measuring techniques utilizing laser scanning
technologies, coordinate measuring machines and industry standard inspection software. CNC
Swiss screw machining will also be included as a new certificate program.
Description of the problem
Program growth of this department can be attributed to recruiting and promotional
activities and growth of the need for manufacturing related employment. Activities have
consisted of the participation in many high school career days, speaking to high school classes

EVALUATION PROCESS

and offering special grant related programs for high school students. During this growth
additional staff and courses were added with addition of a second full time faculty member in
2015. The efforts that created this growth has gone unmeasured and unrecorded leaving
questions as to where gaps are in program recruitment. Some schools and local business could
see the benefits of our program while others appear unreachable. Our local industrial base is in
need of filling high skilled employment positions due to an aging and retiring workforce.
Student graduates of the program have also left untracked. Some do return, some call to advertise
employment positions for their companies. Some are employed as machinists, CNC
programmers, others are now supervisors and some even own their own companies.
Purpose of the Evaluation
This evaluation is to determine where efforts have been successful in recruitment
activities and what areas are in need of improvement. Recruitment and promotional efforts
directed at school counselors needs to be recorded and data used to determine where efforts are
best directed. High school teachers and counselors need to know the available employment
opportunities and also the link between manufacturing education and transfer to four year
engineering schools for students. Business and industry representatives knowledge and hiring of
students graduating our programs also needs to be recorded.
Data Collection Procedures
Data collection was obtained with the aid of surveys directed at each of the four groups
studied in this evaluation, high school students looking for career ideas, student graduates who
have completed the program, high school teachers and counselors and manufacturing
stakeholders. Surveys were designed to record responses of 1-5 ranging to include highly likely,
likely, maybe, probably not and no. Some yes and no questions were included. Fifty surveys

EVALUATION PROCESS

were delivered to participating high school teachers for the high school student surveys. Twenty
academic stakeholders including high school teachers and counselors and twenty surveys
delivered to manufacturing stakeholder business contacts. Fifty surveys were chosen for the high
school group as a smaller return rate was expected. A high rate of survey return was expected
with program graduates, high school teachers and counselors, and also manufacturing industry
representatives. The high school survey return rate was as expected along with those of
graduates and school staff. The surveys delivered to industry stakeholders were returned at a rate
that was less than expected.
Procedures for Analyzing the Data
Survey results will be compiled for each of the four individual survey areas. The Delphi
focus group will be utilized and provided with the results on an individual basis for independent
study and interpretation of the data. A short informal response shall be recorded by each member
of the focus group for comparisons and discussion to follow. A group meeting shall be held
utilizing the Delphi Technique. A group leader will record informal responses of individual
group members and their thoughts on what reasons were for the data findings. Discussion will
follow as to why data was showing positive or negative measurements. Activities that lead to
positive measurements shall be identified and continued across the program. Results of negative
responses will be discussed for reasons of negative measurements and actions that could be taken
to improve those measurements on future evaluations. Group members will vote on individual
ideas and thoughts that would be the top reasons for program success and will also utilize this
process for determining which actions should be taken to improve negative evaluation results.

EVALUATION PROCESS

Results / Findings
Results from the high school student surveys indicate additional information and
recruiting efforts need to be directed at high school students. Students need to be informed of
career opportunities and benefits for students entering engineering programs. The new
curriculum and certificate offerings with our program will be courses that can be transferred
directly to four year engineering schools. High school students who are knowledgeable of our
program recorded very positive results. A low return rate of surveys delivered to high school
students was expected and was a similar rate as the positive ratings of the surveys returned.
These lower results were discussed and were determined to be due to lack of knowledge and
interest in employment, lack of class discussions with teachers and counselors regarding careers
and employment training. Another reason possible for these low results is that high school
students are told they need to go to college to be successful. Careers that dont require college
degrees are not discussed. A common problem is high school teachers and counselors are only
concerned helping students attend college and will forget about the students who are not on that
path.
Graduates of our program yielded the most positive survey results. These results
indicate a high degree of satisfaction with program outcomes. Success of these students and their
current career paths are indicated in the survey as CNC programmers, supervisors, those enrolled
in engineering programs and degreed engineers. With some additional communication with
alumni students the focus group feels these graduates may be some of the best ambassadors to
represent the program. This evaluation indicates a need to involve these former students in
program activities.

EVALUATION PROCESS

Academic surveys indicate more work needs to be done with informing high school
teachers and counselors of the benefits of this program as discussed in the survey findings of the
high school student group. Traditionally academic counselors, teachers and administrators have
been the most difficult group to reach. The only focus is to get students college ready and into a
four year school after high school graduation. This group has completely ignored the students
who are not college bound and are ignorant to employment opportunities and the training that is
required for such employment. When informed of program offerings and benefits this group was
surveyed as showing a more positive result. The academic counseling and teaching groups that
work with our program responded with very positive survey results.
Industry stakeholders are the direct benefactors of this program. Representatives at local
companies are very busy and this was indicated with a lower than expected rate of returned
surveys. Additional efforts are needed to involve manufacturing company owners and
supervisors, they are the benefactors of our program and the focus group feels they should be
included in more department functions. When surveys were returned results were generally
positive. Some training gaps were indicated by employers and these results will provide
feedback to the instructional staff.

EVALUATION PROCESS

7
References

Royse, D., Thyer, B., & Padgett, D. (2010). Program evaluation: An introduction (5th ed.).
Belmont, CA: Cengage Learning.
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco,
CA: Jossey-Bass
Walvoord, B. E., & Banta, T. W. (2010). Assessment clear and simple: A practical guide for
nstitutions, departments, and general education (2nd ed.). San Francisco, CA: JosseyBass.