MS ACCESS

Evaluation Plan

MS ACCESS
Evaluation Plan
Evaluation Process
This Microsoft Access tutorial module will evaluate the quality of its instruction by utilizing Kirkpatrick Level 3 (Behavior) evaluation methodology. In order to accomplish a measurement of behavioral change, incremental student assessments must be administered at various milestones of this course. The data will be collected utilizing the survey method and the use of questionnaire. This questionnaire will utilize Likert-style questions (close-ended) in which the responses translate to numerical values.

Evaluation Plan

Steps for Evaluation

1. 2. 3. 4. Administer Pre-Course Assessment Administer Mid-Course Assessment Administer End-of-Course Assessment Analyze Data and Post Results of Training Evaluation

Tools
Survey instrument will consist of 10 Likert-style questions targeting the following areas: 1. Self-Assessment of Knowledge of Microsoft Access a. I know what a database is. b. I know what databases are used for. c. I know the parts of a Microsoft Access database (i.e. table, form, report, query, etc.).

Evaluation Plan

d. I know how to navigate the Microsoft Access menu bar. e. I can successfully create a simple table, query, form, and report. 2. Personal Attitudes Towards Creating a Microsoft Access database a. I see the value of Microsoft Access databases in the home and in the office. b. I am confident in my ability to operate Microsoft Access. c. I am apprehensive towards learning about Microsoft Access. d. I can never see myself building a Microsoft Access database. e. Microsoft Access is not designed to accommodate beginners. 3. Response Options a. 1 = Strongly Agree b. 2 = Agree

c. 3 = Disagree d. 4 = Strongly Disagree

e. Note: The option for neutral is deliberately omitted because the learner has to either possess some form of knowledge of Access or they do not have any knowledge of Access. Each version of the questionnaire (pre, mid, etc.) will have varied wording in the questions in order to avoid blatant duplication as perceived by the learner and to avoid having the learner become confused as to why they are filling out repeated questionnaires. The pre, mid, and post-course questionnaires will be paper-based. Maximum participation will be encouraged. Participants will be informed that their participation will aide in the refinement of this course. Participation will remain anonymous and an informed consent will be included on the header section of the questionnaire. No personal identifiable data elements will be required in order to assess the quality of the instructional module.

Data Analysis
1. Utilizing Microsoft Excel for data analysis, the numerical data values derived from the completed questionnaires will be input into a spreadsheet and segregated onto separate worksheets by pre-course evaluation, mid-course evaluation, end-of-course evaluation, and 6-month post course evaluation. Within each worksheet the data will be broke out by the two separate questionnaire groupings (i.e. Self-Assessment of Knowledge of Microsoft Access, Personal Attitudes Towards Creating a Microsoft Access database). Each question will be included in each grouping and for all worksheets.

2. For each question, all response data values will be input onto the row. On the far right column will be a cell for overall mode value. This will depict the most frequent response. For the overall group, a mean (average) of mode values will be calculated and also the standard deviation of the group responses. This process will be replicated for the second group and for all worksheets. For each group and for each worksheet, we will use the mean calculation of the values to state the assessed knowledge level and the assessed attitudes for each stage of the course instruction.

a. Example: For the pre-course evaluation knowledge section, the mode value may reflect 2 (agree with the statement) and 3 (strongly disagree with the statement). The same may be indicated for the attitude section. The average calculation of the mode values (e.g. mean = 2 for both sections of this example) may indicate that the audience has some knowledge of MS Access and are positive about learning about MS Access. b. Upon calculating the mean values of each section for each stage of the instructional course, we would have data values that could be trended on a line graph, one graph for knowledge, and the second for attitudes.

Evaluation Plan

Instructional Goals
1. 2. 3. 4. 5. 6. 7. To help learners see the advantages of using Microsoft Access over spreadsheets Learners will be able to select the correct definition of a database Learners will be able to demonstrate the process for starting a new database The learner will be able to list in order the steps for creating a query The learner will be able to recall the steps for creating a report The learner will be able to list the steps for creating a table The learner will be able to recall the steps for creating a form

Alignment of Unit Goals to the Evaluation Process

Goal #1 aligns to the end of the course evaluation and Post Training project. By completing the training the learner will be able to see the benefits of learning how to create and use simple databases versus using multiple spreadsheets to create queries and gather information on more than one topic at a time. By being able to correctly identify and use the different objects of a database the learner shows transference of knowledge from the instructor.

Evaluation Plan

Goals #2 - #7 all align to the Mid-Course evaluation and the post training evaluation. Throughout the module the learner will have the opportunity to learn the different objects of a database and how to use those objects to create and use a simple database. By being able to recognize and define the different objects of the database, the learner is showing a gain in knowledge.

SEQUENCE OF ASSESSMENTS
Method of Evaluation Stage of Instruction Tool Type (Similar questions, wording varied to a small degree) Questionnaire using Likert-style questions Description of Data to be Collected Objective

PreAssessment Survey Method

Prior to beginning instructional module

Mid-Course Survey Method

Questionnaire administered at mid-point of instructional module

Questionnaire using Likert-style questions

End-of-Course Survey Method

Questionnaire administered immediately at end-of-course

Questionnaire using Likert-style questions

Questions designed to ascertain knowledge and attitudes towards MS Access database creation Questions designed to ascertain knowledge and attitudes towards MS Access database creation Questions designed to ascertain knowledge and attitudes towards MS Access database creation

Establish learner knowledge baseline (Preinstructional state) Establish a progress indicator (positive or negative)

Identify state of behavior upon completion of learning module

Evaluation Material & Tools Pre-Assessment Strategies: E3 Learning Solutions has conducted a needs analysis to determine the background, attitudes, infrastructure, and previous knowledge level of the learners. This needs analysis was conducted by using a survey method by means of disseminating a questionnaire to the 4 learners. The Pre-assessment survey is to get a baseline of the learners knowledge of Microsoft Access. This data will later be compared to the mid-assessment and postassessment to see the growth in knowledge of the learners. This level of evaluation corresponds to the Kirkpatrick scale of evaluation at level 2.

Evaluation Plan

 Formative Assessment Strategies: Throughout the training session learner comprehension and skills ability will be checked using question and answer breaks, a list of questions is listed in Appendix A. The learner will have an opportunity to ask questions of the instructor to have a clear understanding of the processes being explained to them when using Access. There will also be a mid-program assessment to check the knowledge growth of the learners. The formative assessment, question and answer sessions are based on level 2 of the Kirkpatrick scale, learning. This level of evaluation is measuring the change in the knowledge and skills of the learners. This data will be compared to the pre-training survey data to measure the change and growth in the level of the learners knowledge at the halfway point of the training.

Summative Assessment Strategies:

Evaluation Plan

At the end of the training session there will be a quiz to test the knowledge and skills of the learners. At the end of the training there will be a post-training assessment to measure the growth of the learners knowledge. This level of assessment will be at level 2 of the Kirkpatrick scale. This assessment survey is to test what knowledge the learners have gained. There will also be a final activity that each learner will complete that will measure the change in their behavior. The activity will consist of the learners using a template to create a new database. The activity will test to see if the learners are able to apply the new knowledge they have gained in a real world situation. This level of assessment is at level 2 of the Kirkpatrick scale, learning. This is normally a long term evaluation that would occur several weeks to months following the training. However, by creating the real-life activity and having the learners apply their new knowledge immediately we are able to get a general scale of the effectiveness of the training.

Rationale for Evaluation Tools and Materials

This training module was developed to teach MS Access to learners that have no knowledge of databases and no knowledge of how they are created. The evaluation tools were created to test the growth of the learners knowledge at different stages of the training. The survey evaluation is based on Likert-style questions and can be quickly evaluated to measure the effectiveness of the training. There is a Post-training project that will allow the learner to immediately put to use in a real world situation the knowledge gained from this module. The learner will create a simple database using Instructor provided data to create tables, forms, queries, and reports form the database of information.
Alignment to the Five Levels of Evaluation This Evaluation Plan meets the first three levels of Kirkpatricks Levels of Evaluation. Level 1: Reaction This level of evaluation measures the learners attitudes toward the module; did they enjoy the module, did they find it helpful, do they feel the module accommodated their style of learning, etc. Level 2: Learning - We start by measuring the learners prior knowledge of the material they are going to be trained on, during the training, and post training. By evaluating the learner throughout the training we can get a more measurable gain in their knowledge. The posttraining project will measure learning based on the ability to transfer the knowledge from the module to a real life situation. Level 3: Behavior Usually behavior is measured after a pre-determined amount of time posttraining. However, since the learners of this module are novices with Microsoft Access, the posttraining evaluation will be used to determine the changes in their behavior by measuring if they can correctly list the steps to creating a simple database.

Summary of Modifications
The original evaluation plan was modified to fit the current level of experience of the learners. The same types of evaluation tools were designed. The 6 month post evaluation has been eliminated since this training is not for the company it was originally designed for. The formative questions for the question and answer breaks throughout the module have been added, as well as the Kirkpatrick level of Evaluation added to each assessment. The full Power Point presentation has also been added to this plan.

Evaluation Plan

Appendix A
Formative Assessment Questions:

Who can define a database? What are some of the common uses of a database? What is an advantage of using a database over a spreadsheet? What is a table? What is a query? How might a query be used? Why is it good to know how to create a report in Access? What is a form? How comfortable are you with using Access?

Evaluation Plan

Appendix B
Sample Materials:
Quick Access Card:

Evaluation Plan

MS Access Power Point:

10

Evaluation Plan

Evaluation Plan

11

12
Evaluation Plan

. Evaluation Plan

13

14
Evaluation Plan

Evaluation Plan

15

16
Evaluation Plan

Evaluation Plan

17

18
Evaluation Plan

Evaluation Plan

19

Appendix C
Evaluation Tools: Pre-Assessment Pre-Training Assessment Kirkpatrick Level 2: Learning 5 Strongly Agree 4 Agree 3 Neutral 2 Disagree 1 Strongly Disagree ______ 1. I know the advantages of using a database versus spreadsheets. ______ 2. I can correctly define what a database is. ______ 3. I can identify the different objects of a database. ______ 4. I can demonstrate the process for starting a new database. Evaluation Plan ______ 5. I can list in order the steps to create a query. ______ 6. I can list in order the steps required for creating a table. ______ 7. I can correctly list the steps required for creating a report. ______ 8. I can correctly list the steps in order to create a form. ______ 9. I can correctly navigate the Microsoft Access menu bar. ______ 10. I can create a simple database using Microsoft Access.

20

Mid-Training Assessment Kirkpatrick Level 1: Reaction 1 (Poor) I can define the advantages of a database over a spreadsheet I can correctly define what a database is I can correctly start a query I can correctly create a report Evaluation Plan 2 (fair) 3 (Good) 4 (Excellent) 5 (Unsure/Not Applicable)

Post-Training Assessment Kirkpatrick Level 2: Learning Rank in order the steps required to create a database from a template. ___ Open MS Access ___ Under the Templates heading choose a type ___ Click Download to retrieve the template ___ Click New on the Sidebar ___ Follow the onscreen Prompts to complete the table ___ Select a Template ___ Select the File Tab

21

Post Training Project - Kirkpatrick Level 3: Behavior (Is the learner able to transfer the knowledge and skills gained by the training?) Open MS Access, create a database based on the following information: Customer Name Mary Jane Watts Address 1234 Your Street Anywhere, NY 12345 Ben Tennison 654 St. Elsewhere Baton Rouge, LA 67854 Evaluation Plan Bruce Wayne 23 Hamburg Pike Gotham City, PA 02589 Clark Kent 4456 Farmview Dr. Metropolis, IL 34896 Bruce Banner 841 Nowhere Drive San Fransico, CA 90225 333-412-9635 Bachelor of Science Mathematics and Philosophy 423-665-325 966-365-8888 Bachelor of Science Information Technology 876-258-1147 Phone Number 789-555-9876 Degree Bachelor of Science Subject Psychology

Master of Arts

Mechanical Engineering

Master of Science

Physics

University Nova Southeastern University

Address 3301 University Drive Fort Lauderdale, FL 33314

Degrees Bachelor Masters Doctoral

Programs Accounting Art Biology Business

Website www.nova.edu

22

Administration Psychology Physics University of Louisville 580 S. Preston Street Louisville, KY 40202 Bachelor Masters Doctoral Art Business Chemistry Engineering Information Technology Philosophy Psychology Stanford University 450 Serra Mall Stanford, CA 94305 Bachelor Masters Doctoral Business Earth Science Education Engineering Law Medicine Evaluation Plan www.waldenu.edu www.ucla.edu www.stanford.edu www.louisville.edu

Walden University

100 Washington Avenue South, Suite 900 Minneapolis, MN 55401

Certificate Bachelor Masters Doctoral

Communication Criminal Justice Information Technology Psychology Nursing

University of California Los Angeles

405 Hilgard Avenue Los Angeles, CA 90095

Bachelor Masters Doctoral

Science Arts Engineering Nursing Theater, Film, Television

23

References
Kenkre, A. (2012). Identifying learning object pedagogical features to decide instructional setting. Informally published manuscript, Indian Institute of Technology Bombay, Mumbai, India. Retrieved from http://www.academia.edu/2614614/Identifying_Learning_Object_Pedagogic al_Features_to_Decide_Instructional_Setting Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.
Evaluation Plan

Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2011). Designing effective instruction (6th ed.). Hoboken, NJ: John Wiley & Sons, Inc. Screenshots captured from http://www.gcflearnfree.org/access2010

Smith, P. L., & Ragan, T. J. (2004). Instructional design. (2nd ed.). Hoboken, NJ: John Wiley & Sons, Inc. Retrieved from http://steinhardtapps.es.its.nyu.edu/create/courses/2174/reading/smith_ra gan_1_2.pdf

24