Table of Contents
Message from the Director of Quality Assurance Division ............................................................. 5
Learner Profile ............................................................................................................................. 6
Reflection on Learner Profiles....................................................................................................... 7
Will Technology-assisted, Work-based, Continuous Assessments Work? ......................................... 8
Using Social Media-based E-Portfolios in Competency-based Assessment ....................................... 9
What is Competency? ................................................................................................................ 11
Packaging of Competency Standard (CS) and Curriculum, Training and Assessment Guide (CTAG) 11
Competency Standard Template................................................................................................. 12
Key Changes to Competency Standard Template ........................................................................ 13
Revised Competency Standard Template .................................................................................... 14
Curriculum, Training and Assessment Guide (CTAG) .................................................................... 14
Curriculum, Training and Assessment Guide (CTAG) Components ................................................ 15
Key Changes to CTAG Template .................................................................................................. 15
Competency-based Assessment ................................................................................................. 17
Requirements of Competency-based Assessment ....................................................................... 17
Types of Assessment .................................................................................................................. 18
Commonly Used Assessment Methods ....................................................................................... 19
Knowledge-based Assessment Methods .......................................................................................... 19
1.
2.
3.
Projects ................................................................................................................................. 20
4.
Portfolio ................................................................................................................................ 20
5.
6.
8.
Observations ......................................................................................................................... 23
-2-
BLANK PAGE
Learner Profile
Learner profile is a set of information collected on how learner learns most
effectively. This information may be collected through observations, surveys,
questionnaires, application forms, interviews, etc. It may include information
like:
Age
Barriers to learning (e.g. institutional, situational, psychosocial, etc)
Educational level
Employability Skills (ES) Level
Experience (e.g. work experience, life experience, etc)
Motivations to learn (e.g. job, self-actualisation, etc)
Orientation to learning (i.e. immediacy of application for problem solving,
response to societal expectations, etc)
Readiness to learn
Self concept (i.e. self-directed, dependent, etc)
______________________________________________
______________________________________________
______________________________________________
______________________________________________
______________________________________________
Low-Skilled Elderly
Non-English Speaking
Physically challenged
Less confident
Fearful of training &
assessments
Wealth of experience
High expectations
Questioning
Intellectual
Tech-Savvy Gen Y
24/7 connected
Multi-tasking
Easily bored
Articulate and confident
Learner Profile
Approach 1
(ACTA V5)
Approach 2
(WTP)
Approach 3
(DACE)
Comments
Validity of
evidence
Sufficiency of
evidence
Currency of
evidence
Authenticity of
evidence
Characteristics
of Each
Approach
Continuous
assessment
Blogs
In-class
performance
Workplace
evidence
Live
observation
Video clip of
presentation
Skype to
cover gaps
Possible Takeaway
I can apply the following ideas to modify the practice in my organisation:
Background
IAL recently conducted a pilot run of a new design for four modules of
Advanced Certificate in Training and Assessment (ACTA). One of the new
methods that were tested was the use of social-media based e-portfolios for
continuous, formative and summative assessment.
Assessment Process
Learners started collecting evidence (artefacts) of learning and competency
from the start of the course. These took the form of pictures, reflective blog
entries capturing their thoughts on the application of learning, and videos of
them practising their delivery of training and assessment. Using a blog host
(Wordpress) and video host (YouTube), learners published their entries
frequently, allowing assessors, trainers and fellow learners to give feedback on
performance, comment on entries, and identify gaps to cover.
An Alternative Expression of Competency-based Assessment
This project built social constructivism, reflection and personalised
contextualisation into competency-based assessment, allowing assessors to
have a holistic, developmental view of the learners journey towards
competence.
Assessment Framework
Based on a portfolio assessment framework developed by Cleveland Clinic
Lerner College of Medicine for medical students1, IAL adapted it to create a
similar framework that includes e-portfolio and social media elements.
Summative
Assessment
Learners meet with
assessor to review all
evidence for decision
on competence
Smartphones, tablets
With photo-taking and
video-capturing
capability
Formative Assessment
Learners meet with assessor
to review evidence presented
so far to determine
performance gaps and
actions to take to close the
gaps
Evidence Portfolio
e-Portfolio using social
media like blog hosts
(e.g. Wordpress) and
video hosts (e.g.
YouTube)
Informal Feedback
Trainers, assessors and fellow learners
give feedback on blog entries for
developmental purposes
Evidence Database
Application to own workplace practice: Reflective entries
on applicability of learning to workplace
Teaching and assessment skills: Videos of skills practices
in class or from the workplace + reflection
Construction of knowledge: Photographs and entries of
own learning + reflection
Professional development: Reflective entries on
professional practice based on self and peer feedback
1
Dannefer, E. F., and Henson, L. C. (2007). The portfolio approach to competency-based assessment at the Cleveland Clinic
Lerner College of Medicine. Academic medicine, 82(5), 493-501.
What is Competency?
A competency is a measurable set of knowledge, skills, and attitudes that a
person needs to perform a task effectively
A competency is not an entire job
In most cases, several competencies are required for a job
As competency is task-based, the person may need to transfer the
competencies to new situations and environment
2nd Generation CS
3rd Generation CS
Status of Revision
Cover page
Retain
Relevant Job
role(s)/occupation(s)
Retain
Performance Statements /
Competency Element /
Performance Criteria
Underpinning Knowledge
Retain
Evidence Sources
Retain
Competency-based Assessment
A collection of evidence to demonstrate that a learner can perform or
behave according to specific standards
A form of assessment that is derived from the specification of a set of
outcomes articulated in the competency standard(s)
The assessment outcome that will allow the assessor to make judgment
with respect to the achievement or non-achievement of these outcomes
(i.e. competent / not yet competent)
Types of Assessment
Diagnostic
o Prior to learning, e.g. training needs assessment
Formative
o During learning, to gauge the learning progress and achievement of
learners for learning (Assessment for learning) does not contribute to
final assessment result
Continuous
o During learning, to gauge the learning acquired for competence/ grading
(Assessment of learning) contributes to final assessment result
Summative
o End of learning, to determine competence
Skills Recognition
o Recognition of Prior Learning (RPL)
Disadvantages
Disadvantage
3. Projects
Projects involve an assessment of a combination of subject knowledge,
process skills and transferable skills. Projects require learners to apply their
learning, encourage pro-activity and need an extended duration for
completion. The outcomes of a project may be presented through different
forms such as a report, portfolio, and/or presentation.
Advantages
Disadvantages
assessment methods
High validity as it requires the
application of skills and knowledge in
4. Portfolio
A portfolio refers to a collection of evidence assembled by the learner to
demonstrate competence. The main purpose of using a portfolio would be
the collection of evidence to establish that the learner has demonstrated a
set of prescribed performance criteria. A key point to note is that evidence
that was accumulated many years ago may not be current and has to be in
line with current practices and standards.
Advantages
Disadvantages
Disadvantage
6. Oral Questions
Oral Questioning require learners to articulate their understanding through
a verbal face-to-face question and answer (Q&A) process. This method is
useful when addressing a diverse group of learners with varying literacy
levels as the assessor is able to paraphrase questions to help the candidates
in understanding.
Advantage
Disadvantage
Performance-based Methods
7. Workplace Performance
This is arguably the best method to collect reliable and authentic evidence
of a learners actual competencies. This is because the assessment context
is set under actual work conditions that require demonstration of actual
occupational competencies and maximises the degree of realism in the
assessment process.
Advantages
Disadvantages
8. Role-Play / Simulation
Role-playing or simulation provides an alternative for workplace
performance. It can be seen as an orchestrated scenario that seeks to
gather evidence on a learners competencies.
Advantages
Disadvantages
Detached from
conditions
realistic
workplace
Disadvantages
Phase I
By 1 Oct 2012
Phase II
By 1 Oct 2013
At least 65% with
Workplace Trainer
Programme (WTP) or
equivalent
Phase III
By 1 Oct 2014
Master Trainer
Curriculum Developer
Note:
New ATOs or existing ATOs who are moving into new areas of training are required to meet
requirements in Phase I with immediate effect.
WSQ In-house ATOs are companies/organisations that train and/or assess their own staff
internally.
Figures are based on declaration during accreditation and deployment
* Can be the same person
Public WSQ ATOs
Role
Phase I
By 1 Oct 2012
Phase II
By 1 Oct 2013
Phase III
By 1 Oct 2014
Curriculum Developer
Note:
New ATOs or existing ATOs who are moving into new areas of training are required to meet
requirements in Phase I with immediate effect.
WSQ ATOs are external training providers that are in the business of conducting training and/or
assessment
Today, curriculum developers of ATOs are already required to meet ACTA or its equivalent. They
will also be required to meet the new DACE requirement by 1 Oct 2015.
Figures are based on declaration during accreditation and deployment. ATOs are required to
ensure that its actual deployed AEs for each class delivery under each approved framework meet
the qualifications requirements
Partial Completion of DACE: Individuals who have already attained DACE competency units (CU)
C2, C3, C5, C6 and C7, may function as WSQ Trainers and Assessors at WSQ ATOs in lieu of
completing the ACTA Qualification. Such individuals are encouraged to complete the remaining
units of the DACE qualification as the full DACE qualification will enable them to practice as a
WSQ Curriculum Developer at WSQ ATOs.
- 25 Singapore Workforce Development Agency Quality Assurance Division
Develop Competency-Based Assessment Plans
Version 1.1 (14 October 2012)
Problem: This role play script does not provide valid evidence of competence as
the learners only needs to follow the script and the flow of the script. In a real
workplace situation, the learners will not be given such a script when providing
customer service. Learners should not be in the know of exactly what to expect
during each customer service provided.
General Advice: Role play scripts should be developed appropriately to guide
and ensure that the learners are adequately assessed through a role play based
on a real workplace situation. Detailed scripts should only be provided to the
actor (i.e. person not being assessed) to ensure that the role play does not
deviate from the intended flow of the role play. A scenario may be provided to
the learner to explain the context of the role play and the instructions to the
learner should generally cover what he/she will be assessed on during role play.
A sample role play script developed to assess learner based on a real work
place situation is as follows:
Role Play Scenario
It is a Sunday morning and your department store is going to launch a new product called the
iRobot. Your organisation anticipates a huge demand for this new electronic item as shown by
the large sales figures in other countries.
Role Play Script (Candidates Copy)
You are the retail assistant at XYZ Company. You turn up for work on time this morning and get
ready for work. You know that there is a new HOT item to be launched today. You are to
deliver service excellence to your customer according to your organisational policies and
procedures.
You will be assessed on the competencies in Deliver Service Excellence, i.e.
Use organisational service culture and values to guide service delivery
- 29 Singapore Workforce Development Agency Quality Assurance Division
Develop Competency-Based Assessment Plans
Version 1.1 (14 October 2012)
Specifications
Guidelines
Assessment Method
Role Play
Performance Criteria
Underpinning Knowledge
All stated UK
Duration
20 minutes
Assessment Venue
Assessment Resources
Assessment Checklist
Assessment Questions and Answers
Conduct of Assessment
Conduct Interview
in
the
10
11
Communicate
the
performance management
system to employees and
their
roles
and
responsibilities
in
performance management.
Assessment Criteria:
Ability to communicate the
importance of performance
management and the roles
and
responsibilities
to
stakeholders
Work Activity:
Able to verbally communicate and adapt
communication methods of the roles and
responsibilities of performance management to
line managers and employees
Role Play
AA1
The assessment
method chosen
here is role play
ASSESSMENT CHECKLIST
Performance Criteria and
Underpinning Knowledge
1.2 Communicate the performance
management system to employees
and their roles and responsibilities
in performance management.
Learning journal
Method
Instructions to
Assessors
1.2
Instructions to
Candidates
Performance Criteria
Assessment
checklist
Assessment Tools
Please Tick
Assessment Criteria
Remarks
C
NYC
The assessment
Suggested Answers:
Candidate is able to explain how to communicate the performance
checklist states that
management system to employees and their roles and responsibilities
in performance management, which may include:
is an oral question
.............................
.............................
it
Conducting the
Role Play
Stakeholders involved:
Areas of Focus:
Assessment criteria and evidence requirements
Assessment environment
Assessment methods and tools
Principles of Assessment / Rules of Evidence*
Practicality
Clarity of instructions
Cost effectiveness
Contextualisation (organisation, industry and legal) and learners
requirements
______________________________________________________
______________________________________________________
Assessment Venue:
Follow up:
Underlying Principle
Assessment
activities
relates directly address
the PS / PERFORMANCE
CRITERIA and UKs
Assessment tools address
the candidates ability to
meet the level of
performance required by
the Competency Standard
Assessment
activities
planned are holistic and
integrated
Compliance
Yes
No
Mitigating Factor
(if noncompliant)
Underlying Principle
Compliance
Yes
Where
applicable,
assessment specifications
and tools are able to
produce
consistent
assessment
outcomes
conducted in different
contexts
Assessment specifications
Assessment tools: case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Assessment / Observation
Checklist/s
Instructions to assessors
Assessment specifications
Other assessment tools (where
applicable): case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Assessment methods
Assessment criteria
Assessment tools: case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Instructions to candidates
Instructions to assessors /
candidates
Assessment specifications
Other assessment tools (where
applicable): case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Assessment methods
Instructions to assessors /
candidates
Assessment criteria
Other assessment tools (where
applicable): case study
- 41 -
No
Mitigating Factor
(if noncompliant)
Underlying Principle
Policies,
procedures,
guidelines and standards
are put in place to
support assessors in the
consistent application of
reasonable adjustment/s
where required
Candidates are given
clear
and
timely
information
on
assessment and what is
required.
Information provided to
candidate/s covers:
the
assessment
method/s
assessment process
Compliance
Instructions to candidates
Yes
No
Mitigating Factor
(if noncompliant)
Underlying Principle
Assessment methods
Instructions to assessors /
candidates
Assessment criteria
Other assessment tools (where
applicable): case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Instructions to assessors /
candidates
Assessment specifications
Instructions to assessors /
candidates
Assessment specifications
Instructions to assessors /
candidates
Assessment specifications
Rule of Evidence Validity
Assessment specifications
Assessment criteria
Other assessment tools (where
applicable): case study
scenarios, role play scripts,
scenarios, assessment
checklists, observation
checklists, etc
Rule of Evidence Sufficiency
Assessment specifications
Assessment criteria
Other assessment tools (where
applicable): case study
scenarios, role play scripts,
- 43 -
Compliance
Yes
No
Mitigating Factor
(if noncompliant)
Underlying Principle
PERFORMANCE CRITERIA
and UK
* As a general guide,
there should be multiple
sources of evidence (e.g.
at least two methods per
assessment plan)
Evidence collected is
based
on
current
situation and is not
outdated.
Compliance
Yes
No
Mitigating Factor
(if noncompliant)
WSQ CA requirements
1.1.2
No
Yes
No
Yes
No
1.1.3
Yes
Is the time allocated for learning and activities appropriate and sufficient?
1.1.4
Yes
No
Yes
No
Yes
No
Note:
If the proposed ratio exceeds or do not meet specified recommendations, ATO
may be asked to provide justification
1.1.5
Glossary provided
Version control
1.2.2
Yes
No
Yes
No
Yes
No
Yes
No
Is Validity ensured?
o evidence meets the specified criteria of the competency standards
Is Authenticity ensured?
o evidence produced is/can be/will be verified to be the candidates own
work
Is Sufficiency ensured?
o enough evidence is gathered to show competency against all specified
criteria of the standard/s
Is Currency ensured?
o Evidence to be gathered are current, relevant and applicable in todays
context
1.2.3
1.2.4
Note:
ATO may be asked to provide explanations and justifications of the
assessment design, methods and tools employed
1.2.5
Note:
ATO may be asked to justify and provide explanations if the proposed ratio
exceeds or do not meet specified / recommendations
1.2.6
Assessment tools are provided e.g. role play scripts, answers to written
tests
Glossary provided
Version control
Yes
No
Yes
No
Domain qualification
AE qualifications
AE experience
Note:
CVs of all AEs to be submitted for reference and verification. ATO may be
further asked to provide copies of relevant certificates/scrolls/transcripts.
If subject matter experts (SMEs) are used, the SME/s possess adequate:
o
Note:
CVs of all SMEs to be submitted for reference and verification. ATO may be
further asked to provide copies of relevant certificates/scrolls/transcripts.
ATO is to also note that submission of profiles for SMEs is not sufficient.
1.4 Facilities and Equipment
1.4.1
Yes
No
Yes
No
1.4.2
Training and assessment site/s comply with all fire, health and safety
requirements
References
Dannefer, E. F., and Henson, L. C. (2007). The portfolio approach to
competency-based assessment at the Cleveland Clinic Lerner College of
Medicine. Academic medicine, 82(5), 493-501.
Singapore Workforce Development Agency (2007). WSQ Assessment
Design Guide. Assessment methods 1322.
Institute for Adult Learning (2011), DACE Develop Assessment Tools,
Learner Guide from IAL
Institute for Adult Learning (2008), CU5 Develop a Competency-Based
Assessment, Learner Guide from IAL
Useful Websites
CA self-assessment checklist http://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQCurriculum%20Development%20&%20PreAccreditation%20Checklist(1Aug11)v1.pdf
WSQ terms and conditions https://www.skillsconnect.gov.sg/web/guest/termsAndConditions#
External Guide for Continuous Improvement Review http://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8
)4Nov11(A).pdf
Guidelines for submitting results and printing SOAs http://app2.wda.gov.sg/data/ImgCont/487/OnlineSubmissionForm.zip
Practical Guide on Managing For Training and Assessment Excellence http://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForu
mheldon24May07.pdf
Guide on updating WDA on changes in Course Information
http://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.
pdf
Guide on updating WDA on changes in Organisation Information http://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationP
rofile.pdf
Credit exemption guide - http://www.ial.edu.sg/index.aspx?id=81
Effective Date
1.0
27 August 2012
1.1
14 October 2012
th
th
Changes
Initial version
Inclusion of additional notes for
CD workshop for Public ATO
Author/s
QAD
QAD / IAL