Evaluating

Psychoeducational
Groups
Lecture team
Basic and Group Intervention Techniques
Universitas Islam Bandung
INTRODUCTION (1)
• Evaluasi terlalu sering diabaikan atau diabaikan tetapi sangat penting untuk
perbaikan dan melayani beberapa tujuan penting. Ini menyediakan data untuk
menilai yang berikut:
• Seberapa baik peserta mempelajari materi.
• Perubahan perilaku dan sikap.
• Kurang atau lemahnya materi yang disajikan dan / atau
strategi pembelajaran.
• Keberhasilan perencanaan.
• Kepuasan peserta.
• Hasil dan efek yang tidak terduga.
INTRODUCTION (2)
• Tidak peduli seberapa baik Anda merencanakan, ada kalanya Anda mungkin
memiliki beberapa kelemahan atau kekurangan materi.
• Anda mungkin telah menggunakan berbagai strategi dengan grup Anda, dan
Anda akan ingin mendapatkan reaksi peserta untuk masing-masing untuk
membantu dalam perencanaan masa depan.
• Hasil evaluasi memberikan data tentang kemanjuran perencanaan Anda. Ini
dapat memperkuat untuk menghabiskan begitu banyak waktu dan upaya untuk
tugas ini atau dapat menunjukkan di mana lebih banyak perhatian diperlukan.
• Menilai kepuasan peserta dengan kelompok, kepemimpinan Anda,
pembelajaran mereka, dan prestasi mereka adalah dasar untuk memahami
keberhasilan atau kegagalan pengalaman.
DEFINITION OF TERMS (McMillan
&Schumacher, 1997)
• Evaluasi menentukan nilai dan / atau penggunaan suatu program, produk, prosedur, tujuan,
atau proses. Ini adalah kerangka kerja untuk membuat penilaian dan keputusan.
• Assesment / Penilaian mengukur suatu variabel seperti sikap atau kepuasan.
• Measurement / Pengukuran, melibatkan penetapan angka ke variabel atau karakteristik
sehingga mereka dapat dikumpulkan, dianalisis, atau dibedakan. Mode untuk pengukuran
menggunakan salah satu dari empat skala: nominal, ordinal, interval, atau rasio.
• Formative evaluation / Evaluasi formatif terus mengumpulkan dan menganalisis data untuk
membuat perubahan atau penyesuaian langsung saat acara, seperti kelompok, sedang
berlangsung.
• Summative evaluation / Evaluasi sumatif mengumpulkan dan menganalisis data setelah
acara selesai.
• Instruments adalah perangkat dan proses pengumpulan data seperti tes, kuesioner, laporan
diri, skala penilaian, sistem observasi, dan jadwal wawancara.
SO, THE DEFINITION IS ?
• Evaluation is the total process that incorporates all the other topics:
assessment, measurement, formative and summative evaluations,
and instruments.
• Planning for evaluation begins before the group ever meets. Deciding
what information you want, planning for when and how to collect it,
and selecting suitable instruments will help you prepare for effective
evaluation.
EVALUATION STEPS
1. Start with the goals and objectives, and decide how to assess them.
2. Identify the various strategies you will use, such as exercises and
lectures, and how they will be assessed.
3. Identify expected outcomes, such as behavior changes, and
determine if or how they can be assessed.
4. Plan to assess participants’ satisfaction.
5. Select or develop needed instruments.
EVALUATING GOALS, OBJECTIVES,
AND STRATEGIES
• One way to begin the process is to list the group’s objectives and
strategies, then tie them to the method of evaluation.
• Ex : one objective may be to have participants learn a body of
material. The strategy to be used is a minilecture. Evaluation of
learning might include having participants take a test, brainstorm or
write applications of the material, rate the presentation, or judge
their impression of how much they learned. You could also use pre-
and post-testing to determine gains in learning.
IMPORTANT !
• Listing objectives and strategies allows for prioritizing so that you
evaluate what is most important and do not waste time over-
evaluating.
• It is not necessary or desirable to evaluate everything in-depth.
Instead, focus on the important elements and plan a short evaluation.
Sometimes all you need is feedback on participants’ perception of the
major activities.
• A simple form listing these activities, a method for rating them, and
space for written comments may suffice.
ASSESSING LEARNING (1)
• When you assess learning, it is helpful to know the baseline
knowledge possessed by participants as a beginning point.
• What you really want to know is how much learning occurred because
of group activities.
• So, just testing after the group is finished is not sufficient. You really
need to do a pre-group test and a post-group test. The ideal situation
is to do these two tests and a follow-up test weeks, months, or years
later to determine how much material was retained.
ASSESSING LEARNING (2)
• Learning focuses on specific material, such as definitions and facts. Thus, assessing
learning will involve items that are directly related to the material presented.
• Assessment is similar to classroom tests, and there are considerable resources that
describe how to construct valid and reliable tests.
• It is seldom that psychoeducational groups will involve higher levels of thinking
such as synthesis, which is best assessed in other ways. It is best to use objective
items because of the group's short-term status, the briefness of material
presented, and the highly focused nature of the presentation.
• Objective items such as multiple choice, true-false, and matching are suggested as
appropriate. Be sure to ask for information that was presented to get a realistic
picture of what was learned andunderstood.
FORMATIVE AND SUMMATIVE
EVALUATION (1)
• Formative evaluation that is used to make adjustments and
adaptations for continual improvement.
• This approach allows you to have concrete justifications for changes,
not just your subjective judgment.
• It provides for a more systematic process for making decisions.
• However, you do have to plan to gather and analyze information in
advance for formative evaluation.
• It is not feasible to try to do this after the group has begun if it was not
in the original plan, because you will be distracted and possibly cause
disruption to the group
FORMATIVE AND SUMMATIVE
EVALUATION (2)
• Summative evaluation where data are gathered at the end of the experience and
analyzed at some later time.
• Summative evaluation can give you information about gains in learning, changes in
behavior and attitudes, and the effectiveness of the various group components.
• The gains in learning and changes in behaviour and attitudes are possible when you
assess these at the beginning and end of the group.
• This procedure can allow for you to make some judgments that are supported with
data about the extent of participants’ learning and the efficacy of the group.
• However, unless you are doing this with a well-constructed research plan, these
judgments can only be very tentative and general.
• You cannot make definitive conclusions; you can only consider possibilities suggested
by the data.
INSTRUMENTS
• Instruments are the means by which you gather data.
• Once you decide on the elements you want to evaluate, you must consider your
instruments
• In reviewing them, you should pay attention to the established validity for the kind of
participants in the group, particularly with regard to age, gender, racial/ethnic group,
and educational level.
• Most often, however, leaders of psychoeducational groups do not use standardized
instruments for evaluation because they are too expensive, they do not meet specific
group needs, they are not designed around the group’s topic, or the leader lacks the
training to use them.
• If a standardized instrument is desirable and available, and cost is not a concern, a
testing consultant may be used.
Questionnaires (read again
psychometry)
• Questionnaires are used to gather information about perceptions, needs, and
reactions.
• They are very useful when constructed properly and with attention to detail. It is
much more time-consuming to analyze the data, and responses cannot be easily
aggregated.
• When constructing a questionnaire, it is important to pay attention to the following:
• Make sure the items are stated clearly.
1. Avoid using two or more ideas in the same item.
2. Consider the respondents’ abilities to read and understand the items.
3. Make sure the items are relevant to the subject.
4. Present ideas, concepts, and the like in as simple terms as possible
5. Avoid using general negative terms such as all, but, not, and except
Assessing Opinions, Attitudes, and
Affective Responses
• There are several ways to obtain feedback on participants’ opinions,
attitudes, and affective responses. Commonly used methods include
discussion, survey or rating forms, attitude and behavioral ratings.
• The best way to evaluate a psychoeducational group—and to receive
information that can help you in making modifications, revisions, or a
change of direction is to use a form tailored for your specific group.
Selecting Areas
Likert style