PILOT STUDY

A SMALL SCALE REHEARSAL OR TRIAL RUN OF THE

ACTUAL STUDY

MINIATURE STUDY INVOLVING COLLECTION OF DATA

FROM A LIMITED NUMBER OF SUBJECTS SELECTED FROM THE

POPULATION TARGETED BY THE RESEARCH PROJECT

 PURPOSES
• TO TEST THE EFFECTIVENESS OF TOOLS AND TECHNIQUE

• TO ACQUAINT WITH PROBLEMS FACED

• TIME REQUIRED

• TO CONFIRM THE VARIABLES ARE OBSERVABLE AND MEASURABLE

• TO IDENTIFY WEAKNESS AND DIFFICULTIES

• TO DETECT PROBLEMS AREAS HIDDEN

• TO CONFIRM THE STATISTICAL PROCEDURE

• TO MODIFY THE TOOL AND TECHNIQUE

 Population

Machinery Ambiguous
Problem Situations

DIMENSIONS

Data
Blind Spot and
Management
oversights
problem
 ADVANTAGE OF PILOT STUDY
• REFINE THE TOOLS AND TECHNIQUE

• GIVES SELF CONFIDENCE

• IDENTIFY PROBLEMS AND DIFFICULTIES

• REVISE THE PLAN

• IDEA ABOUT UNNOTICED PITFALLS

VALIDITY
VALIDITY REFERS TO WHETHER A MEASUREMENT

INSTRUMENT ACCURATELY MEASURES WHAT IT IS SUPPOSED

TO MEASURE
 TYPES OF VALIDITY
• CONTENT/ FACE VALIDITY

• CRITERION VALIDITY

• PREDICTIVE VALIDITY

• FORMATIVE VALIDITY

• CONSTRUCT VALIDITY
 CONTENT/ FACE VALIDITY

ASCERTAINS THAT THE MEASURE APPEARS TO BE

ASSESSING THE INTENDED VARIABLE/ CONSTRUCT UNDER
STUDY

- PCI KNOWLEDGE ASSESSMENT TOOL HAVING

QUESTIONS REGARDING VIEWS ON CABG
 CONSTRUCT VALIDITY
ENSURE THAT THE MEASURE IS ACTUALLY MEASURING
WHAT IT IS INTENDED TO MEASURE (I.E. THE CONSTRUCT)
ONLY, AND NOT OTHER VARIABLES

- TOOLS WITH COMPLICATED WORDINGS AND PHRASES

 CRITERION-RELATED VALIDITY

TO WHAT DEGREE THE SUBJECTS PERFORMANCE ON THE

TOOL AND THE SUBJECTS ACTUAL BEHAVIOR ARE RELATED

- PERFORMANCE IMPROVEMENT PROGRAMME AND EXAM

SCORES
 FORMATIVE VALIDITY
ASSESS HOW WELL A MEASURE IS ABLE TO PROVIDE
INFORMATION TO HELP IMPROVE THE PROGRAM UNDER
STUDY
 PREDICTIVE VALIDITY
CORRELATION BETWEEN THE MEASURE OF THE CONCEPT
AND SOME FUTURE MEASURE OF THE SAME CONCEPT. ABILITY
TO PREDICT FUTURE EVENTS
 HOW TO IMPROVE VALIDITY

• CLEAR GOALS AND OBJECTIVES

• MATCHING ASSESSMENT WITH GOALS

• EXPERT ADVICE

• PROOF READING

• EXTENSIVE LITERATURE REFERENCE AND COMPARISON

RELIABILITY
 THE EXTENT TO WHICH INSTRUMENT YIELDS THE

SAME RESULTS ON REPEATED MEASURES

THE DEGREE TO WHICH AN ASSESSMENT TOOL

PRODUCES STABLE AND CONSISTENT RESULTS

 FACTORS INFLUENCING RELIABILITY
• GREATER NUMBER OF ITEMS MORE RELIABLE

• OBJECTIVE SCORING – MORE RELIABLE

• MORE HOMOGENOUS – GREATER RELIABILITY

• LENGTHY ITEM LOWER RELIABILITY

• INTERVAL BETWEEN TEST AND RETEST AFFECT RELIABILITY

• INADEQUATE OR FAULT DIRECTION LOWER RELIABILITY

• PERSONAL FACTOR – PHYSICAL, PSYCHOLOGICAL

 TYPES OF RELIABILITY
• TEST-RETEST RELIABILITY

• PARALLEL OR ALTERNATIVE RELIABILITY

• SPLIT HALF RELIABILITY

• INTER RATER RELIABILITY

 TEST-RETEST RELIABILITY

• MEASURE COEFFICIENT OF STABILITY

• ADMINISTER THE SAME TEST TWICE OVER A PERIOD OF TIME

 SPLIT HALF RELIABILITY

• MEASURES INTERNAL CONSISTENCY

• “SPLIT INTO HALF” ALL ITEMS OF A TEST THAT ARE INTENDED TO

PROBE THE SAME AREA OF KNOWLEDGE, THEN ADMINISTER BOTH

SETS OF THE TEST

 INTER RATER RELIABILITY

• MEASURE OF EQUIVALENCE

• ASSESS THE DEGREE TO WHICH DIFFERENT JUDGES OR RATERS AGREE

IN THEIR ASSESSMENT DECISIONS

 PARALLEL OR ALTERNATIVE RELIABILITY

• ADMINISTER DIFFERENT VERSIONS OF AN ASSESSMENT TOOL

WITH SAME CONSTRUCT