Running head: EXPLORING RELIABILITY 1

Exploring Reliability and Validity

Liberty University

Maggie R. Moran
EXPLORING RELIABILITY AND VALIDITY 2

Exploring Reliability and Validity

Types of Reliability and Validity Used

In Psytech’s (n.d.) Values and Motives Questionairre: the technical manual, there were a

few types of reliability and validity used. Reliability refers to “the consistency of such

measurements when the testing procedure is repeated on a population of individuals or groups”

(Whiston, 2013, p. 40). Validity on the other hand, refers to whether the claims and decisions

made on the basis of assessment results are sound, meaningful, and useful for the intended

purpose of the results (Drummond, Sheperis, & Jones, 2016, p. 110). According to Whiston

(2013), there will be variations in the measured constructs that are considered to be errors and

they can be calculated as reliability coefficients to determine how much error is in a given

instrument. These indications of error in measurements are referred to as standard error of

measurement (SEM). The above definitions will be used throughout this paper.

Reliability: Areas of concern/strength for the Cronbach Alpha Coefficients

When reviewing the Values and Motives Questionnaire, also known as the Values and

Motives Inventory, (referred to as the VMQ for the remainder of this paper), the reliability was

estimated using the internal consistency measures of reliability. According to Whiston (2013),

this type of estimation utilizes one administration and the instrument form is singular. The VMQ

was not dichotomous (i.e. right or wrong), therefore the coefficient alpha or Cronbach’s

Alpha was used to determine reliability with this instrument. The responses in the VMQ ranged

from strongly agree to strongly disagree in a likert scale form.

Sample Size and Nature of the Population

Validity

Validity is typically “subdivided into three distinct types: content validity, criterion
EXPLORING RELIABILITY AND VALIDITY 3

validity, and construct validity” (Drummond et al., 2016, p. 112). Content related validity

examines questions that signify what they claim to signify. Criterion related validity determines

whether the instrument is a good predictor of the criterion. Construct related validity determines

how well the instrument measures a theoretical construct or trait. According to the population

used for the VMQ and tests used to evaluate the VMQ’s validity, it seems that because the

majority of the population was students studying psychology in college, there may be challenges

raised when comparing the assessment’s validity compared with other tests. Because other tests

weren’t necessarily administered to students in the field of psychology, the responses will most

likely have a more general outcome than those participants who are already directly involved in

the field.

VMQ Norming Population

The norm group according to Drummond et al. (2016) are defined as, “a reference group

of people who participated in the standardization of the test to which researchers and

professionals can compare the performance of their subjects or clients” (p. 22). There is an

unbalanced number of participants when comparing gender in the norming population used for

the VMQ, which may have a significant impact on the findings; whereas males “rat(ed) higher

than females” on the Financial status scale (The Technical Manual, n.d.). Had this test been

administered to males and females at different times, rather than simultaneously, there may be

significant findings that outline important information relayed from the VMQ. I do not believe

the results of the group taking the VMQ is broad enough to generalize the other populations who

may be taking the assessment in the future. I also do not believe enough people were sampled in

order to properly dictate results representing an entire population of test takers.

Opinion of Information Found in Report

EXPLORING RELIABILITY AND VALIDITY 4

The VMQ provides the validity information for construct and content validity but not for

criterion validity. The relationship between each person’s test performance and their

performance in the future (predictive validity), would be difficult to know without a follow-up

on each person. Concurrent validity would be possible, if the information was present at the time

of the testing. Then the results of the test could be compared with the information already

available. When looking at the inter-correlations between the different scales, content validity

can be shown. The scales are independent from one to the other and have low correlations. The

content is proven through a review taken in the values arena and showed the test items were

relevant to the given construct. Interpersonal, extrinsic and intrinsic values were utilized to

ensure coverage. I believe the VMQ’s authors did not establish reliability and validity, because

of the makeup of the population of college students studying psychology, versus using a variety

of participants and the minimal sample size of the group used.

EXPLORING RELIABILITY AND VALIDITY 5

References

Drummond, R.J., Sheperis, C.J. and Jones, K.D. (2016) Assessment procedures for counselors

and helping professionals. 8th edn. United States: Pearson.

The technical manual 5 VALUES and MOTIVES QUESTIONNAIRE measures of personality

values & interests (no date) Available at:

http://www.psytech.co.za/images/PsytechSA/VMI/VMIMan.pdf (Accessed: 1 August

2016).

Whiston, S. C. (2013). Principles and Applications of Assessment in Counseling (Fourth

Edition). Belmont, CA: Brooks/Cole