Anda di halaman 1dari 5

ELSEVIER Patient Education and Counseling 24 (1994) 341-345

Development and evaluation of an inventory for rating client

satisfaction with outcome in HIV counseling: the Albion Center

Kim Begley”, Michael W. Rossb *, Patricia Austina, Kathleen Casey”,

Patricia Collins”, Grant Hennings”, Larry Agriestia, Keith Marshall’
aAlbion Street(AIDS) Center, Division of Medicine, Prince of Wales Hospital, Surry Hills. NSW 2010. Sydney, Australia
bCenter for Health Promotion Research and Development, School of Public Health, University of Texas, PO Bo.x 20186. Houston.
TX 77225, USA
‘AIDS Council of New South Wales, Surry Hills, NSW 2010, Sydney. Australia

Received 25 September 1993; accepted 23 March 1994


We developed an inventory for rating client satisfaction with outcome in HIV counseling, based on interview
responses with clients. The resulting 19-item scale was subject to factor analysis and four factors, accounting for 56%
of the variance, emerged. The factors described dimensions of Perception of progress and improved mood; Recognition
of a specific need for counseling; Behavior change from counseling; and Counseling climate. Factor-scored scales were
significantly associated with time in counseling and for the Specific need for counseling scale, HIV-seropositive
respondents had a significantly higher score. Scale reliabilities (Cronbach’s alpha) were between 0.85 and 0.50. Concur-
rent administration with the Counseling Evaluation Inventory indicated that there were significant correlations be-
tween the two scales.

Keywords: HIV; AIDS; Counseling; Evaluation; Client satisfaction; Measurement


1. Introduction HIV counseling content. A number of significant

publications provide guides to clinical issues and
With the recognition that HIV disease and counseling in HIV/AIDS [l-5], but there is little
AIDS and the issues surrounding HIV infection evaluation of the satisfaction of clients with coun-
are highly psychologically charged, and the seling. This is despite the emphasis on the impor-
stigmatization associated with HIV issues, there tance of counseling before HIV testing as well as
has been considerable focus in the literature on post-test counseling and the very widespread pro-
vision of such counseling in the western world, and
* Corresponding author. the recognition of the extensive psychological con-

0738-3991/94/%07.00 0 1994 Elsevier Science Ireland Ltd. All rights reserved

SSDI 0738-3991(94)00645-3
342 K. Begley et al. /Patient Educ. Couns. 24 (1994) 341-345

comitants of HIV-related concerns [6]. What data and negative HIV status, as well as the more gener-
are available on the impact of interventions sug- al ones such as counseling climate and client satis-
gest that, using standardized distress measures, de- faction must be addressed. However, the issues
pression, anxiety and psychological symptoms in associated with HIV-related counseling satisfac-
HIV-seropositive clients decrease after stress tion did not, based on reading conventional in-
reduction training, and do not increase following dices, relate particularly closely to the issues which
standard counseling protocols and informational our clients, on the basis of exit interviews, sug-
interventions [7], while in HIV-seronegative clients gested were related to their satisfaction with the
there are equal decreases in these measures follow- service. These issues included support from coun-
ing each intervention. A similar cognitive- seling, perception of improvement in coping and
behavioral stress management intervention show- mood, behavior change, and specific aspects of the
ed significantly lower depression after notification clinic and counseling climate. Traditional mea-
of positive HIV serostatus compared with con- sures such as the CEI are based on scales measur-
trols, suggesting that stress reduction has a buffer- ing counseling climate, counselor comfort, and
ing effect [I?]. While such studies document the client satisfaction. We determined to develop a
efficacy of counselling in terms of changes in mood client satisfaction scale to evaluate HIV counseling
states, evaluation of client satisfaction has not which was both reliable and valid, and which con-
been described with regard to HIV counseling. It tained scales which were specific to client-
must be recognized that satisfaction and mood expressed HIV-related satisfaction issues as well as
changes are not necessarily related issues: im- more general questions relating to client satisfac-
provement in mood may occur without satisfac- tion as demonstrated by the CEI.
tion, and vice versa. We emphasise satisfaction
with counseling is an evaluation of the counseling 2. Method
service and climate as much as it is with outcome.
Satisfaction with HIV-related services is frequently Questions were generated by examining the ob-
an essential part of program evaluation, and the jectives of HIV counseling and developing ques-
need for a measure of client satisfaction for such tions which would tap these. All psychologists at
evaluation was the basis for the present research. the Albion (AIDS) Center and AIDS Council of
There are a number of existing indices of client New South Wales provided questions and 19 items
satisfaction which are generic rather than directed were selected from the comments of client’s exit in-
toward specific areas of counseling, such as the terviews for the questionnaire. All were rated on a
Counseling Evaluation Inventory (CEI) [9, lo], 5-point Likert scale (strongly agree, agree, uncer-
and the Counselor Rating Form and the tain, disagree, strongly disagree). Demographic
Counselor Effectiveness Rating Scale. Review of and other data collected included age, gender, HIV
these instruments [ 1l] emphasizes the centrality of status, highest educational level, and months in
evaluation of counselor effectiveness to counseling HIV counseling. In addition, the CEI [9,10] was
practice and research, and the importance of hav- administered to all clients who had been in coun-
ing psychometrically reliable and valid measures. seling for a minimum of four sessions. Responses
However, very little has been done in the area of were placed in a plain envelope and then in a box
effectively evaluating counseling in the HIV/AIDS at the clinic reception to ensure anonymity. The
area, although in a pilot study of 20 clients, Broad- study was approved by the relevant ethics com-
bent [12] found that counseling did help clients to mittee,
cope better with the medical and social aspects of The sample consisted of 62 clients at the Albion
HIV disease. (AIDS) Center, 52 men and 10 women, mean age
In evaluation of the effectiveness of HIV coun- 35.0, median 33.0, S.D. 7.0. Over one-third
seling services, specific issues such as perceived (33.9%) had completed tertiary education, with
mood and behavior changes associated with only 21.0% not having completed high school.
HIV/AIDS counseling of those with both positive Mean number of months of counseling (with
K. Begley et al. /Patient Educ. Cauns. 24 (1994) 341-345 343

weekly appointments) was 12.7, median 7.0, SD. 3. Results

14.7 (range, l-78). All therapists were licensed
psychologists. Factor analysis of the Albion Center Scale
Data analysis was by factor analysis (principal (ACS) produced seven factors with eigenvalues
components analysis followed by varimax rota- > 1. However, several of these factors were
tion), and the factors were factor-scored by monofactors or had high loadings on more than
multiplying the factor loading by the item score one factor, and when the criteria of Walkey and
and adding the products for each factor. Com- McCormick [ 131 were applied, a four-factor solu-
parisons of factors were carried out using Pearson tion appeared to provide both the most inter-
product-moment correlation coefftcients for con- pretable factors and account for the highest
tinuous data, and t-test for comparison of groups amount of variance (56.10/o). The principal com-
on factors. Reliability was calculated using Cron- ponents solution was thus rotated to four factors,
bath’s alpha. which encompassed the domains of Progress and

Table I
Factor structure of ACS questionnaire

Item Loading Percentage


Factor I: Progress and improved mood

I 1 have made progress with the issues that brought me in to counseling 0.87 95
5 The depression 1 experienced before entering counseling has been reduced by the sessions 0.83 94
1I I am satisfied with the progress I have made so far in counseling 0.76 94
6 I feel more satisfied with myself as a result of counseling 0.69 x9
I4 My level of anxiety has not been reduced by counseling -0.59 9
I9 I would recommend this counseling service to others 0.56 100
2 I feel satisfied with the counseling services I receive at this center 0.54 100
(26.2% of variance)

Factor 2: Specific need for counseling

9 I feel the support I receive from friends is as beneficial as the counseling
I receive from this
center -0.82 I6
16 If I did not have counseling available I would not be able to discuss my fears and concerns as
easily with anyone else 0.72 74
IO There are things I can only discuss with my counselor and no one else 0.67 71
8 I believe my medical practitioner has the ability to provide the counseling I receive from my
counselor -0.52 2
(12.0% of variance)

Factor 3: Behavior change from counseling

12 Counseling has helped me maintain safer sex practices 0.77 44
I3 Counseling has helped me seek support from other places and people 0.74 77
I5 I feel more hopeful as a result of counseling 0.52 92
(9.9% of variance)

Factor 4: Counseling climate

4 The staff keep me waiting when 1 attend for an appointment 0.70 7
17 I feel more distressed as a result of counseling 0.68 0
I8 The counselors at this center have insufficient understanding about HIV 0.58 5
7 I feel uncomfortable and distressed in needing to attend for counseling 0.47 5
3 I have found it difficult to get appointments to see a counselor when I need one at this center 0.40 I9
(8.0% of variance)

‘Percentage responding ‘Agree’ or ‘Strongly agree’ to item.

344 K. Begley et al. /Patient Educ. Couns. 24 (1994) 341-345

improved mood, Specific need for counseling, Be- higher score (4.28 vs. 5.34, t = -2.52, df = 31.28,
havior change from counseling, and Counseling P < 0.02).
climate (Table 1). Reliabilities for the four scales of the ACS were,
Intercorrelations of the ACS factors and the respectively 0.85, 0.70, 0.73 and 0.50. Reliabilities
CEI factors indicated that there were modest cor- for the three scales of the CEI were, respectively
relations between the factors (Table 2), with the -1.17, -0.35 and 0.65. Mean scores (*S.D.) and
exception of the correlation between the CEI medians on the scales as scored multiplying by fac-
Client satisfaction factor and the ACS Behavior tor score (possible total in parentheses) were Pro-
change factor which was substantial. ACS factor gress and improved mood, 7.5 f 2.3, Md 1.2
intercorrelations indicated that there was a (24.2); Specific need for counseling, 5.1 f 1.9, Md
moderate intercorrelation between factors 1 and 3, 5.0 (13.7); Behavior change from counseling,
which had nearly 25% of their variance in 4.4 f 1.5 (10.2); and Counseling climate,
common. 12.3 f 1.5, Md 12.5 (14.1). Percentage responding
There were significant correlations between ‘Agree’ or ‘Strongly agree’ to each question is in-
months of counseling and three of the four ACS dicated in the second column of Table 1.
factors, and the CEI Client satisfaction factor. No
correlations between age and the ACS and CEI 4. Discussion
scales were significant, nor those with educational
level. t-Tests comparing the ACS and CEI factors These data are based on a relatively small sam-
with those HIV-seropositive and -seronegative, ple of people attending for counseling, who are
and gender, were all insignificant with the excep- unlikely to be representative of those attending for
tion of ACS factor, Specific need for counseling, counseling or of those with HIV related concerns,
on which the HIV-positive respondents had a and thus of uncertain external validity. Never-
theless, these data do indicate that the ACS has
four distinct and empirically derived subscales
which have acceptable reliabilities. These
Table 2
Correlations between ACS questionnaire, CEI and demo- psychometric properties and its relatively brief for-
graphic variables mat recommend it as a useful instrument for
measuring client satisfaction with HIV-related
Variable ACSl ACB ACS3 ACS4 counseling. Further, there is a degree of associa-
tion with the existing CEI which provides an index
ACSl (Progress and 0.24 -0.10
improved mood) of concurrent validity. On the other hand, the
ACS2 (Need 0.19 0.10 amount of variance in some of the responses was
counseling) low, which may (if this is replicated on broader
ACS3 (Behavior -0.20 samples) make it more difficult to measure pro-
gress. However, measures of progress in outcome
ACS4 (Counseling
climate) will commonly use longitudinal measures of client
mood and attitude rather than changes in evalu-
CEI 1 (Counseling -0.09 -0.09 0.03 0.25’ ation of service impact, so the issue of service eval-
climate) uation is usually based on cross-sectional designs.
CEI2 (Counselor -0.42** -0.17 -0.37** 0.24
The significance of the ACS is that it was
CE13 (Client 0.46** 0.17 0.61** -0.38** developed with the intention of measuring those
satisfaction) aspects of counseling which were most closely link-
Months counseling -0.34** -0.31; -0.31* -0.01 ed to HIV/AIDS issues and counseling practice,
Age 0.20 0.08 0.16 -0.10 rather than using a more general instrument such
Education level -0.18 -0.01 -0.07 0.13
as the CEI. The factor analysis revealed that there
P < 0.05.
?? are four discrete dimensions of client satisfaction
**p < 0.01. in HIV/AIDS counseling: improved mood and
K. Begley et al. /Patient Educ. Couns. 24 (1994) 341-345 345

feeling of making progress; a recognition of a spe- needs to be further validated with other measures
cific need for counseling, as opposed to social sup- of progress, psychological symptomatology and
port; a recognition that behavior change has use of HIV/AIDS services to confirm its efficacy as
resulted from counseling; and an assessment of the a measure of HIV/AIDS counseling satisfaction.
counseling climate. All except the last were
significantly associated with months in counseling. References
The fact that three of the four ACS scales had
1 Miller D, Weber J, Green J. The Management of AIDS
significant correlations with months in counseling Patients. London: Macmillan, 1986.
suggests that it measures some aspect of satisfac- 2 Green J, McCreaner A. Counseling in HIV Infection and
tion, assuming that greater time in counseling is AIDS. Oxford: Blackwell, 1989.
associated with increased satisfaction. However, it 3 Ross MW, Channon-Little LD. Discussing Sexuality: A
Guide for Health Practitioners. Sydney: Maclennan &
may also be the case that time in counseling
Petty, 1991.
reflects severity of the concerns which lead the in- 4 Miller R, Bor R, Duiley JW, AIDS. A Guide to Clinical
dividual to counseling in the first place, and vali- Counseling (second edn). Philadelphia: Science Press,
dation of this scale with other less ambiguous 1991.
measures is needed. Alternatively, greater time in 5 Bor R, Miller R, Goldman E. Theory and Practice of HIV
Counseling. A Systemic Approach. London: Cassell,
counseling may lead to clients to have greater in-
vestment in the outcome and to want it to be effec- 6 Ross MW, Rosser BRS. Psychological issues in AIDS-
tive: as no data on social desirability were related syndromes. Patient Educ Couns 1988; 11: 17-28.
obtained, this cannot be discounted. Further, the 7 Perry S, Fishman B, Jacobsberg L, Young J, Frances A.
lack of significant association with age, educa- Effectiveness of psychoeducational interventions in reduc-
ing emotional distress after Human Immunodeficiency
tional level and gender suggests that the ACS is
Virus antibody testing. Arch Gen Psychiatry 1991; 48:
not unduly biased by demographic characteristics. 143-147.
The significant association between HIV status 8 Antoni MH, Baggett L, Ironson G, LaPerriere A, August
and the ACS scale measuring specific need for S, Klimas N, Schneiderman N, Fletcher MA. Cognitive-
counseling confirms that those with HIV infection behavioral stress management intervention butlers dis-
do have a greater perceived need for counseling. tress responses and immunologic changes following
notification of HIV-I seropositivity. J Consult Clin
Further, the item responses indicate that in the Psycho1 1991; 59: 906-915.
study population there was a strong recognition 9 Linden JD, Stone SC, Shertzer B. Development and eval-
both of the efficacy of HIV/AIDS counseling as ef- uation of an inventory for rating counseling. Pers Guid J
fective and as providing a need that could not be 1965; 44: 267-216.
10 Haase RF, Miller CD. Comparison of factor analytic
met elsewhere. A degree of concurrent validation
studies of the Counselling Evaluation Inventory. J Couns
was provided by the correlations between the CEI Psycho1 1968; 15: 363-367.
and the ACS subscales. II Ponterotto JG, Furlong MJ. Evaluating counselor effec-
In summary, the ACS provides an index of tiveness: a critical review of rating scale instruments. J
client satisfaction with HIV/AIDS counseling is- Couns Psycho1 1985; 32: 597-616.
12 Broadbent J. Assessing the counseling of HIV positive
sues which is generated from items more specific to
clients. Health Visitor 1987; 60: 262-264.
the area than those encountered in more generic 13 Walkey FH, McCormick IA (1985). Multiple replication
counseling satisfaction scales, and has adequate of factor structure: a logical solution for a number of fac-
psychometric properties for this. However, it tors problem. Mult Behav Res 1985; 20: 57-67.