Anda di halaman 1dari 6

Acta Psychiatr Scand 2005: 111: 144–149 Copyright Ó Blackwell Munksgaard 2004

All rights reserved ACTA PSYCHIATRICA


DOI: 10.1111/j.1600-0447.2004.00440.x
SCANDINAVICA

Validation of the Bech–Rafaelsen


Melancholia Scale and the Hamilton
Depression Scale in patients with major
depression; is the total score a valid measure
of illness severity?
Licht RW, Qvitzau S, Allerup P, Bech P. Validation of the Bech– R. W. Licht1, S. Qvitzau2,
Rafaelsen Melancholia Scale and the Hamilton Depression Scale in P. Allerup3, P. Bech4
patients with major depression; is the total score a valid measure of 1
Mood Disorders Research Unit, Aarhus University
illness severity? Psychiatric Hospital, Risskov, 2Pfizer Denmark,
Acta Psychiatr Scand 2005: 111: 144–149. Ó Blackwell Munksgaard 2004. Copenhagen, 3The Danish University of Education,
Copenhagen and 4Psychiatric Research Unit,
Objective: Evaluation of antidepressant drug efficacy requires Frederiksborg General Hospital, Hillerød, Denmark
adequate rating scales for measuring the severity of depression.
However, to measure the illness severity by such a total score, the scale
needs to fulfil criteria of unidimensionality. On this background, we
aimed at comparing the unidimensionality of the Bech–Rafaelsen
Melancholia Scale (MES) and the 17-item Hamilton Depression
Rating Scale (HAM-D17).
Method: A total of 1629 patients aged between 18 and 65 years with a
major depressive episode were treated openly with sertraline at a fixed
oral dose of 50 mg daily during 4 weeks. The HAM-D17 and the MES
were applied at baseline and at weeks 2 and 4. Unidimensionality was
tested with Mokken and Rasch analysis.
Results: Unidimensionality of the HAM-D17 could not be confirmed.
However, the 6-item Hamilton Depression Subscale (HAM-D6), was Key words: major depressive disorder; acute therapy;
accepted by the Rasch analysis both at baseline and after 2 and 4 weeks sertraline; rating scales; validity; antidepressant drug
of therapy. For the MES (as well as for the HAM-D6), a Loevinger trials
coefficient of homogeneity above 0.40 (suggesting acceptance) was Rasmus W. Licht, Mood Disorders Research Unit,
found at week 4. Aarhus University Psychiatric Hospital, 8240 Risskov,
Conclusion: The HAM-D6 and the MES did fulfil criteria for Denmark.
unidimensionality while the HAM-D17 did not. Therefore, the E-mail: rl@psykiatri.aaa.dk
extended use of the HAM-D17 in drug trials may be questioned.
Accepted for publication July 16, 2004

implies that scorings on lower prevalence items


Introduction
presuppose scorings on higher prevalence items.
Evaluation of antidepressant drug efficacy requires The statistical analysis for unidimensionality of
adequate rating scales for measuring the severity of rating scales is referred to as item response theory
depression. When such rating scales are applied, analysis (1, 2).
generally the individual item scores are summed up Only a few of the available rating scales for
to give a total score. However, to measure the depression have been properly tested for the
illness severity by such a total score, the scale needs fulfilment of criteria of unidimensionality by item
to fulfil essential criteria of unidimensionality. response theory analysis. Among the rating scales,
Thus, to be additive, the individual items of a the Bech–Rafaelsen Melancholia Scale (MES) (3)
scale must be consistently rank-ordered according has been shown to fulfil such criteria (4–6), whereas
to their relation to the severity of illness. This the Hamilton Depression Rating Scale (HAM-D17)

144
Internal validity of two depression rating scales

(7) and the Montgomery-Åsberg Depression scale divided into two parts. In the first part, the patients
(8) have been shown not to fulfil such criteria (4, 6, were treated openly with sertraline at a fixed oral
9). However, a subscale of the HAM-D17, the dose of 50 mg daily during the first 4 weeks.
HAM-D6 (10), consisting of the items of depressed Patients not responding (achieving a 50% reduc-
mood, psychic anxiety, guilt, work and interests, tion or more in the HAM-D17 total score) after
tiredness and motor retardation, was found to be these 4 weeks of therapy were treated for two
unidimensional (11). What is mentioned here about additional weeks with 100 mg sertraline daily. In
the validity of the HAM-D17 can be generalized to the second part, the patients who had still not
the 21-item version (the HAM-D21), which include responded after the 6 weeks of treatment were now
additional four items of diurnal variation, dereal- randomly and double-blindly allocated to one of
ization (or depersonalization), paranoid symptoms the following three arms over the next 5 weeks:
and obsessional symptoms (12). 100 mg sertraline daily plus 30 mg mianserin,
100 mg sertraline daily plus placebo-mianserin or
200 mg sertraline daily plus placebo-mianserin.
Aims of the study
Despite the consistency in the reports cited above,
Rating scales
a major limitation in these studies is the relatively
small sample sizes and the heterogeneity of the In the trial, the HAM-D17 was used as the primary
samples. On this background, we aimed at com- outcome scale, and the MES was used as the
paring the unidimensionality of the MES, the secondary outcome scale.
HAM-D17 and the HAM-D6 in a large sample of The HAM-D17 consists of 17 items, nine of
patients with major depression, all treated with which are scored on a five-category Likert scale
sertraline at a fixed daily dose of 50 mg in the (from 0 to 4) and the remaining eight on a three-
context of a drug trial. category Likert scale (from 0 to 2), giving a
theoretical total score range from 0 to 52. From
the HAM-D17, the total score on the HAM-D6
Material and methods (defined in the Introduction) was extracted, ranging
theoretically from 0 to 22. The MES consists of 11
Patients
items which are all scored on a five-category Likert
The patients were originally collected for the scale giving a theoretical total score range from 0
purpose of a drug trial (13). To be included, a to 44. These instruments were applied at baseline
patient should be between 18 and 65 years of age, and at week 2, 4, 6, 7, 8 and 11. For each patient,
suffering from a major depressive episode without the clinical assessments were made by the same
psychotic symptoms within a major depressive clinician at the same time of the day, preferably
disorder according to DSM-IV (14) and have a between 8 and 11 am. The assessment of inter-rater
HAM-D17 total score of 18 or more. The exclusion reliability with respect to the total rating scale
criteria were: current substance abuse; risk of scores was achieved through joint ratings of video-
suicide; treatment with any antidepressant or taped patient ratings.
lithium in the last 2 weeks (4 weeks in the case of As the responding patients by design were
fluoxetine); currently receiving and unable to withdrawn from the study after the first 4 weeks
discontinue any psychotropic apart from hypnotics of treatment and as the dose was increased (in non-
and oxazepam; previous adequate treatment with responders) hereafter, only the ratings from the
sertraline without a clinically significant effect; and first 4 weeks were used in the present psychometric
serious somatic illness. According to these criteria, analyses.
a total of 1629 patients were recruited from 30
Danish and nine Icelandic centres between Novem-
Statistical analysis
ber 1996 and January 2000. All patients gave
informed, written consent to participate in the Both the one-parameter Rasch analysis (1) and the
study, which was approved by all relevant Ethical non-parametric Mokken analysis (2) have been
Committees and by the National Board of Health used to test rating scales by item response theory
in Denmark and Iceland. models. The one-parameter Rasch analysis (1, 15)
is based on a logistic function analysis of the
response probabilities for each patient–item com-
Design
bination. By this procedure, item-specific curves,
The design of the drug trial has been described in describing the relationship of each item to the
details elsewhere (13). Briefly, the drug trial was latent dimension (e.g. severity of depression) are

145
Licht et al.

generated. In the one-parameter model, the items eigenvalue could be observed, indicating unidimen-
only differ by their threshold parameter (the sionality (21).
location of the item with respect to the latent
dimension), assuming that all items in the scale
Results
under examination have the same slope. The Rasch
analysis is used to evaluate how well the data Table 1 shows the mean and standard deviation of
obtained with a rating scale fit into the model. The the scores of the three outcome scales during the
one-parameter Rasch Model is equivalent to total 4 weeks of treatment with a daily oral dose of
scores being sufficient statistics, i.e. exhaust all 50 mg sertraline. Of the total of 1629 included
item- and patient-information which, if rejected, patients, 170 (10.4%) patients dropped out during
have practical negative consequences for the prac- this period. The mean score of the three outcome
tical use of simple sum scores across the items (16, scales after 4 weeks of therapy was still at the level
17). The two-parameter alternative may be consid- of clinical depression as defined by the various
ered, especially if prior knowledge to the item scales. The response rate after 4 weeks (number of
discriminations (slopes of the item-specific curves), patients achieving a 50% reduction in the total
being the second parameter, is available. In this score, in percentage of included patients) with the
case weighted item scores are sufficient statistics MES and the HAM-D17 was 28 and 26%,
and are the means for comparing patients. The respectively.
fundamental property of objective comparisons is, Table 2 shows the internal validity of the three
however lost with the introduction of item dis- outcome scales in period 1 using baseline data as
crimination parameters (18). The so-called three- well as data after 2 and 4 weeks of therapy. After
parameter Rasch Model includes a parameter for 4 weeks of therapy, factor analysis still identified
guessing and therefore for the purpose of this five factors in the HAM-D17, whereas in the HAM-
study, it is not considered as a relevant alternative D6 only one factor was identified. In the MES, two
to the one-parameter Rasch Model. factors were identified after 4 weeks. However, the
The non-parametric Mokken analysis (2, 19) is first factor in the MES explained approximately
concerned with a weaker property of the scale items 50% of the variance. The second factor of the five
compared with the requirements set by the Rasch HAM-D17 factors was an agitation-anxiety factor
Model. In fact this type of analysis aims at assessing and the third a somatic factor including the sleep
the level of simple ordinal scaling by the items, i.e. items and gastrointestinal symptoms.
whether the items create a simple ordering.
The Rasch analysis has been performed by the Table 1. Mean values (standard deviation) of total scores on the MES, the HAM-
method described by Allerup (9). In the analysis, a D6 and the HAM-D17
level of rejection of P < 0.01 has been used. The
N MES HAM-D6 HAM-D17
Mokken analysis is based on a Loevinger coeffi-
cient of homogeneity, which assess the degree of Baseline 1629 21.1 (3.6) 12.2 (1.8) 23.3 (3.2)
rank-ordering of items in terms of transmitted 2 weeks 1581 17.8 (4.6) 10.5 (2.6) 19.4 (5.0)
4 weeks 1459 14.0 (6.3) 8.4 (3.5) 15.1 (6.5)
information concerning the dimension under
investigation (20). The level of acceptance is a
Loevinger coefficient of homogeneity of 0.40 or
Table 2. The internal validity of the MES, the HAM-D6 and the HAM-D17
higher, while a coefficient of 0.30 to 0.39 is
considered to be only just acceptable. A coefficient MES HAM-D6 HAM-D17
between 0.20 and 0.29 is doubtful.
Cronbach alpha
Classical psychometric means for testing internal Baseline 0.56 0.20 0.14
validity of a rating scale, i.e. the Cronbach coef- 2 weeks 0.77 0.59 0.67
ficient alpha and factor analysis, have been inclu- 4 weeks 0.88 0.80 0.83
ded for illustrative purposes. In particular the issue Loevinger coefficent
Baseline 0.13 0.06 0.01
of unidimensionality was addressed in these ana- 2 weeks 0.28 0.24 0.13
lyses. A coefficient alpha between 0.60 and 0.80 has 4 weeks 0.49 0.49 0.28
been used to indicate internal consistency (21). The Number of factors in factor analysis
Baseline 4 3 7
factor analysis approach was run as a standard
2 weeks 3 3 6
Principal Component Analysis with orthogonal 4 weeks 2 1 5
varimax rotation of the components. A key point Explained variance by first factor (%)
was to study the sequence of ordered eigenvalues Baseline 23.7 23.8 14.6
2 weeks 32.2 34.3 17.5
(considering only those above 1.0) to see, if a 4 weeks 46.8 50.9 27.8
substantial shift between the first and the second

146
Internal validity of two depression rating scales

For the MES (as well as for the HAM-D6), a validity of a scale, namely its responsiveness, i.e.
Loevinger coefficient of homogeneity above 0.40 the ability of a scale to detect changes over time,
was found at week 4. For the HAM-D17 the has substantial impact on clinical research. It has
Loevinger coefficient of homogeneity was below been found that the HAM-D6 had a higher effect
0.30, even after 4 weeks. Furthermore, by the size than the HAM-D17 when all placebo-con-
Rasch analysis, the HAM-D17 was neither accep- trolled trials with fluoxetine were analysed (0.38 vs.
ted at baseline nor after 2 or 4 weeks of therapy, 0.30) (25), and subsequently, that it would there-
whereas the Rasch analysis showed that the HAM- fore require one-third fewer patients in studies
D6 was accepted both at baseline and after 2 and using the HAM-D6 than in studies using the
4 weeks of therapy. In the Rasch analysis of the HAM-D17 to detect a given difference (26).
MES, two of the MES items (suicidal thoughts and Generally, the responsiveness of the MES and
sleep disturbances) were not accepted by the one- the HAM-D6 is considered to be higher than that
parameter Rasch analysis. of the HAM-D17, as recently reviewed by Bech
(27). Only a few studies have compared the
responsiveness of the MES directly with that of
Discussion
the HAM-D17 (4, 6). In a psychometric reanalysis
This study has definitely shown that both the of the pooled results of four randomized compar-
HAM-D6 and the MES did fulfil criteria for isons of moclobemid with a tricyclic (nortriptyline
unidimensionality while the HAM-D17 did not. or clomipramine) the responsiveness of the MES
Thus, after 4 weeks of treatment with sertraline, an was found to exceed that of the HAM-D17 in terms
acceptable Loevinger coefficient was obtained for of effect sizes (showing superiority of the tricyc-
the HAM-D6 and the MES but not for the HAM- lics), with values of 0.53 and 0.47 for the MES and
D17. Not surprisingly, the coefficients were lower at the HAM-D17, respectively (6).
baseline where the dispersion of scale scores was Unfortunately, the database of the present drug
more limited than after 4 weeks of treatment. trial did not allow a direct comparison of the scales
Therefore, these findings robustly show that sum- in terms of responsiveness, as the patients who did
ming up the scores of individual items to a total respond (showing at least a 50% reduction of the
score is valid for the MES and the HAM-D6 but HAM-D17 total score) at 4 weeks or at 6 weeks
not for the HAM-D17. At least this may be were withdrawn from the study by design, and
generalized to moderately depressed populations beyond these points in time they were not followed
of outpatients receiving SSRIs. However, the with ratings. However, the overall response pat-
superiority of the MES in comparison with the terns may provide some relevant information.
HAM-D17 in terms of unidimensionality has also When the three treatment groups, i.e. patients
been shown in populations of depressed inpatients receiving 100 mg sertraline daily plus 30 mg
receiving other antidepressant than SSRIs (4, 6) mianserin, 100 mg sertraline daily plus placebo-
and the unidimensionality of the MES has been mianserin or 200 mg sertraline daily plus placebo-
confirmed in depressed in-patients receiving no mianserin (from week 6 and on) were pooled, the
treatment (5). accumulated rates of response (defined as at least a
Given the fact that the HAM-D17 has been used 50% reduction of total rating scale score from
in over 95% of all controlled trials with SSRIs (22), baseline to end-point) after 11 weeks were 73 and
these findings are quit remarkable. One way of 71% with the MES and the HAM-D17, respect-
understanding this, is that the HAM-D17 may ively. However, when comparing the three treat-
simply be used by tradition, i.e. due to the fact that ment groups (intention-to-treat population) in
most clinicians (and reviewers) are familiar with terms of response (with the HAM-D17 as the
this instrument. This was the background for primary outcome measure), which was the aim of
choosing the HAM-D17 as the primary outcome the study, continuing the treatment with 100 mg/
measure in the present drug trial. Using a specific day of sertraline resulted in response in 70% of the
rating scale by tradition is also what is seen in the patients still non-responding after the first 6 weeks
field of mania, where the Young Mania Rating of treatment, similar to the response rate (67%)
Scale (23) is the most frequently used rating scale obtained in the patients who had mianserin added,
despite its potential limitations concerning unidi- whereas increasing the sertraline dose to 200 mg/
mensionality (24). Another potential reason for the day resulted in a lower response rate at 56%
wide use of the HAM-D17 despite its lack of (P < 0.05) (13). Similar results were seen in the
unidimensionality is that unidimensionality may be completers. When comparing the three groups on
considered only to have theoretical interest. How- the secondary measure (response defined as at least
ever, it is well known that another aspect of the a 50% reduction of the MES total score from

147
Licht et al.

baseline to end-point), response rates of 65, 67 and As to the relation between unidimensionality
64% appeared. Although the overall responsive- and responsiveness, i.e. the two major aspects of
ness of the MES and the HAM-D17 in terms of internal and external validity, respectively, the first
accumulated response rates seem comparable, the characteristic is a prerequisite of the latter. Lack of
differential responsiveness of the HAM-D17 in unidimensionality may as exemplified above and
the three treatment groups may be a result of the by the present data set lead to an artificially
multidimensional nature of the HAM-D17, as increased (biased) responsiveness (as in the case of
suggested by the present factor analysis and evaluation of sedative tricyclics) or to an artificially
confirmed by the latent structure analysis, as reduced responsiveness (as in the case of evaluation
serotonergic side-effects my be captured on the of SSRIs).
agitation-anxiety dimension or on the somatic A major implication of the present findings is
dimension (including the sleep items and gastroin- that the HAM-D17 total score definitely cannot be
testinal symptoms) leading to an artificially considered as a valid measure of the severity of
increased HAM-D17 total score. This is an example depression, which generally may question the
of a drug-specific (or dose specific) reduced respon- extended use of this scale in drug trials. As the
siveness due to lack of unidimensionality of a scale. unidimensionality of the MES and the HAM-D6
This phenomenon may also explain that the HAM- could be confirmed, these scales are to be preferred
D6 had a higher effect size than the HAM-D17 over the HAM-D17 in trials evaluating drugs for
when all placebo-controlled trials with fluoxetine depression. The most appropriate use of HAM-D17
were analysed as mentioned above. On the other data is to present the profile scores of the various
hand, lack of unidimensionality may also lead to a dimensions of the scale.
falsely increased responsiveness. As recently poin-
ted out and exemplified by Möller (28), the HAM-
Acknowledgements
D17 (and the HAM-D21) includes three items of
sleep and one of weight, which may lead to an The authors thank all the investigators for providing data on
inherent bias in favour of measuring antidepres- individual patients.
sant response to tricyclic antidepressants with a
sedative profile when using the HAM-D17 (or the References
HAM-D21) total score as outcome measure. Based 1. Bond TG, Fox CH. Applying the Rasch Model. London:
on a psychometric reanalysis of the results of two Lawrence Erlbaum, 2001.
randomized trials comparing sertraline and ami- 2. Sijtsma K, Molemaar IW. Introduction to nonparametric
triptyline (a sedative tricyclic) in depressed in- item response theory. London: Sage Publications, 2002.
patients (29) and in depressed outpatients (30), it 3. Bech P, Rafaelsen OJ. The use of rating scales exemplified
by a comparison of the Hamilton and the Bech-Rafaelsen
was shown that the apparent tendency towards a Melancholia Scale. Acta Psychiatr Scand 1980;62(suppl.
superior efficacy of amitriptyline could be 285):128–132.
explained by psychometric bias due to the multi- 4. Maier W, Philipp M, Heuser I, Schlegel S, Buller R, Wetzel
dimensional structure of the the HAM-D21 (28). As H. Improving depression severity assessment – I. Reliabil-
pointed out by Möller (28), such bias would be ity, internal validity and sensitivity to change of three
observer depression scales. J Psychiatr Res 1988;22:3–12.
most likely to occur with sedative antidepressants, 5. Chambon O, Cialdella P, Kiss L, Poncet F, Chevance M,
and with any drug given in doses associated with Milani-Bachman D. Study of the unidimensionality of the
considerable side-effects as illustrated by the pre- Bech-Rafaelsen Melancholia Scale using Rasch analysis in
sent data set. Another example of a potential a French sample of major depressive disorders. Pharma-
psychometrically induced bias is a randomised 5- copsychiatry 1990;23:243–245.
6. Bech P, Stage KB, Nair NP, Larsen JK, Kragh-Sorensen P,
week comparison of citalopram and clomipramine Gjerris A. The Major Depression Rating Scale (MDS).
showing clear superiority of clomipramine in terms Inter-rater reliability and validity across different settings
of complete response defined on the basis of the in randomized moclobemide trials. Danish University
HAM-D17 total score (31). As mentioned by the Antidepressant Group. J Affect Disord 1997;42:39–48.
authors, at the third to fifth week, the three sleep 7. Hamilton M. A rating scale for depression. J Neurol Neu-
rosurg Psychiatry 1960;23:56–62.
items accounted for about 40% of the drug 8. Montgomery SA, Asberg M. A new depression scale de-
difference on the HAM-D17. signed to be sensitive to change. Br J Psychiatry 1979;
As mianserin is a sedative agent, one could 134:382–389.
expect that using the HAM-D17 as outcome 9. Allerup P. Statistical analysis of rating scales. Copenhagen:
measure in the present drug trial would favour Danish Institute of Educational Research, 1986.
10. Bech P, Gram LF, Dein E, Jacobsen O, Vitger J, Bolwig TG.
the treatment arm containing this drug. However, Quantitative rating of depressive states. Acta Psychiatr
no superiority of the combination of sertraline and Scand 1975;51:161–170.
mianserine could be found (13).

148
Internal validity of two depression rating scales

11. Bech P, Allerup P, Gram LF et al. The Hamilton Depres- WPA series evidence and experience in psychiatry. Chi-
sion Scale. Evaluation of objectivity using logistic models. chester: John Wiley & Sons, 1999:89–127.
Acta Psychiatr Scand 1981;63:290–299. 23. Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale
12. Hamilton M. Development of a rating scale for primary for mania: reliability, validity and sensitivity. Br J Psychi-
depressive illness. Br J Soc Clin Psychol 1967;6:278–296. atry 1978;133:429–435.
13. Licht RW, Qvitzau S. Treatment strategies in patients with 24. Licht RW, Jensen J. Internal validation of the Bech Ra-
major depression not responding to first-line sertraline faelsen Mania Scale using latent structure analysis. Acta
treatment. A randomised study of extended duration of Psychiatr Scand 1997;96:367–372.
treatment, dose increase or mianserin augmentation. Psy- 25. Bech P, Cialdella P, Haugh MC et al. Meta-analysis of
chopharmacology (Berl) 2002;161:143–151. randomised controlled trials of fluoxetine v. placebo and
14. American Psychiatric Association. Diagnostic and statis- tricyclic antidepressants in the short-term treatment of
tical manual of mental disorders; (DSM-IV), 4th edn. major depression. Br J Psychiatry 2000;176:421–428.
Washington, DC: American Psychiatric Association, 1994. 26. Faries D, Herrera J, Rayamajhi J, DeBrota D, Demitrack M,
15. Rasch G. Probabilistic models for some intelligence and Potter WZ. The responsiveness of the Hamilton Depres-
attainment tests. Studies in mathematical psychology. I. sion Rating Scale. J Psychiatr Res 2000;34:3–10.
Copenhagen: Pedagogic Institute in Denmark, 1960. 27. Bech P. The Bech-Rafaelsen Melancholia Scale (MES) in
16. Fischer G, Molenaar IW. Rasch models: foundations, clinical trials of therapies in depressive disorders: a 20-year
recent developments and applications. Berlin: Springer, review of its use as outcome measure. Acta Psychiatr Scand
1994. 2002;106:252–264.
17. Allerup P. Rasch measurement, Theory of. In: Husen T, 28. Möller HJ. Methodological aspects in the assessment of
Postlethwaite N. The International Encyclopedia of severity of depression by the Hamilton Depression Scale.
Education, Vol. 8. London: Pergamon, 1994:4902–4913. Eur Arch Psychiatry Clin Neurosci 2001;251(suppl. 2):
18. Allerup P. Statistical analysis of data from the IEA reading II13–II20.
literacy study. In: Rost J, Langeheine R, eds. Applications 29. Möller HJ, Gallinat J, Hegerl U et al. Double-blind,
of latent trait and latent class models in social sciences. multicenter comparative study of sertraline and amitript-
Münster: Waxman, 1997:50–58. yline in hospitalized patients with major depression.
19. Mokken RJ. Theory and procedure of scale analysis. Berlin: Pharmacopsychiatry 1998;31:170–177.
Monton, 1971. 30. Möller HJ, Glaser K, Leverkus F, Gobel C. Double-blind,
20. de Jong A, Molenar IW. An application of Mokken’s model multicenter comparative study of sertraline versus ami-
for stochastic, cumulative scaling in psychiatric research. triptyline in outpatients with major depression. Pharma-
J Psychiatr Res 1987;2:137–149. copsychiatry 2000;33:206–212.
21. Nunnally JC, Bernstein LH. Psychometric theory. New 31. Danish University Antidepressant Group. Citalopram:
York: McGraw-Hill, 1994. clinical effect profile in comparison with clomipramine.
22. Bech P. Pharmacological treatment of depressive disorders: A controlled multicenter study. Psychopharmacology
a review. In: Maj M, Sartorius N, eds. Depressive disorders. (Berl) 1986;90:131–138.

149

Anda mungkin juga menyukai