Anda di halaman 1dari 7

The Distress Thermometer and Its Validity: A First

Psychometric Study in Indonesian Women with Breast

Aulia Iskandarsyah1,2,3*, Cora de Klerk1, Dradjat R. Suardi4, Monty P. Soemitro4, Sawitri S. Sadarjoen3,
Jan Passchier1,2
1 Department of Psychiatry, Section Medical Psychology and Psychotherapy, Erasmus MC University Medical Center, Rotterdam, The Netherlands, 2 Department of Clinical
Psychology, VU University, Amsterdam, The Netherlands, 3 Department of Clinical Psychology, Padjadjaran University, Bandung, Indonesia, 4 Department of Surgical
Oncology, Hasan Sadikin Hospital, Bandung, Indonesia

Purpose: This study aims to translate the Distress Thermometer (DT) into Indonesian, test its validity in Indonesian women
with breast cancer and determine norm scores of the Indonesian DT for clinically relevant distress.

Methods: First, the original version of the DT was translated using a forward and backward translation procedure according
to the guidelines. Next, a group of 120 breast cancer patients who were treated at the Outpatient Surgical Oncology Clinic
in Hasan Sadikin Hospital in Indonesia completed a standard socio-demographic form, the DT and the Problem List, the
Hospital Anxiety and Depression Scale (HADS) and the WHO Quality of Life (WHOQOL-BREF).

Results: Receiver operating characteristic (ROC) curve analyses identified an area under the curve = 0.81 when compared to
the HADS cutoff score of 15. A cutoff score of 5 on the DT had the best sensitivity (0.81) and specificity (0.64). Patients who
scored above this cutoff reported more problems in the practical, family, emotional, spiritual/religious and physical domains
(30 out of 36 problems, p-value,0.05) than patients below the cutoff score. Patients at advanced stages of cancer
experienced more emotional and physical problems. Patient’s distress level was negatively correlated with overall quality of
life, general health and all quality of life domains.

Conclusions: The DT was found to be a valid tool for screening distress in Indonesian breast cancer patients. We
recommend using a cutoff score of 5 in this population.

Citation: Iskandarsyah A, de Klerk C, Suardi DR, Soemitro MP, Sadarjoen SS, et al. (2013) The Distress Thermometer and Its Validity: A First Psychometric Study in
Indonesian Women with Breast Cancer. PLoS ONE 8(2): e56353. doi:10.1371/journal.pone.0056353
Editor: Wendy Wing Tak Lam, The University of Hong Kong, Hong Kong
Received October 9, 2012; Accepted January 8, 2013; Published February 13, 2013
Copyright: ß 2013 Iskandarsyah et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: AI received a PhD scholarship from the Directorate General of Higher Education of Indonesia ( The funder had no role in study
design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail:

Introduction including low adherence to treatment recommendation [10], poor

satisfaction with care [11] and poor quality of life [12].
In Indonesia, cancer incidence has an estimated number about Similar to developed countries [13–14], distress among cancer
300,000 cases per year [1]. However, only 10% of these cases are patients often goes unrecognized by health care professionals in
treated in the health care system as the majority of these people do Indonesia. The ratio between the amount of health care
not seek medical help due to several factors, such as strong beliefs professionals and cancer patients is still far from ideal in Indonesia.
in traditional healers, fear and denial, and cultural taboos [2–3]. Data from 506 Government Hospitals in Indonesia showed that in
As one of the ten identifiable main diseases causing death in average there are only 14 General Practitioners and 16 Specialists
Indonesia [4], the diagnosis of cancer and its treatment often per hospital [4]. This condition may lead to several practical
causes considerable psychological distress in patients. It has been issues, including limited consultation time. In addition, a
recognized and reported in previous studies that 20–40% of cancer paternalistic style of doctor-patient communication and patients’
patients experience a significant level of distress [5–6]. Breast unassertiveness are quite common in Indonesia [15]. These factors
cancer is the primary cancer in Indonesia and its incidence and may also cause consultations to be focused primarily on physical
mortality rate is increasing [7]. Previous findings have shown that aspects of cancer.
women with breast cancer experience psychological distress [8], The National Comprehensive Cancer Network (NCCN) states
even years after disease diagnosis and treatment [9]. Patients’ that distress should be recognized, monitored, documented and
distress is associated with a number of negative outcomes, treated promptly at all stages of the disease and in all settings [16].
Considering the high patients load and the unbalanced ratio

PLOS ONE | 1 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

between patients and health care professionals in Indonesia, there cancer patients were explored. After receiving written permission
is an urgent need for a short and effective screening tool to detect from the NCCN, we used the forward and back translation
distress among patients. Ideally, such a tool should be able to assess method to translate the DT, since this method is the most
distress across the physical, psychological, social and spiritual frequently recommended and used method in translation guide-
domains [17]. As current screening tools are long and burdensome lines for cross-cultural studies [31]. One of the authors of this study
for patients to complete, there is a need for a brief, valid and easy (A.I) who is a clinical psychologist translated the DT from English
to complete measure of distress in this population. into the Indonesian language; the back translation into English
In order to meet this demand, the NCCN has developed the was carried out by an English language teacher (J.H) who is a
Distress Thermometer (DT) which is a single item that asks the Native American who speaks the Indonesian language fluently and
patients to rate their distress using a visual analogue scale. It is who has been living in Indonesia for 6 years. Upon completing the
accompanied by the Problem List (PL) that asks patients to identify translation, a linguist (A.C) examined the original English version
any of 36 issues that have been a problem for them in the past and the back translation version of the DT to assess the
week. The DT is very brief, easy to administer and it uses a word significance of any discrepancies. After some discussions with
for psychological problems with non stigmatizing connotations, A.C, we finalized the Indonesian version of the Distress
namely distress [16]. This tool was initially developed by the Thermometer.
NCCN and many studies have reported that the DT is an effective A member of the administration staff of HSH identified eligible
screening tool for detecting distress among various medical patients, explained the study purpose to them and asked for their
conditions, such as prostate carcinoma [18], bone marrow initial consent to participate. One week later, those who wanted to
transplantation [19], lung cancer [20], breast cancer [8] and participate were approached by one of the research assistants
mixed site cancer [21]. The NCCN suggests that a score of 4 or before their next visit to their physician. Ten master’s students in
higher on the DT indicates a clinically significant distress level clinical psychology were trained as research assistants and were
[16]. Some validation studies using the Hospital Anxiety and supervised by S.S (clinical psychologist) and A.I. The research
Depression Scale (HADS) found the same cutoff score of 4 [22– assistant provided further information about the study and
24], whereas other authors found that a cut off score of 5 [25–28] instructions on how to fill in the questionnaires. After informed
best distinguished distressed patients from non-distressed ones. consent had been obtained, participants filled in the DT, the
Most studies found that DT scores above the cutoff are correlated HADS, the World Health Organization Quality of Life (WHO-
with emotional, family and physical problems as measured by the QOL-BREF) and a demographic/background data form. Partic-
Problem List. However, results on spiritual and religious concerns ipants filled out the questionnaires in the waiting room before their
are inconclusive [21–23]. consultations. Ten of the participants were illiterate, but they were
The Distress Thermometer has been successfully translated able to speak and understand the Indonesian language. In these
from English into several languages, such as Arabic [29], Dutch cases, the research assistants read both the informed consent form
[30], Japanese [25], Korean [23], Turkish [24] and Italian, and the questionnaires out loud. After the participants signed the
Spanish and Portuguese [27], but it has not yet been used in informed consent form, the research assistants helped them to fill
Indonesian cancer patients. Therefore, this study aims to translate in the questionnaires.
the DT into Indonesian, test its validity in Indonesian women with
breast cancer by comparing it with a well-established distress Measures
measure, i.e. the HADS, and to determine norm scores of the Socio-demographic and medical status. A standard socio-
Indonesian DT for clinically relevant distress. The other aim was demographic form was used to collect self-report data on age,
to establish the validity of the DT by examined its associations with marital status, education level, employment status, insurance status
the Problem List scores, socio-demographic and clinical charac- and family history of breast cancer. The patients’ medical status,
teristics, and quality of life. such as type and stage of cancer as defined by the TNM stadium
classification system [32], type of treatment and time since
Methods diagnosis were obtained via a medical chart review.
Distress Thermometer (DT). The DT is a 1-item, self-
Participants report measure of psychological distress developed by the NCCN
Consecutive sampling was used to recruit 120 women with [16]. Patients are asked to rate their distress in the past week on an
breast cancer from the outpatient surgical oncology clinic at Hasan 11-point visual analogue scale ranging from 0 (no distress) to 10
Sadikin Hospital (HSH) Bandung in two phases. The first group of (extreme distress). Afterwards, patients are asked to fill in the
50 patients was recruited between April–June 2010; the second Problem List (PL) that accompanies the visual image of the DT to
group of 70 patients was recruited between June–October 2011, check whether or not (yes/no) they experienced any of the
due to logistical reasons. Inclusion criteria were age $18 years, problems listed during the previous 7 days. The PL version used in
first diagnosis of breast cancer and adequate command of the this study consisted of 36 problems that were grouped into five
Indonesian language. Patients who had been treated by psychi- categories, namely practical problems, family problems, emotional
atrists were excluded from the study. problems, spiritual/religious concerns and physical problems. The
PL aims to better define the nature of the problems which possibly
Ethics statement cause the reported distress. To assess its association with the DT
The study was approved by the Indonesian medical ethical scores, the total amount of problems checked was calculated
committee and the Board of Directors of Hasan Sadikin Hospital. (range 0–36).
All samples were obtained with written informed consent reviewed Hospital Anxiety and Depression Scale (HADS). The
by the ethical board. HADS is a 14-item self-report questionnaire that has been
developed to assess psychological distress in people with medical
Procedures illness [33]. It consists of 2 subscales; one subscale consists of 7
This validation study was part of a larger investigation in which items to measure anxiety (HADS-A) and one subscale consists of 7
the correlates of non-adherence behavior in Indonesian breast items to measure depressive symptoms (HADS-D). Respondents

PLOS ONE | 2 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

are asked to indicate which of 4 options (rated 3-0) best describes association between the DT and the total score in the PL was
their feelings during the previous week, which results in a explored by Pearson’s correlation coefficient; associations between
maximum score of 21 on each subscale. The sum scores of the the DT cutoff scores and individual items in the PL were explored
two subscales can be added up to a total score (HADS-T). The by the Chi-square analyses.
HADS has been widely used to validate the DT because of the
similarity in their conceptual background [18,22–27,30,34–35]. Results
The HADS is available in the Indonesian language, but has not
yet been psychometrically validated in Indonesian patients and Demographic and clinical characteristics
cut-off scores for clinically relevant symptoms are not yet available. A total of 120 patients participated in this study. The response
Therefore, in the present study we used the global cutoff score of rate was 91%. Twelve out of 132 women approached declined to
the HADS total ($15) that in studies elsewhere distinguished best participate because they were too ill to fill in the questionnaires. As
between people with and without clinically significant emotional shown in Table 1, the mean age of the women in this sample was
distress [36–37]. Factor analysis of the Indonesian version of the approximately 45.5 years of age (range; 28–66). Most of the
HADS demonstrated a two factor solution in good accordance participants were married (84%). The majority of the participants
with the HADS-A and HADS-D subscales, except for item 3: I feel had middle school or lower education (i.e. 49% had elementary
cheerful and item 4: I feel as if I am slowed down. The solution school, 20% had junior high school and 8% had no education).
accounted for 45% of variance. Both subscales were found to be Seventy-three percent of the participants (73%) were housewives
internally consistent, with values of Cronbach’s coefficient (alpha) or unemployed. The mean number of months since diagnosis was
being 0.77 and 0.74, respectively. 21.5 (SD = 20.3, range = 1–120 months). More than half of the
World Health Organization Quality of Life (WHOQOL- study participants (52%) were in the disease stages III or IV. Fifty-
BREF). The WHOQOL-BREF was developed as an abbrevi- six percent underwent mastectomy, 83% underwent chemother-
ation of the WHOQOL-100 to provide a short form quality of life apy and 23% underwent radiotherapy. Ninety-three percent of the
assessment [38] It was developed by the WHO through a participants had health insurance provided by the government to
multicentre field trial situated within 23 countries. This tool is a poor people (e.g. Jakesmas, ASKES, Gakin and Gakinda) and only
self-report questionnaire which consists of 26 items, each item 7% financed their own medical expenses. Twenty-five percent of
representing one facet of life that is considered to have a the participants had a family history of breast cancer.
contribution to a person’s quality of life. Twenty-four items
measure four broad domains, namely physical health (e.g.
Average score on the DT and the Problem list
mobility, pain and discomfort; 7 items), psychological health (e.g.
The average score of the patients on the DT was 4.7 (SD = 2.6).
body image and appearance, negative feelings, self esteem; 6
The most frequent problems checked in descending order in the
items), social relationships (e.g. personal relationships, social
practical domain were insurance/financial (60%), transportation
support; 3 items) and environment (e.g. financial resources, health
(48%), housing (32%), work/school (24%) and child care (21%).
and social care, physical environment; 8 items). Two other items
The most frequently checked problems in the family problems
measure the overall perception of quality of life and general health.
category were: dealing with children (14%), the ability to have
The WHOQOL-BREF employs a 5-points scale (1 to 5) with a
higher score indicating a higher level of self-perceived quality of children (11%) and dealing with a partner (11%). In the emotional
life. The WHOQOL-BREF is available in a validated Indonesian problems category, the most frequently checked problems were
version [39]. worry (81%), sadness (80%), fears (54%), depression (41%),
nervousness (41%) and loss of interest in usual activities (33%).
Eleven percent of the patients checked the item about spiritual/
Data Analysis
religious concerns. The ten most frequently checked problems in
We used the Statistical Package for Social Science (SPSS 17.0)
the physical problems category were pain (71%), fatigue (68%),
for data analysis. The mean score, the standard deviation, the
nausea (55%), sleep (52%), getting around (51%), tingling in
median score and the frequency distribution of the DT were
hands/feet (51%), eating (41%), appearance (36%), memory/
explored using descriptive statistical analysis. The concurrent and
concentration (36%) and skin dry/itchy (36%).
convergent validity of the DT with the HADS and the
WHOQOL-BREF were examined by Pearson’s correlation
coefficient analyses. Receiver operating characteristic (ROC) Establishment of a DT cutoff score
analysis was used to identify the optimal DT cutoff score for The Pearson’s correlation coefficient between the DT scores
distinguishing whether a patient experiences clinically significant and the HADS total was 0.58 (p,0.01); the correlation coefficients
distress as defined by the HADS. The Area Under the Curve between the DT and the HADS-Anxiety and the HADS-
(AUC) was used to estimate the overall discriminative accuracy of Depression scales were 0.58 (p,0.01) and 0.48 (p,0.01),
the DT cutoff score relative to the established cutoff score of the respectively. Using the HADS cutoff score of 15 as the criterion,
HADS$15. We used a qualitative guideline for interpreting AUC sixty-two women (52%) were identified as experiencing clinically
values by Hosmer and Lemeshow [40], namely AUC = 0.50 as an significant distress. The ROC analysis obtained the AUC of 0.81
indication that the test has no discrimination, AUC#0.70 as an (SE = 0.04; 95%CI = 0.73–0.88; p,0.001) (Figure 1). This AUC
acceptable discrimination, AUC#0.80 as a good discrimination value indicates an excellent discrimination. Table 2 lists the
and AUC#0.90 as an excellent discrimination. ROC curves were Sensitivity, Specificity, Positive predictive values and Negative
used to show the trade-off between the sensitivity (true-positive predictive value on each the DT cut-off point. A cutoff score of 5
rate) and specificity (true-negative rate) for every possible cutoff on the DT optimally identified 81% of the HADS cases (sensitivity)
score of the DT. and 64% of the HADS non cases (specificity) with positive and
To explore the association between the DT cutoff score and the negative predictive values of 70% and 76%, respectively. Of those
Problem List, the demographic variables and the clinical variables, screened positive by the DT, 30% would be false positives and of
Chi-square analyses were conducted for categorical variables and those screened negative by the DT 24% would be false negatives.
t-test analyses were conducted for continuous variables. The

PLOS ONE | 3 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

Table 1. Demographic and clinical characteristics of study scored above the cutoff experienced more spiritual/religious
participants. concerns (p#0.05). Finally, in the physical problems category
(p#0.05), the DT cutoff score was significantly associated with 17
out of 21 problems (i.e. appearance, bathing/dressing, breathing,
Variable n (%) changes in urination, constipation, diarrhea, eating, fatigue, feeling
swollen, fever, indigestion, memory/concentration, mouth sores,
Age (M6SD) 45.568.04 nose dry/congested, pain, sexual and skin dry/itchy), and was not
Marital Status associated with four other problems (i.e. getting around, nausea,
Married 101 (84%) sleep and tingling in hand/feet).
Single 2 (2%)
Divorced 0 (0%) Demographic and clinical characteristics associated with
Widowed 17 (14%)
the DT cutoff score and the Problem List items
Marital status and insurance status were excluded in the Chi-
Education (highest)
square analyses, as some categories did not fulfill the minimum
None 10 (8%) number of expected observations. We found that women with a
Elementary school 59 (49%) score below the DT cutoff score of 5 did not differ significantly
Junior high school 24 (20%) from women at or above the DT cutoff score of 5 on age, time
Senior high school 21 (18%) since diagnosis, education, employment status and family history
of cancer. However, we found a significantly difference in stage of
College or university 6 (5%)
cancer (x2 = 3.90, df = 1, p = 0.048). Women with a score $5 were
more likely to be at an advanced stage of cancer.
Housewife/unemployed 88 (73%) We found several significant associations between the PL-scores
Laborer/irregular job 25 (21%) and the demographic and clinical characteristics. The advanced
Private employee 2 (2%) cancer patients (stage III or IV) had higher PL-total scores
Government officer 5 (4%)
(t = 23.32, p,0.001), more emotional problems (t = 23.55,
p,0.001) and more physical problems (t = 22.62, p,0.01) than
Months since diagnosis (M6SD) 21.5620.3
the stage I or II cancer patients. Age was negatively correlated
Range (months) 1–120 with physical problems (r = 20.21, p,0.05) and the PL-total scores
Current stage of cancer (r = 20.182, p,0.05). PL scores were not associated with marital
1 3 (3%) status, employment status, family history of cancer and time since
2 54 (45%)
3 46 (38%)
The DT and the HADS correlations with the WHOQOL-
4 17 (14%)
BREF scores
Table 3 shows the correlation coefficients of distress and quality
Mastectomy 67 (56%) of life. The DT, the HADS total, the HADS Anxiety and the
Chemotherapy 99 (83%) HADS Depression scores were significantly negatively correlated
Radiotherapy 28 (23%) with overall quality of life, general health and all quality of life
Health insurance
Yes 112 (93%)
No 8 (7%)
Family history of breast cancer In this study, we examined the validity of the DT and its
Yes 30 (25%)
screening efficacy in detecting distress in Indonesian cancer
patients. Our results showed that the Indonesian version of the
No 90 (75%)
DT has concurrent validity with the HADS, which is a well-
doi:10.1371/journal.pone.0056353.t001 established screening tool for distress. A cutoff score of 5 on the
DT yielded optimal sensitivity and specificity. Patients who had a
score above the cutoff score of 5 experienced more problems in the
Associations between the DT cutoff score and the practical, family, emotional, spiritual/religious and physical
Problem List items domains than women with DT scores below this cut off score.
The DT scores were statistically significantly correlated with the Also, they were more likely to be at an advanced stage of cancer.
Problem List total score (r = 0.47, p,0.01). In the practical Finally, distress as measured with the DT was found to be
problems category (p#0.05), the DT cutoff score was significantly negatively correlated with overall quality of life, general health and
associated with four problems (i.e. child care, housing, insurance/ all quality of life domains which establish the convergent validity of
financial and work/school), and was not associated with transpor- the Indonesian version of the DT.
tation. The DT cutoff score was significantly associated with each The ROC analysis comparing the DT scores with the well-
of the problems in the family problems category (p#0.05). Patients established HADS cutoff score of 15 obtained an AUC which
who scored above the cutoff experienced more problems in indicates a good discrimination. Using the DT cutoff score of 5,
dealing with children, dealing with their partner and the ability to eighty-one percent patients were identified correctly as being
have children. In the emotional problems category (p#0.05), the distressed and 64% identified correctly as not being distressed
DT cutoff score was significantly associated with five problems (i.e. which is comparable to the result of the meta-analysis study by
depression, nervousness, sadness, worry and loss of interest in Mitchell [41]. This evidence shows that the DT has a screening
usual activities), and was not associated with fears. Patients who efficacy for distress in Indonesian breast cancer patients. The

PLOS ONE | 4 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

Figure 1. Receiving operation characteristic (ROC) curve of Distress Thermometer scores versus Hospital Anxiety and Depression
Scale cutoff scores.

current Distress Management Guidelines from the NCCN yielded an optimal combination of sensitivity and specificity, to
recommend that a DT score of 4 or higher indicates that a avoid a large number of false positive cases being diagnosed.
patient has a clinical significant level of distress and should be Patients who may not require further intervention may feel
referred to a psychosocial care team [16]. However, we obtained a burdened by further screening procedures. Moreover, false
sensitivity of 90% and a specificity of 50% at a cutoff score of 4, positive screening leads to higher health care costs and an
resulting in a large proportion of patients incorrectly being increased need for health professionals. The DT cutoff score of 5
identified as experiencing clinically significant distress. Consider- found in this study corresponds with the cutoff score found by
ing the lack of health care professionals in Indonesia, we believe other validation studies using the HADS [25–27,34–35]
that it is more appropriate to use the cutoff score of 5 which

Table 2. Sensitivity, Specificity, Positive and Negative predictive values on each the Distress Thermometer cut-off point.

Cut-off point Sensitivity Specificity Positive predictive value Negative predictive value

0/1 0.98 0.11 0.51 0.88

1/2 0.95 0.21 0.56 0.80
2/3 0.92 0.40 0.62 0.82
3/4 0.90 0.50 0.66 0.83
4/5 0.81 0.64 0.70 0.76
5/6 0.52 0.91 0.86 0.64
6/7 0.42 0.95 0.90 0.60
7/8 0.24 0.98 0.94 0.54
8/9 0.15 1.00 1.00 0.52
9/10 0.10 1.00 1.00 0.51


PLOS ONE | 5 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

Table 3. Association between distress and quality of life. to be able to extrapolate these results of the present study to other
patient groups. Secondly, all measures used were self-rating
questionnaires. Nevertheless, we included ten illiterate participants
DT HAD-A HAD-D HADS-T and they were helped to fill out the questionnaires. This may have
** ** **
led to some bias. Thirdly, the HADS Indonesian version has only
Overall quality of life 20.36 20.40 20.32 20.39**
been linguistically validated by the MAPI Institute which may
** ** **
General health 20.43 20.44 20.31 20.41** have lead to some cultural bias. However, the basic psychometric
** ** **
Physical health domain 20.45 20.45 20.53 20.54** examination results indicated that the HADS Indonesian version
Psychological domain 20.55 **
20.55 **
20.53 **
20.59** can be considered as a good instrument in terms of factor structure
Social relationships domain 20.22 *
20.29 **
20.38 **
and internal consistency. Since the Geriatric Depression Scale,
** ** **
which is an instrument that is similar to the HADS has been
Environment domain 20.31 20.30 20.36 20.36**
shown to have the same optimal cut off point in both Western and
DT: Distress Thermometer; HAD-A: HAD Anxiety subscale score; HAD-D: HAD Asian countries [48], we used the general HADS cutoff score
Depression subscale score; HADS-T: Hospital Anxiety and Depression Scale total suggested for Western countries in our study. Finally, this study
score. examined the validity of the DT, but further research is required
*Correlation is significant at the 0.05 level (2-tailed).
involving oncologists and nurses to confirm the feasibility of its use
**Correlation is significant at the 0.01 level (2-tailed).
doi:10.1371/journal.pone.0056353.t003 in daily care practice.
Bearing these limitations in mind, our findings suggest that the
Patients who had significant distress were more likely to report Indonesian version of DT could be used as a screening tool in daily
more problems in the practical, family, emotional, spiritual/ cancer care in Indonesia. As the DT is brief and easy to
religious and physical domains. Interestingly, patients who had administer, it might be an acceptable tool for oncologists in
clinically significant distress were more likely to experience Indonesia. The NCCN suggests that early detection and treatment
spiritual/religious concerns which is similar to the results of a of distress leads to better adherence to treatment, better
study conducted in Korea [23]. In contrast, most studies communication and prevents severe anxiety and depression [6].
conducted in Western countries found that clinically significant According to our findings, cancer patients who experience distress
distress was not associated with spiritual/religious concerns above the DT cutoff score of 5 should be referred to a psychologist
[20,22,24], or only weakly related [34]. The significant correlation or another health professional to manage their distress and get
between high distress and spiritual/religious concerns is possibly appropriate treatment of their main distress sources as indicated in
due to the fact that Indonesian people are religious and have a the PL. The use of the DT in daily cancer care in Indonesia may
strong belief in God. Many people rely on God to heal their help oncologists to prevent potential severe psychological problems
disease. We hypothesize that people who do not feel any change in in cancer patients and provide additional interventions to patients
their illness will be more convinced their cancer as the will of God who need it. Our results suggest that patients in an advanced stage
and they cannot change their destiny which in turn might trigger of cancer should be given priority for psychological intervention.
higher levels of distress. Such interventions are often part of medical psychology. Given
Results of studies on associations between distress and socio- that the field of medical psychology is new in Indonesia, we
demographic and clinical characteristics in cancer patients are recommend its development by psychological faculties with
inconsistent [42]. In the present study, high distress was only found academic hospitals in order to be able to provide adequate
to be associated with stage of cancer, but not with other socio- psychological resources to patients and doctors.
demographic or clinical characteristics. This finding is concor- Our study did not only confirm the validity of the DT in
dance with previous studies that also unable to find significant Indonesian population, but also showed specific associations with
associations between the DT and socio-demographic and clinical several problems in the problem list. We found that women with
characteristics [18–19,23–24,27,43]. Our finding that distress is breast cancer in Indonesia, most of whom are very religious, have
associated with lower overall quality of life, general health and all different sources of distress than breast cancer patients in Western
quality of life domains is in line with the studies by Skarstein et al. countries. In this respect, our study sheds light on cultural factors
[12] and Ozalp et al. [24], and further proves the validity of the explaining cancer-related distress, thereby generating knowledge
Indonesian version of the DT. that is not only useful for physicians working in Asian countries,
The Problem List scores were associated with several demo- but also for physicians working with Asian populations in Western
graphic and clinical characteristics in the expected direction, countries.
suggesting that the Indonesian version of the Problem List is also a
valid tool. Advanced cancer patients experienced more emotional Acknowledgments
problems and physical problems than patients at an early stage of
The authors would like to thank the NCCN for their permission to
cancer, and younger patients experienced more physical problems. translate and use the Distress Thermometer. They would also like to thank
These results are in line with previous studies results [44–45]. all patients and the staff at the surgical oncology clinic Hasan Sadikin
Several limitations of this study should be noted. Firstly, we used Hospital, Indonesia for their generous participation in this study. Mr.
only breast cancer patients as our sample. Furthermore, we James Holmboe and Dr. Anne de la Croix are greatly acknowledged for
conducted this study at HSH which is a referral hospital that their contribution in the translation process.
provides health services to the poor people. Therefore, the
majority of the study participants had middle to low socio- Author Contributions
economic and educational level. However, demographic and
Obtained permission from the medical ethics: DRS MPS SSS. Conceived
clinical characteristics of the patients (e.g. mean age, education and designed the experiments: AI CdK DRS MPS SSS JP . Performed the
level, marital status and stage of cancer) were similar to previous experiments: AI. Analyzed the data: AI CdK. Contributed reagents/
studies in Indonesian breast cancer patients [46–47]. Multi-center materials/analysis tools: AI CdK DRS MPS SSS JP . Wrote the paper: AI
studies with a larger sample of various patient groups are needed CdK JP.

PLOS ONE | 6 February 2013 | Volume 8 | Issue 2 | e56353

The Distress Thermometer and Its Validity

1. Al-Shahri M (2002) The future of palliative care in the Islamic world. West J Med 26. Gessler S, Low J, Daniells E, Williams R, Brough V, et al. (2008) Screening for
176: 60–61. distress in cancer patients: is the distress thermometer a valid measure in the UK
2. Lickiss JN (1993) Indonesia: status of cancer pain and palliative care. J Pain and does it measure change over time? A prospective validation study.
Symptom Manage 8: 423–424. Psychooncology 17: 538–547.
3. Moore MA, Manan AA, Chow KY, Cornain SF, Devi CR, et al. (2010) Cancer 27. Gil F, Grassi L, Travado L, Tomamichel M, Gonzalez JR, et al. (2005) Use of
epidemiology and control in peninsular and island South-East Asia - past, distress and depression thermometers to measure psychosocial morbidity among
present and future. Asian Pac J Cancer Prev 11 Suppl 2: 81–98. southern European cancer patients. Support Care Cancer 13: 600–606.
4. Departemen Kesehatan RI (2011) Profil Kesehatan Indonesia 2010. Pusat data 28. Patrick-Miller LJ, Broccoli TL, Much JK, Levine E (2004) Validation of the
dan informasi Depkes RI, Jakarta. Distress Thermometer: A single item screen to detect clinically significant
5. Zabora J, BrintzenhofeSzoc K, Curbow B, Hooker C, Piantadosi S (2001) The psychological distress in ambulatory oncology patients. J Clin Oncol (Meeting
prevalence of psychological distress by cancer site. Psychooncology 10: 19–28. Abstracts) 22, 14S (July 15 Supplement): 6024.
6. National Comprehensive Cancer Network (2003) Distress management. Clinical 29. Khatib J, Salhi R, Awad G (2004) Distress in cancer inpatients in King Hussein
practice guidelines in Oncology. J Natl Compr Canc Netw Jul;1: 344–374. Cancer Center (KHCC): a study using the Arabic-modified version of the
7. Ferlay J, Shin HR, Bray F, Forman D, Mathers C, et al. (2010) GLOBOCAN distress thermometer. Psycho-Oncology 12(S1):S42.
2008 v1.2, Cancer Incidence and Mortality Worldwide: IARC CancerBase
30. van Dooren S, Duivenvoorden HJ, Passchier J, Bannink M, Tan MB, et al.
No. 10 [Internet]. Lyon, France: International Agency for Research on Cancer.
(2009) The Distress Thermometer assessed in women at risk of developing
pp. Available: Accessed on 2012 Feb 12.
hereditary breast cancer. Psychooncology 18: 1080–1087.
8. Hegel MT, Moore CP, Collins ED, Kearing S, Gillock KL, et al. (2006) Distress,
psychiatric syndromes, and impairment of function in women with newly 31. Peters M, Passchier J (2006) Translating instruments for cross-cultural studies in
diagnosed breast cancer. Cancer 107: 2924–2931. headache research. Headache 46: 82–91.
9. Montazeri A (2008) Health-related quality of life in breast cancer patients: a 32. Albar ZA, Tjindarbumi D, Ramli M, Lukitto P, Reksoprawiro S, et al. (2004)
bibliographic review of the literature from 1974 to 2007. J Exp Clin Cancer Res Protokol PERABOI 2003. Perhimpunan Ahli Bedah Onkologi Indonesia.
27: 32. 33. Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta
10. Kennard BD, Smith SM, Olvera R, Bawdon RE, O hAilin A, et al. (2004) Psychiatr Scand 67: 361–370.
Nonadherence in adolescent oncology patients: Preliminary data on psycholog- 34. Tuinman MA, Gazendam-Donofrio SM, Hoekstra-Weebers JE (2008) Screen-
ical risk factors and relationships to outcome. J Clin Psychol Med Settings 11: ing and referral for psychosocial distress in oncologic practice: use of the Distress
30–39. Thermometer. Cancer 113: 870–878.
11. Von Essen L, Larsson G, Oberg K, Sjoden PO (2002) ‘Satisfaction with care’: 35. Patrick-Miller LJ, Broccoli TL, Much JK, Levine E (2004) Validation of the
associations with health-related quality of life and psychosocial function among Distress Thermometer: A single item screen to detect clinically significant
Swedish patients with endocrine gastrointestinal tumours. Eur J Cancer Care psychological distress in ambulatory oncology patients. J Clin Oncol (Meeting
(Engl) 11: 91–99. Abstracts) 22, 14S (July 15 Supplement): 6024.
12. Skarstein J, Aass N, Fossa SD, Skovlund E, Dahl AA (2000) Anxiety and 36. Ibbotson T, Maguire P, Selby P, Priestman T, Wallace L (1994) Screening for
depression in cancer patients: relation between the Hospital Anxiety and anxiety and depression in cancer patients: the effects of disease and treatment.
Depression Scale and the European Organization for Research and Treatment Eur J Cancer 30A: 37–40.
of Cancer Core Quality of Life Questionnaire. J Psychosom Res 49: 27–34. 37. Herrmann C (1997) International experiences with the Hospital Anxiety and
13. Passik SD, Dugan W, McDonald MV, Rosenfeld B, Theobald DE, et al. (1998) Depression Scale–a review of validation data and clinical results. J Psychosom
Oncologists’ recognition of depression in their patients with cancer. J Clin Oncol Res 42: 17–41.
16: 1594–1600. 38. Skevington SM, Lotfy M, O’Connell KA, Group W (2004) The World Health
14. Sollner W, DeVries A, Steixner E, Lukas P, Sprinzl G, et al. (2001) How Organization’s WHOQOL-BREF quality of life assessment: psychometric
successful are oncologists in identifying patient distress, perceived social support, properties and results of the international field trial. A report from the
and need for psychosocial counselling? Br J Cancer 84: 179–185. WHOQOL group. Qual Life Res 13: 299–310.
15. Claramita M, Utarini A, Soebono H, Van Dalen J, Van der Vleuten C (2011) 39. Salim OC, Sudharma NI, Rina K, Kusumaratna RK, Hidayat A (2007) Validity
Doctor-patient communication in a Southeast Asian setting: the conflict between and reliability of World Health Organization Quality of Life-BREF to assess the
ideal and reality. Adv Health Sci Educ Theory Pract 16: 69–80. quality of life in the elderly. Univ Med 26: 27–38.
16. National Comprehensive Cancer Network (2010) Clinical practice guidelines in
40. Hosmer D, Lemeshow S (2000) Applied Logistic Regression. Wiley-Interscience
oncology-v.1.2010. Distress management: version1.
: New York.
17. National Institute for Clinical Excellence (2004) Guidance on Cancer Services:
41. Mitchell AJ (2007) Pooled results from 38 analyses of the accuracy of distress
Improving Supportive and Palliative Care for adults with Cancer: The Manual.
thermometer and other ultra-short methods of detecting cancer-related mood
18. Roth AJ, Kornblith AB, Batel-Copel L, Peabody E, Scher HI, et al. (1998) Rapid disorders. J Clin Oncol 25: 4670–4681.
screening for psychologic distress in men with prostate carcinoma: a pilot study. 42. van’t Spijker A, Trijsburg RW, Duivenvoorden HJ (1997) Psychological sequelae
Cancer 82: 1904–1908. of cancer diagnosis: a meta-analytical review of 58 studies after 1980. Psychosom
19. Ransom S, Jacobsen PB, Booth-Jones M (2006) Validation of the Distress Med 59: 280–293.
Thermometer with bone marrow transplant patients. Psychooncology 15: 604– 43. Dabrowski M, Boucher K, Ward JH, Lovell MM, Sandre A, et al. (2007)
612. Clinical experience with the NCCN distress thermometer in breast cancer
20. Graves KD, Arnold SM, Love CL, Kirsh KL, Moore PG, et al. (2007) Distress patients. J Natl Compr Canc Netw 5: 104–111.
screening in a multidisciplinary lung cancer clinic: prevalence and predictors of 44. Chen ML, Tseng HC (2006) Symptom clusters in cancer patients. Support Care
clinically significant distress. Lung Cancer 55: 215–224. Cancer 14: 825–830.
21. Hoffman BM, Zevon MA, D’Arrigo MC, Cecchini TB (2004) Screening for 45. Walsh D, Donnelly S, Rybicki L (2000) The symptoms of advanced cancer:
distress in cancer patients: the NCCN rapid-screening measure. Psychooncology relationship to age, gender, and performance status in 1,000 patients. Support
13: 792–799. Care Cancer 8: 175–179.
22. Jacobsen PB, Donovan KA, Trask PC, Fleishman SB, Zabora J, et al. (2005) 46. Ng CH, Pathy NB, Taib NA, Teh YC, Mun KS, et al. (2011) Comparison of
Screening for psychologic distress in ambulatory cancer patients. Cancer 103: breast cancer in Indonesia and Malaysia–a clinico-pathological study between
1494–1502. Dharmais Cancer Centre Jakarta and University Malaya Medical Centre, Kuala
23. Shim EJ, Shin YW, Jeon HJ, Hahm BJ (2008) Distress and its correlates in Lumpur. Asian Pac J Cancer Prev 12: 2943–2946.
Korean cancer patients: pilot use of the distress thermometer and the problem 47. Wakai K, Dillon DS, Ohno Y, Prihartono J, Budiningsih S, et al. (2000) Fat
list. Psychooncology 17: 548–555. intake and breast cancer risk in an area where fat intake is low: a case-control
24. Ozalp E, Cankurtaran ES, Soygur H, Geyik PO, Jacobsen PB (2007) Screening study in Indonesia. Int J Epidemiol. 2000/04/06 ed. pp. 20–28.
for psychological distress in Turkish cancer patients. Psychooncology 16: 304– 48. Wada T, Ishine M, Sakagami T, Kita T, Okumiya K, et al. (2005) Depression,
311. activities of daily living, and quality of life of community-dwelling elderly in three
25. Akizuki N, Akechi T, Nakanishi T, Yoshikawa E, Okamura M, et al. (2003) Asian countries: Indonesia, Vietnam, and Japan. Arch Gerontol Geriatr 41:
Development of a brief screening interview for adjustment disorders and major 271–280.
depression in patients with cancer. Cancer 97: 2605–2613.

PLOS ONE | 7 February 2013 | Volume 8 | Issue 2 | e56353