Accesare Resursa PDF

Multimodal Treatments for Childhood Attention-decit/
Hyperactivity Disorder: Interpreting Outcomes in the Context

of Study Designs
Betsy Hoza Nina M. Kaiser Elizabeth Hurt
Published online: 21 August 2007
Springer Science+Business Media, LLC 2007
Abstract The goal of this article was to outline issues
critical to evaluating the literature on incremental benet of
multiple effective treatments used together, vs. a single
effective treatment, for childhood ADHD. These issues
include: (1) sequencing and dosage of treatments being
combined and compared; (2) difculty drawing valid
conclusions about individual components of treatment
when treatment packages are employed; (3) differing
results emerging from measurement tools that purportedly
measure the same domain; and (4) the resultant difculty in
reaching a summary conclusion when multiple outcome
measures yielding conicting results are used. The impli-
cations of these issues for the design and conduct of future
studies are discussed, and recommendations are made for
future research.
Keywords Attention-decit/hyperactivity disorder
ADHD Multimodal treatment Combined treatment
Behavior therapy
In recent years, a number of multimodal treatment studies
have emerged in the empirical treatment literature for
childhood attention-decit/hyperactivity disorder (ADHD).
These studies are important both for the level of design
sophistication they contribute to the literature on ADHD
and also because their results have inuenced important
documents, such as pediatric treatment guidelines and
insurance coverage policies. The importance of these
studies is indisputable, as is their impact on the way in
which pediatricians, mental health professionals, schools,
and parents approach treatment of childhood ADHD. We
propose, however, that greater knowledge can be gained
from these important studies by evaluating their outcomes
in the context of the studies designs. This article repre-
sents an attempt to focus greater attention on this issue.
Our view is that no single study design can address all
issues pertinent to treatment of a specic disorder. Indeed,
there is no penultimate design, but rather, different designs
address specic issues in a better or worse fashion. Hence,
results of studies may be most informative to the eld when
interpreted in the context of their designs and greatest
weight given to the questions best addressed by each
design. Our purposes in writing this article are rst, to
illustrate the differential conclusions that may be reached
as a function of whether design factors are emphasized in
interpreting outcomes and second, to outline key issues for
future research.
In this article, we review multimodal treatment studies
published roughly in the past decade (1995 to the present).
We choose this time frame in order to restrict our studies to
those using diagnostic criteria from the DSM-IV (Diag-
nostic and Statistical Manual of Mental Disorders, Fourth
Edition; American Psychiatric Association 1994). This
strategy allows us to eliminate confounds that may arise,
when considering varying denitions of ADHD across
Present Address:
B. Hoza (&) E. Hurt
Department of Psychology, University of Vermont,
2 Colchester Ave., Burlington, VT 05405-0134, USA
e-mail: bhoza@uvm.edu
N. M. Kaiser
Department of Psychiatry, University of California, San
Francisco, 401 Parnassus Ave., CPT, San Francisco,
CA 94143-0984, USA
e-mail: nina.kaiser@ucsf.edu
E. Hurt
Purdue University, West Lafayette, IN, USA
1 3
Clin Child Fam Psychol Rev (2007) 10:318334
DOI 10.1007/s10567-007-0025-5
multiple versions of the DSM. Within this time frame, we
discuss issues pertaining to the way in which the studies
designs may have affected their outcomes and conclusions.
For purposes of this review, we limit our discussion to
multimodal treatment studies examining incremental ben-
et, a subset of which have the added strength of also being
multisite studies (Abikoff et al. 2004a, b; Hechtman et al.
2004a, b; Hoza et al. 2005; MTA Cooperative Group 1999;
Pelham et al. 2000; Wells et al. 2000, 2006). As in our
prior work, we dene incremental benet studies as those
wherein two distinct forms of active intervention [were]
included (e.g., medication, behavior modication) and the
use of one active treatment [was] compared to the use of
two or more active treatments (Hoza et al. in press, p. 6).
Of course the need for incremental benet studies
implies that single modality treatments for ADHD are
insufcient to yield desired outcomes, a premise that
applies better to some domains than to others. For example,
whereas medication often controls core symptoms of
ADHD reasonably well, when used alone, non-symptom
domains, such as parentchild relationships, peer relation-
ships, aggression, and internalizing symptoms may fare
better with a multimodal approach (MTA Cooperative
Group 1999). Furthermore, neither medication nor behav-
ior therapy are effective for all children (Hoza et al. 1999),
and, even in children who may be responders to these
treatments, non-treatment factors play a role in whether or
not either treatment is utilized (Hoza et al. 2006). For
instance, parental concerns regarding safety and accept-
ability of medication use may interfere with initial
implementation of treatment or sustained treatment
adherence (Hoza et al. 2006). Importantly, after decades of
debate, recent evidence documents that ongoing use of
stimulant medications may reduce childrens growth for up
to three years (Swanson et al. 2007) and recent concerns
have been raised based on animal research regarding the
long-term effects of stimulants on the developing brain
(Volkow and Insel 2003). Similarly, behavioral treatments
have drawbacks with criticisms centering on difculty of
implementation (Mrug et al. 2001) and lack of evidence for
generalization outside the treatment setting (Richters et al.
1995). These limitations of pharmacological and behav-
ioral treatments, however, can be viewed as indicating an
even greater need for consideration of incremental benet,
as combined approaches often permit lower doses of
medications and less complex behavioral strategies to be
used (Pelham and Hinshaw 1992). For these reasons, we
choose to focus herein on incremental benet.
Aside from this key requirement of considering incre-
mental benet, other requirements for inclusion in this
review, also consistent with Hoza et al. (in press), were:
First, studies were selected if investigators applied DSM-
IV diagnostic criteria for ADHD, using either DSM-IV-
based clinical assessments or diagnostic cutoffs on parent
and/or teacher-completed dimensional (i.e., rating) mea-
sures of ADHD symptoms. Importantly, prior work
examining comparability of samples selected according to
these two approaches provide evidence that they are rea-
sonably comparable (Owens and Hoza 2003), although
impairment and age of onset criteria are not necessarily
captured to the same degree by the latter approach. Second,
researchers included a multimodal intervention, at least one
component of which could be classied as a psychosocial
treatment based on well-established behavioral principles.
Third, researchers employed some type of control condi-
tion (this control could be pill or psychological placebo, an
alternative treatment, a wait-list or no-treatment control, or
treatment reversal), although we focus on evaluation of
incremental benet rather than comparisons to these con-
trols in the current article. Fourth, researchers examined
outcomes in at least one of six outcome domains of primary
interest to us: ADHD symptoms (assessed via ADHD
symptom rating scales completed by adult informants or
behavior counts evaluating frequencies of behaviors that
are DSM-IV ADHD symptoms); behavior (oppositional
and aggressive behavior, general behavioral or conduct
problems); social/peer functioning (parent- and teacher-
rated social function/dysfunction, peer nominations/ratings
of friendship/liking/disliking, as well as adult evaluations
of social skills), academic functioning, parenting practices/
parentchild relationship measures, and child internalizing
symptoms. Importantly, we include child self-report mea-
sures only in those domains in which children generally are
accepted as valid reporters (e.g., internalizing problems but
not ADHD symptoms). Fifth, the participants were children
of preschool- or school-age (with the goal of ensuring
comparability of interventions across studies). Sixth, data
necessary to compute effect sizes for post-treatment dif-
ferences between treatment conditions were provided in the
original article and/or contact information for the authors
was available and the authors provided this information
upon request.
In the context of these parameters, we identied 13
different studies (listed in Table 1) described in 18 articles,
as more than one article was published on some studies.
Specically, there were four articles based on the study by
Abikoff and colleagues (hereafter collectively referred to
as the Abikoff Group; Abikoff et al. 2004a, b; Hechtman
et al. 2004a, b). There were three articles from the Multi-
modal Treatment Study of Children with ADHD (hereafter
collectively referred to as the MTA Group; MTA Coop-
erative Group 1999; Wells et al. 2000, 2006). Ten of these
studies employed between-subjects designs; two employed
within-subjects designs, and one employed a single-subject
design (see Table 1 for a listing of all studies according to
these design classications). In many instances, data were
Clin Child Fam Psychol Rev (2007) 10:318334 319
1 3
T
a
b
l
e
1
S
t
u
d
i
e
s
E
m
p
l
o
y
i
n
g
M
u
l
t
i
m
o
d
a
l
I
n
t
e
r
v
e
n
t
i
o
n
s
A
1
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
b
e
t
w
e
e
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
s
y
m
p
t
o
m
a
n
d
b
e
h
a
v
i
o
r
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
S
y
m
p
t
o
m
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
B
e
h
a
v
i
o
r
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
A
b
i
k
o
f
f
G
r
o
u
p
1
0
3
s
a
,
b
,
k
2
4
m
o
n
t
h
s
1
,
2
/
4
,
6
,
8
,
1
0
,
1
1
1
c
,
d
,
k
,
l
,
m
0
.
0
3
(
0
.
2
8
,
0
.
2
1
)
c
,
d
,
k
,
m
0
.
0
3
(
0
.
3
9
,
0
.
4
5
)
1
+
2
1
c
,
d
,
k
,
l
,
m
0
.
2
9
(
0
.
5
3
,
0
.
0
0
)
c
,
d
,
k
,
m
0
.
0
3
(
0
.
4
5
,
1
.
2
6
)
B
a
r
k
l
e
y
e
t
a
l
.
(
2
0
0
0
)
1
5
8
s
d
1
2
m
o
n
t
h
s
2
,
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
d
,
k
,
l
0
.
1
8
(
0
.
1
2
,
0
.
4
1
)
d
,
i
,
k
,
l
,
m
0
.
2
3
(
0
.
1
2
,
0
.
5
7
)
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
d
,
k
,
l
0
.
0
1
(
0
.
3
2
,
0
.
5
9
)
d
,
i
,
k
,
l
,
m
0
.
0
5
(
0
.
2
2
,
0
.
3
7
)
2
4
d
,
k
,
l
0
.
0
4
(
0
.
1
6
,
0
.
2
8
)
d
,
i
,
k
,
l
,
m
0
.
0
9
(
0
.
1
2
,
0
.
3
4
)
2
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
d
,
k
,
l
0
.
2
1
(
0
.
9
0
,
0
.
5
9
)
d
,
i
,
k
,
l
,
m
0
.
2
7
(
0
.
6
3
,
0
.
1
8
)
2
4
d
,
k
,
l
0
.
1
5
(
0
.
5
1
,
0
.
2
8
)
d
,
i
,
k
,
l
,
m
0
.
1
4
(
0
.
3
6
,
0
.
1
3
)
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
4
d
,
k
,
l
0
.
0
3
(
0
.
7
4
,
0
.
3
1
)
d
,
i
,
k
,
l
,
m
0
.
1
3
(
0
.
1
6
,
0
.
4
7
)
E
r
c
a
n
e
t
a
l
.
(
2
0
0
5
)
8
3
s
b
,
d
,
k
6
m
o
n
t
h
s
m
e
d
i
c
a
t
i
o
n
;
e
i
g
h
t
s
e
s
s
i
o
n
s
o
f
p
a
r
e
n
t
t
r
a
i
n
i
n
g
1
,
2
1
d
,
k
0
.
0
5
(
0
.
3
4
,
0
.
1
7
)
d
,
k
0
.
0
5
(
0
.
1
5
,
0
.
2
6
)
K
l
e
i
n
a
n
d
A
b
i
k
o
f
f
(
1
9
9
7
)
8
9
s
b
,
k
8
w
e
e
k
s
1
,
2
,
1
3
,
1
4
1
c
,
d
,
k
,
l
,
m
0
.
3
8
(
0
.
1
0
,
0
.
9
9
)
c
,
d
,
k
,
l
,
m
0
.
2
4
(
0
.
5
0
,
0
.
7
7
)
2
,
1
3
,
1
4
c
,
d
,
k
,
l
,
m
0
.
9
1
(
0
.
4
2
,
1
.
8
2
)
c
,
d
,
k
,
l
,
m
0
.
8
2
(
0
.
0
0
,
2
.
1
4
)
2
,
1
3
,
1
4
1
c
,
d
,
k
,
l
,
m
0
.
5
5
(
0
.
8
8
,
0
.
1
3
)
c
,
d
,
k
,
l
,
m
0
.
5
5
(
1
.
1
3
,
0
.
0
0
)
M
T
A
C
o
o
p
.
G
r
o
u
p
5
7
9
s
a
,
b
,
d
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
,
1
2
,
1
3
,
1
4
1
d
,
k
,
l
0
.
3
0
(
1
.
4
7
,
0
.
1
5
)
d
,
i
,
k
,
l
0
.
0
3
(
0
.
2
3
,
0
.
2
6
)
2
,
9
,
1
2
,
1
3
,
1
4
d
,
k
,
l
0
.
1
9
(
0
.
9
0
,
0
.
5
7
)
d
,
i
,
k
,
l
0
.
2
9
(
0
.
0
7
,
0
.
4
8
)
2
2
d
,
k
,
l
0
.
1
8
(
0
.
7
4
,
0
.
7
1
)
d
,
i
,
k
,
l
0
.
3
1
(
0
.
0
7
,
0
.
5
3
)
2
,
9
,
1
2
,
1
3
,
1
4
1
d
,
k
,
l
0
.
4
7
(
0
.
6
1
,
0
.
3
8
)
d
,
i
,
k
,
l
0
.
2
6
(
0
.
4
3
,
0
.
0
5
)
2
2
d
,
k
,
l
0
.
0
1
(
0
.
5
2
,
0
.
1
9
)
d
,
i
,
k
,
l
0
.
0
2
(
0
.
2
5
,
0
.
2
1
)
P
e
l
h
a
m
e
t
a
l
.
(
2
0
0
0
)
1
1
7
s
b
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
(
6
,
1
1
,
1
5
,
1
6
,
1
8
,
2
5
,
2
6
)
,
1
2
,
1
3
,
1
4
2
,
9
j
,
k
0
.
4
5
(
0
.
3
1
,
0
.
5
8
)
j
,
k
,
t
0
.
2
1
(
0
.
0
3
,
0
.
7
2
)
P
f
f
n
e
r
a
n
d
M
c
B
u
r
n
e
t
t
(
1
9
9
7
)
2
7
s
a
,
b
,
d
8
w
k
s
2
,
6
(
1
5
,
1
9
)
,
1
4
6
N
A
d
,
k
0
.
0
4
(
0
.
0
4
,
0
.
1
1
)
2
0
d
,
k
0
.
9
9
(
0
.
4
2
,
1
.
5
6
)
6
2
0
d
,
k
1
.
0
7
(
0
.
6
0
,
1
.
5
3
)
T
u
t
t
y
e
t
a
l
.
(
2
0
0
3
)
1
0
0
s
a
,
b
,
d
,
k
8
w
e
e
k
l
y
s
e
s
s
i
o
n
s
1
,
2
,
6
,
7
1
,
2
2
d
,
k
0
.
4
4
(
0
.
1
1
,
0
.
7
7
)
N
A
v
a
n
d
e
r
O
o
r
d
e
t
a
l
.
(
2
0
0
7
)
4
5
s
b
1
0
w
e
e
k
s
1
,
2
,
6
/
7
/
1
0
,
1
4
1
d
,
k
0
.
1
0
(
0
.
2
2
,
0
.
4
4
)
d
,
k
0
.
2
4
(
0
.
0
6
,
0
.
5
8
)
320 Clin Child Fam Psychol Rev (2007) 10:318334
1 3
T
a
b
l
e
1
c
o
n
t
i
n
u
e
d
A
2
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
w
i
t
h
i
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
s
y
m
p
t
o
m
a
n
d
b
e
h
a
v
i
o
r
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
S
y
m
p
t
o
m
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
B
e
h
a
v
i
o
r
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
K
o
l
k
o
e
t
a
l
.
(
1
9
9
9
)
1
6
s
a
,
b
,
d
,
k
8
w
e
e
k
s
;
b
-
m
o
d
e
v
e
r
y
o
t
h
e
r
w
k
,
2
m
e
d
d
o
s
e
s
a
d
m
i
n
r
a
n
d
o
m
l
y
1
x
/
w
k
d
u
r
i
n
g
w
k
s
3
8
1
,
9
(
1
5
,
1
6
,
1
8
,
2
5
)
1
j
0
.
0
5
(
0
.
2
3
,
0
.
3
3
)
j
,
t
0
.
1
3
(
0
.
4
1
,
0
.
7
2
)
9
j
0
.
5
1
(
0
.
2
2
,
0
.
7
9
)
j
,
t
0
.
2
7
(
0
.
2
6
,
0
.
8
5
)
2
3
j
1
.
9
2
(
1
.
7
2
,
2
.
1
2
)
j
,
t
0
.
7
7
(
0
.
4
1
,
1
.
4
7
)
9
1
j
0
.
4
4
(
0
.
9
8
,
0
.
1
0
)
j
,
t
0
.
1
3
(
0
.
6
9
,
0
.
6
8
)
2
3
j
1
.
1
4
(
0
.
5
6
,
1
.
7
1
)
j
,
t
0
.
5
0
(
0
.
4
7
,
1
.
0
9
)
P
e
l
h
a
m
e
t
a
l
.
(
2
0
0
5
a
)
2
1
s
a
,
b
,
d
,
k
8
w
e
e
k
s
;
b
-
m
o
d
e
v
e
r
y
o
t
h
e
r
w
e
e
k
f
o
r
4
w
k
s
;
3
m
e
d
d
o
s
e
s
e
a
c
h
r
a
n
d
o
m
l
y
a
d
m
i
n
1
x
/
w
k
f
o
r
6
w
k
s
1
,
9
(
2
,
1
4
,
1
5
,
1
6
,
1
8
,
2
5
)
1
j
,
k
0
.
9
3
(
0
.
7
0
,
1
.
1
5
)
j
,
k
,
t
0
.
7
7
(
0
.
5
3
,
1
.
3
1
)
9
j
,
k
1
.
5
5
(
1
.
3
1
,
1
.
7
8
)
j
,
k
,
t
1
.
1
9
(
0
.
7
2
,
1
.
8
6
)
2
3
j
,
k
3
.
0
2
(
1
.
8
9
,
4
.
1
5
)
j
,
k
,
t
1
.
8
3
(
0
.
6
5
,
3
.
9
7
)
9
1
j
,
k
0
.
6
1
(
0
.
6
5
,
0
.
5
6
)
j
,
k
,
t
0
.
4
0
(
0
.
7
9
,
0
.
0
1
)
2
3
j
,
k
0
.
9
6
(
0
.
6
3
,
1
.
2
9
)
j
,
k
,
t
0
.
7
7
(
0
.
2
5
,
1
.
8
1
)
A
3
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
s
i
n
g
l
e
c
a
s
e
d
e
s
i
g
n
s
:
s
y
m
p
t
o
m
a
n
d
b
e
h
a
v
i
o
r
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
S
y
m
p
t
o
m
m
e
a
s
u
r
e
*
*
E
f
f
e
c
t
B
e
h
a
v
i
o
r
m
e
a
s
u
r
e
*
*
E
f
f
e
c
t
R
e
i
t
m
a
n
e
t
a
l
.
(
2
0
0
1
)
3
s
a
,
b
,
d
2
0
d
a
y
s
;
3
0
4
0
o
b
s
e
r
v
a
t
i
o
n
s
1
,
1
5
2
3
m
I
M
t
I
M
1
,
1
5
1
m
I
M
t
I
M
1
,
1
5
1
5
m
I
M
t
I
M
1
5
1
m
I
M
t
I
M
1
5
2
3
m
I
M
t
I
M
1
2
3
m
I
M
t
I
M
B
1
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
b
e
t
w
e
e
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
p
a
r
e
n
t
i
n
g
/
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
a
n
d
i
n
t
e
r
n
a
l
i
z
i
n
g
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
P
a
r
e
n
t
i
n
g
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
I
n
t
e
r
n
a
l
i
z
i
n
g
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
A
b
i
k
o
f
f
G
r
o
u
p
1
0
3
s
a
,
b
,
k
2
4
m
o
n
t
h
s
1
,
2
/
4
,
6
,
8
,
1
0
,
1
1
1
e
,
f
0
.
1
2
(
0
.
3
5
,
0
.
2
5
)
s
0
.
2
9
1
+
2
1
e
,
f
0
.
3
1
(
0
.
5
7
,
0
.
1
3
)
s
0
.
3
B
a
r
k
l
e
y
e
t
a
l
.
(
2
0
0
0
)
1
5
8
s
d
1
2
m
o
n
t
h
s
2
,
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
e
,
i
0
.
0
9
(
0
.
0
3
,
0
.
2
9
)
d
,
k
0
.
2
8
(
0
.
0
3
,
0
.
3
5
)
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
e
,
i
0
.
0
4
(
0
.
0
9
,
0
.
0
3
)
d
,
k
0
.
0
7
(
0
.
3
9
,
0
.
1
0
)
2
4
e
,
i
0
.
0
3
(
0
.
0
9
,
0
.
1
9
)
d
,
k
0
.
1
3
(
0
.
0
0
,
0
.
2
1
)
1 3
T
a
b
l
e
1
c
o
n
t
i
n
u
e
d
B
1
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
b
e
t
w
e
e
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
p
a
r
e
n
t
i
n
g
/
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
a
n
d
i
n
t
e
r
n
a
l
i
z
i
n
g
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
S
y
m
p
t
o
m
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
B
e
h
a
v
i
o
r
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
2
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
e
,
i
0
.
1
5
(
0
.
4
2
,
0
.
0
6
)
d
,
k
0
.
3
5
(
0
.
5
0
,
0
.
2
0
)
2
4
e
,
i
0
.
0
6
(
0
.
1
0
,
0
.
0
2
)
d
,
k
0
.
1
6
(
0
.
3
1
,
0
.
0
9
)
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
4
e
,
i
0
.
0
9
(
0
.
0
2
,
0
.
3
0
)
d
,
k
0
.
2
0
(
0
.
0
4
,
0
.
5
5
)
E
r
c
a
n
e
t
a
l
.
(
2
0
0
5
)
8
3
s
b
,
d
,
k
6
m
o
n
t
h
s
m
e
d
i
c
a
t
i
o
n
;
e
i
g
h
t
s
e
s
s
i
o
n
s
p
a
r
e
n
t
t
r
a
i
n
i
n
g
1
,
2
1
g
0
.
0
2
N
A
K
l
e
i
n
a
n
d
A
b
i
k
o
f
f
(
1
9
9
7
)
8
9
s
b
,
k
8
w
e
e
k
s
1
,
2
,
1
3
,
1
4
1
N
A
d
,
k
0
.
1
0
(
0
.
2
8
,
0
.
3
2
)
2
,
1
3
,
1
4
d
,
k
0
.
0
9
(
0
.
5
7
,
0
.
2
0
)
2
,
1
3
,
1
4
1
d
,
k
0
.
1
9
(
0
.
0
0
,
0
.
3
2
)
M
T
A
G
r
o
u
p
5
7
9
s
a
,
b
,
d
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
,
1
2
,
1
3
,
1
4
1
e
,
f
,
g
,
h
,
i
0
.
2
2
(
0
.
0
1
,
0
.
4
2
)
d
,
k
,
s
0
.
1
1
(
0
.
2
3
,
0
.
0
0
)
2
,
9
,
1
2
,
1
3
,
1
4
e
,
f
,
g
,
h
,
i
0
.
1
8
(
0
.
0
5
,
0
.
3
7
)
d
,
k
,
s
0
.
0
3
(
0
.
2
4
,
0
.
2
6
)
2
2
e
,
f
,
g
,
h
,
i
0
.
1
8
(
0
.
0
7
,
0
.
5
0
)
d
,
k
,
s
0
.
0
9
(
0
.
1
3
,
0
.
3
7
)
2
,
9
,
1
2
,
1
3
,
1
4
1
e
,
f
,
g
,
h
,
i
0
.
0
5
(
0
.
1
5
,
0
.
1
2
)
d
,
k
,
s
0
.
0
8
(
0
.
2
5
,
0
.
1
1
)
2
2
e
,
f
,
g
,
h
,
i
0
.
0
0
2
(
0
.
1
4
,
0
.
2
3
)
d
,
k
,
s
0
.
1
3
(
0
.
0
0
,
0
.
2
6
)
P
f
f
n
e
r
a
n
d
M
c
B
u
r
n
e
t
t
(
1
9
9
7
)
2
7
s
a
,
b
,
d
8
w
k
s
2
,
6
(
1
5
,
1
9
)
,
1
4
6
N
A
d
,
k
0
.
3
2
(
0
.
7
6
,
0
.
1
3
)
2
0
d
,
k
0
.
3
1
(
0
.
2
1
,
0
.
4
0
)
6
2
0
d
,
k
0
.
6
7
(
0
.
2
6
,
1
.
0
8
)
T
u
t
t
y
e
t
a
l
.
(
2
0
0
3
)
1
0
0
s
a
,
b
,
d
,
k
8
w
e
e
k
l
y
s
e
s
s
i
o
n
s
1
,
2
,
6
,
7
1
,
2
2
e
0
.
4
5
N
A
v
a
n
d
e
r
O
o
r
d
e
t
a
l
.
(
2
0
0
7
)
4
5
s
b
1
0
w
e
e
k
s
1
,
2
,
6
/
7
/
1
0
,
1
4
1
N
A
s
0
.
0
3
B
2
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
w
i
t
h
i
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
p
a
r
e
n
t
i
n
g
/
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
a
n
d
i
n
t
e
r
n
a
l
i
z
i
n
g
d
o
m
a
i
n
s
:
n
o
n
e
B
3
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
s
i
n
g
l
e
c
a
s
e
d
e
s
i
g
n
s
:
p
a
r
e
n
t
i
n
g
/
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
a
n
d
i
n
t
e
r
n
a
l
i
z
i
n
g
d
o
m
a
i
n
s
:
n
o
n
e
C
1
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
b
e
t
w
e
e
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
a
c
a
d
e
m
i
c
a
n
d
s
o
c
i
a
l
/
p
e
e
r
d
o
m
a
i
n
s
S
t
u
d
y
N
D
i
a
g
n
o
s
t
i
c
p
r
o
c
e
d
u
r
e
s
*
*
T
r
e
a
t
m
e
n
t
l
e
n
g
t
h
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
*
C
o
n
t
r
o
l
s
*
A
c
a
d
e
m
i
c
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
S
o
c
i
a
l
/
p
e
e
r
m
e
a
s
u
r
e
*
*
M
e
a
n
E
S
(
r
a
n
g
e
)
A
b
i
k
o
f
f
G
r
o
u
p
1
0
3
s
a
,
b
,
k
2
4
m
o
n
t
h
s
1
,
2
/
4
,
6
,
8
,
1
0
,
1
1
1
d
,
q
0
.
1
5
(
0
.
6
4
,
0
.
3
1
)
d
,
k
,
l
,
s
0
.
0
1
(
0
.
3
1
,
0
.
4
2
)
1
+
2
1
d
,
q
0
.
1
8
(
0
.
8
2
,
0
.
1
5
)
d
,
k
,
l
,
s
0
.
1
0
(
0
.
4
6
,
0
.
1
6
)
B
a
r
k
l
e
y
e
t
a
l
.
(
2
0
0
0
)
1
5
8
s
d
1
2
m
o
n
t
h
s
2
,
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
k
,
q
0
.
1
2
(
0
.
2
9
,
0
.
1
7
)
d
,
k
0
.
4
8
(
0
.
2
0
,
0
.
7
1
)
1 3
T
a
b
l
e
1
c
o
n
t
i
n
u
e
d
C
1
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
b
e
t
w
e
e
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
a
c
a
d
e
m
i
c
a
n
d
s
o
c
i
a
l
/
p
e
e
r
d
o
m
a
i
n
s
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
k
,
q
0
.
2
5
(
0
.
5
6
,
0
.
0
4
)
d
,
k
0
.
1
7
(
0
.
3
1
,
0
.
0
2
)
2
4
k
,
q
0
.
0
8
(
0
.
3
7
,
0
.
1
7
)
d
,
k
0
.
0
3
(
0
.
3
0
,
0
.
3
3
)
2
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
k
,
q
0
.
1
2
(
0
.
4
2
,
0
.
0
4
)
d
,
k
0
.
5
8
(
0
.
7
6
,
0
.
4
5
)
2
4
k
,
q
0
.
0
4
(
0
.
2
8
,
0
.
2
6
)
d
,
k
0
.
4
2
(
0
.
4
8
,
0
.
3
4
)
6
,
7
,
1
1
,
1
3
,
1
4
,
1
5
,
1
6
,
1
8
2
4
k
,
q
0
.
1
6
(
0
.
0
7
,
0
.
4
5
)
d
,
k
0
.
1
8
(
0
.
0
4
,
0
.
4
5
)
H
o
z
a
e
t
a
l
.
(
2
0
0
5
)
2
8
5
s
b
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
,
1
2
,
1
3
,
1
4
1
N
A
r
0
.
0
6
(
0
.
2
1
,
0
.
0
6
)
2
,
9
,
1
2
,
1
3
,
1
4
r
0
.
2
1
(
0
.
1
2
,
0
.
3
7
)
2
2
r
0
.
0
5
(
0
.
0
1
,
0
.
1
1
)
2
,
9
,
1
2
,
1
3
,
1
4
1
r
0
.
2
7
(
0
.
4
1
,
0
.
0
8
)
2
2
r
0
.
1
7
(
0
.
2
7
,
0
.
0
9
)
K
l
e
i
n
a
n
d
A
b
i
k
o
f
f
(
1
9
9
7
)
8
9
s
b
,
k
8
w
e
e
k
s
1
,
2
,
1
3
,
1
4
1
p
,
q
0
.
1
6
(
0
.
2
7
,
0
.
0
8
)
c
,
d
,
k
,
m
0
.
0
6
(
0
.
1
8
,
0
.
0
0
)
2
,
1
3
,
1
4
p
,
q
0
.
1
1
(
0
.
0
2
,
0
.
3
3
)
c
,
d
,
k
,
m
0
.
1
8
(
0
.
7
2
,
0
.
9
6
)
2
,
1
3
,
1
4
1
p
,
q
0
.
3
5
(
0
.
9
2
,
0
.
0
9
)
c
,
d
,
k
,
m
0
.
2
3
(
0
.
9
1
,
0
.
5
0
)
M
T
A
G
r
o
u
p
5
7
9
s
a
,
b
,
d
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
,
1
2
,
1
3
,
1
4
1
q
0
.
0
7
(
0
.
0
5
,
0
.
1
0
)
d
,
k
,
r
0
.
0
9
(
0
.
0
5
,
0
.
1
9
)
2
,
9
,
1
2
,
1
3
,
1
4
q
0
.
1
5
(
0
.
0
1
,
0
.
2
3
)
d
,
k
,
r
0
.
3
5
(
0
.
2
7
,
0
.
4
2
)
2
2
q
0
.
1
6
(
0
.
0
1
,
0
.
2
7
)
d
,
k
,
r
0
.
3
0
(
0
.
1
8
,
0
.
4
6
)
2
,
9
,
1
2
,
1
3
,
1
4
1
q
0
.
0
8
(
0
.
1
6
,
0
.
0
4
)
d
,
k
,
r
0
.
2
5
(
0
.
4
0
,
0
.
0
8
)
2
2
q
0
.
0
0
(
0
.
0
4
,
0
.
0
5
)
d
,
k
,
r
0
.
0
5
(
0
.
1
9
,
0
.
0
3
)
P
e
l
h
a
m
e
t
a
l
.
(
2
0
0
0
)
1
1
7
s
b
,
k
1
4
m
o
n
t
h
s
1
,
2
,
9
(
6
,
1
1
,
1
5
,
1
6
,
1
8
,
2
5
,
2
6
)
,
1
2
,
1
3
,
1
4
2
,
9
n
,
o
0
.
1
5
(
0
.
0
9
,
0
.
2
5
)
j
,
r
,
t
0
.
3
9
(
0
.
1
8
,
0
.
7
7
)
P
f
f
n
e
r
a
n
d
M
c
B
u
r
n
e
t
t
(
1
9
9
7
)
2
7
s
a
,
b
,
d
8
w
k
s
2
,
6
(
1
5
,
1
9
)
,
1
4
6
N
A
d
,
k
,
s
0
.
3
7
(
0
.
5
2
,
0
.
2
7
)
2
0
d
,
k
,
s
1
.
5
3
(
0
.
4
2
,
2
.
4
1
)
6
2
0
d
,
k
,
s
2
.
1
1
(
0
.
6
8
,
3
.
4
1
)
v
a
n
d
e
r
O
o
r
d
e
t
a
l
.
(
2
0
0
7
)
4
5
s
b
1
0
w
e
e
k
s
b
e
h
a
v
i
o
r
a
l
t
r
e
a
t
m
e
n
t
1
,
2
,
6
/
7
/
1
0
,
1
4
1
N
A
d
,
k
0
.
1
4
(
0
.
0
8
,
0
.
1
9
)
C
2
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
w
i
t
h
i
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
a
c
a
d
e
m
i
c
a
n
d
s
o
c
i
a
l
/
p
e
e
r
d
o
m
a
i
n
s
K
o
l
k
o
e
t
a
l
.
(
1
9
9
9
)
1
6
s
a
,
b
,
d
,
k
8
w
e
e
k
s
;
b
-
m
o
d
e
v
e
r
y
o
t
h
e
r
w
k
,
2
m
e
d
d
o
s
e
s
a
d
m
i
n
r
a
n
d
o
m
l
y
1
x
/
w
k
d
u
r
i
n
g
w
k
s
3
8
1
,
9
(
1
5
,
1
6
,
1
8
,
2
5
)
1
N
A
j
0
.
1
5
(
0
.
3
7
,
0
.
6
6
)
9
j
0
.
4
6
(
0
.
3
9
,
0
.
5
3
)
2
3
j
0
.
9
8
(
0
.
6
5
,
1
.
3
1
)
9
1
j
0
.
1
3
(
0
.
5
6
,
0
.
3
1
)
2
3
j
0
.
6
0
(
0
.
2
1
,
0
.
9
8
)
1 3
T
a
b
l
e
1
c
o
n
t
i
n
u
e
d
C
2
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
w
i
t
h
i
n
-
g
r
o
u
p
s
d
e
s
i
g
n
s
:
a
c
a
d
e
m
i
c
a
n
d
s
o
c
i
a
l
/
p
e
e
r
d
o
m
a
i
n
s
P
e
l
h
a
m
e
t
a
l
.
(
2
0
0
5
a
)
2
1
s
a
,
b
,
d
,
k
8
w
e
e
k
s
;
b
-
m
o
d
e
v
e
r
y
o
t
h
e
r
w
e
e
k
f
o
r
4
w
k
s
;
3
m
e
d
d
o
s
e
s
e
a
c
h
r
a
n
d
o
m
l
y
a
d
m
i
n
1
x
/
w
k
f
o
r
6
w
k
s
1
,
9
1
n
,
o
0
.
0
1
(
0
.
4
4
,
0
.
4
6
)
j
,
k
0
.
3
5
(
0
.
4
8
,
0
.
2
2
)
9
n
,
o
1
.
1
0
(
0
.
7
1
,
1
.
4
9
)
j
,
k
0
.
4
9
(
0
.
3
7
,
0
.
6
1
)
2
3
n
,
o
1
.
2
2
(
0
.
6
0
,
1
.
8
3
)
j
,
k
2
.
3
2
(
1
.
1
1
,
3
.
5
3
)
9
1
n
,
o
0
.
8
8
(
0
.
9
4
,
0
.
8
2
)
j
,
k
0
.
7
6
(
0
.
9
9
,
0
.
5
2
)
2
3
n
,
o
0
.
1
5
(
0
.
0
5
,
0
.
3
5
)
j
,
k
1
.
2
8
(
0
.
7
2
,
1
.
8
4
)
C
3
:
M
u
l
t
i
m
o
d
a
l
i
n
t
e
r
v
e
n
t
i
o
n
s
t
u
d
i
e
s
e
m
p
l
o
y
i
n
g
s
i
n
g
l
e
c
a
s
e
d
e
s
i
g
n
s
:
a
c
a
d
e
m
i
c
a
n
d
s
o
c
i
a
l
/
p
e
e
r
d
o
m
a
i
n
s
:
n
o
n
e
N
o
t
e
:
N
A
=
n
o
t
a
p
p
l
i
c
a
b
l
e
.
U
n
d
e
r
N
:
p
=
p
r
e
s
c
h
o
o
l
;
s
=
s
c
h
o
o
l
-
a
g
e
.
*
T
r
e
a
t
m
e
n
t
c
o
m
p
o
n
e
n
t
s
a
n
d
c
o
n
t
r
o
l
c
o
n
d
i
t
i
o
n
s
:
1
=
s
t
i
m
u
l
a
n
t
m
e
d
i
c
a
t
i
o
n
o
r
m
e
d
i
c
a
t
i
o
n
a
s
s
e
s
s
m
e
n
t
/
r
e
f
e
r
r
a
l
;
2
=
p
a
r
e
n
t
t
r
a
i
n
i
n
g
;
3
=
p
a
r
e
n
t
-
t
a
r
g
e
t
e
d
p
a
r
t
n
e
r
s
u
p
p
o
r
t
a
n
d
/
o
r
c
o
p
i
n
g
s
k
i
l
l
t
r
a
i
n
i
n
g
;
4
=
f
a
m
i
l
y
p
r
o
b
l
e
m
-
s
o
l
v
i
n
g
o
r
c
o
m
m
u
n
i
c
a
t
i
o
n
t
h
e
r
a
p
y
;
5
=
s
u
p
p
o
r
t
g
r
o
u
p
;
6
=
c
h
i
l
d
-
t
a
r
g
e
t
e
d
s
o
c
i
a
l
s
k
i
l
l
s
t
r
a
i
n
i
n
g
;
7
=
c
h
i
l
d
-
t
a
r
g
e
t
e
d
s
e
l
f
-
c
o
n
t
r
o
l
/
a
n
g
e
r
m
a
n
a
g
e
m
e
n
t
t
r
a
i
n
i
n
g
;
8
=
i
n
d
i
v
i
d
u
a
l
c
h
i
l
d
-
t
a
r
g
e
t
e
d
p
s
y
c
h
o
t
h
e
r
a
p
y
;
9
=
b
e
h
a
v
i
o
r
a
l
s
u
m
m
e
r
t
r
e
a
t
m
e
n
t
p
r
o
g
r
a
m
;
1
0
=
c
h
i
l
d
f
o
c
u
s
e
d
a
c
a
d
e
m
i
c
o
r
g
a
n
i
z
a
t
i
o
n
a
l
s
k
i
l
l
s
t
r
a
i
n
i
n
g
;
1
1
=
i
n
d
i
v
i
d
u
a
l
i
z
e
d
a
c
a
d
e
m
i
c
a
s
s
i
s
t
a
n
c
e
/
r
e
m
e
d
i
a
t
i
o
n
/
t
u
t
o
r
i
n
g
;
1
2
=
b
e
h
a
v
i
o
r
a
l
l
y
t
r
a
i
n
e
d
i
n
d
i
v
i
d
u
a
l
p
a
r
a
p
r
o
f
e
s
s
i
o
n
a
l
s
u
p
p
o
r
t
;
1
3
=
s
c
h
o
o
l
c
o
n
s
u
l
t
a
t
i
o
n
a
n
d
/
o
r
t
e
a
c
h
e
r
p
s
y
c
h
o
-
e
d
u
c
a
t
i
o
n
;
1
4
=
d
a
i
l
y
r
e
p
o
r
t
c
a
r
d
;
1
5
=
t
o
k
e
n
r
e
i
n
f
o
r
c
e
m
e
n
t
;
1
6
=
t
o
k
e
n
r
e
s
p
o
n
s
e
-
c
o
s
t
;
1
7
=
d
e
l
a
y
e
d
r
e
w
a
r
d
;
1
8
=
t
i
m
e
o
u
t
;
1
9
=
p
e
e
r
a
t
t
e
n
t
i
o
n
/
f
e
e
d
b
a
c
k
/
p
o
s
i
t
i
v
e
r
e
i
n
f
o
r
c
e
m
e
n
t
;
2
0
=
w
a
i
t
-
l
i
s
t
c
o
n
t
r
o
l
;
2
1
=
a
t
t
e
n
t
i
o
n
c
o
n
t
r
o
l
;
2
2
=
c
o
m
m
u
n
i
t
y
c
a
r
e
;
2
3
=
t
r
e
a
t
m
e
n
t
w
i
t
h
d
r
a
w
a
l
;
2
4
=
n
o
i
n
t
e
r
v
e
n
t
i
o
n
;
2
5
=
a
d
u
l
t
a
d
m
i
n
i
s
t
e
r
e
d
p
o
s
i
t
i
v
e
r
e
i
n
f
o
r
c
e
m
e
n
t
/
p
r
a
i
s
e
;
2
6
=
s
p
o
r
t
s
s
k
i
l
l
s
t
r
a
i
n
i
n
g
.
*
*
D
i
a
g
n
o
s
t
i
c
a
n
d
o
u
t
c
o
m
e
m
e
a
s
u
r
e
s
:
a
=
c
l
i
n
i
c
a
l
d
i
a
g
n
o
s
i
s
b
y
p
e
d
i
a
t
r
i
c
i
a
n
o
r
m
e
n
t
a
l
h
e
a
l
t
h
p
r
o
f
e
s
s
i
o
n
a
l
;
b
=
c
l
i
n
i
c
i
a
n
a
d
m
i
n
i
s
t
e
r
e
d
i
n
t
e
r
v
i
e
w
w
i
t
h
p
a
r
e
n
t
,
c
h
i
l
d
,
a
n
d
/
o
r
t
e
a
c
h
e
r
;
c
=
c
l
i
n
i
c
i
a
n
r
a
t
i
n
g
s
;
d
=
p
a
r
e
n
t
r
a
t
i
n
g
s
;
e
=
p
a
r
e
n
t
r
a
t
i
n
g
s
o
f
p
a
r
e
n
t
i
n
g
p
r
a
c
t
i
c
e
s
/
s
t
r
a
t
e
g
i
e
s
;
f
=
c
h
i
l
d
r
a
t
i
n
g
s
o
f
p
a
r
e
n
t
i
n
g
p
r
a
c
t
i
c
e
s
/
s
t
r
a
t
e
g
i
e
s
;
g
=
p
a
r
e
n
t
r
a
t
i
n
g
s
o
f
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
;
h
=
c
h
i
l
d
r
a
t
i
n
g
s
o
f
p
a
r
e
n
t
c
h
i
l
d
r
e
l
a
t
i
o
n
s
h
i
p
;
i
=
p
a
r
e
n
t
c
h
i
l
d
o
b
s
e
r
v
a
t
i
o
n
s
;
j
=
c
o
u
n
s
e
l
o
r
r
a
t
i
n
g
s
;
k
=
t
e
a
c
h
e
r
r
a
t
i
n
g
s
;
l
=
c
l
a
s
s
r
o
o
m
o
b
s
e
r
v
a
t
i
o
n
s
;
m
=
o
b
s
e
r
v
e
r
r
a
t
i
n
g
s
;
n
=
a
c
a
d
e
m
i
c
p
r
o
d
u
c
t
i
v
i
t
y
;
o
=
a
c
a
-
d
e
m
i
c
p
e
r
f
o
r
m
a
n
c
e
;
p
=
c
o
g
n
i
t
i
v
e
t
e
s
t
s
;
q
=
a
c
h
i
e
v
e
m
e
n
t
t
e
s
t
s
;
r
=
p
e
e
r
r
a
t
i
n
g
s
o
r
n
o
m
i
n
a
t
i
o
n
s
;
s
=
c
h
i
l
d
s
e
l
f
-
r
a
t
i
n
g
s
;
t
=
b
e
h
a
v
i
o
r
c
o
u
n
t
s
.
F
o
r
c
a
s
e
s
t
u
d
i
e
s
,
I
M
=
i
m
p
r
o
v
e
d
;
N
C
=
n
o
c
h
a
n
g
e
;
D
T
=
d
e
t
e
r
i
o
r
a
t
e
d
.
T
h
e
r
a
n
g
e
o
f
e
f
f
e
c
t
s
i
z
e
s
i
s
d
e
s
i
g
n
a
t
e
d
b
y
:
(
l
o
w
e
r
b
o
u
n
d
,
u
p
p
e
r
b
o
u
n
d
)
A
b
i
k
o
f
f
G
r
o
u
p
s
t
u
d
i
e
s
i
n
c
l
u
d
e
d
:
A
b
i
k
o
f
f
e
t
a
l
.
2
0
0
4
a
,
b
;
H
e
c
h
t
m
a
n
e
t
a
l
.
2
0
0
4
a
,
b
M
T
A
G
r
o
u
p
s
t
u
d
i
e
s
i
n
c
l
u
d
e
d
:
M
T
A
C
o
o
p
e
r
a
t
i
v
e
G
r
o
u
p
1
9
9
9
;
W
e
l
l
s
e
t
a
l
.
2
0
0
0
,
2
0
0
6
H
o
z
a
e
t
a
l
.
(
2
0
0
5
)
a
n
d
P
e
l
h
a
m
e
t
a
l
.
(
2
0
0
0
)
a
r
e
c
o
n
s
i
d
e
r
e
d
s
e
p
a
r
a
t
e
l
y
f
r
o
m
t
h
e
M
T
A
G
r
o
u
p
s
t
u
d
i
e
s
b
e
c
a
u
s
e
t
h
e
s
e
s
t
u
d
i
e
s
a
r
e
b
a
s
e
d
o
n
o
n
l
y
a
s
u
b
s
e
t
o
f
M
T
A
p
a
r
t
i
c
i
p
a
n
t
s
1 3
available in the articles to make other comparisons in
addition to examining incremental benet (e.g., compari-
son of a multimodal or unimodal treatment to a no
intervention group). Although not explicitly discussed here,
these comparisons also are included in the table for the
interested reader.
In evaluating incremental benet, we provide data
regarding the magnitude of incremental benet treatment
effects to the best of our ability, given the constraints of
original articles. Specically, within each relevant outcome
domain, we report Cohens d effect sizes that assess post-
treatment differences between the multimodal treatment vs.
the initial or baseline treatment condition. Importantly,
because treatment packages often were used as either the
baseline or incremental treatment, a single treatment con-
dition could include multiple treatment components; in
interpreting results from these studies, effects sizes repor-
ted pertain to the effects of the packages as a whole and
conclusions may not be drawn about the individual com-
ponent interventions. In computing effect sizes for within-
subjects studies, we averaged data across all repetitions of
the same treatment condition. Further, and regardless of
type of study design, we averaged effect sizes across all
dependent measures within each outcome domain of any
given study in order to generate a single, overall effect size
for the treatment effect in that domain and study. We note
that results for different individual measures within a single
domain did not always produce consistent results; conse-
quently, our averaged results do not necessarily produce
the same conclusions as those reported by the original
authors based on individual measures. Finally, the reader
should note that we altered the direction of effect sizes with
the goal of improving ease of comparison across outcome
domains. Thus, positive effect sizes denote improved
functioning on the part of the multimodal treatment group
or condition (i.e., incremental benet) relative to the uni-
modal treatment group or condition (e.g., lower ADHD
symptoms, better academic performance). Effect sizes
were interpreted according to the guidelines of Cohen
(1988), who categorized values below 0.20 as non-impor-
tant, values from 0.20 to \0.50 as small in magnitude,
values from 0.50 to \0.80 as moderate in magnitude,
and values at or above 0.80 as large. Where we discuss
case studies that employed visual inspection of graphs to
draw conclusions, we limit our discussion to descriptive
comments and report for each domain, whether improve-
ment, deterioration, or no change was observed (in the
absence of any better method with which to quantify these
effects). We recognize the limited generalizability of
results from single-subjects designs to the ADHD popula-
tion at large, yet include them for purposes of
comprehensiveness.
Using these parameters to structure our discussion, we
organize our review around ve key issues that we view as
particularly relevant in examining evidence for incremental
benet of treatment for childhood ADHD: (1) How does
choice of baseline or initial treatment(s), and particularly
initial treatment dosage, affect conclusions about incre-
mental benet?; (2) How do we interpret studies involving
treatment packages?; (3) How does choice of outcome
measure(s) affect conclusions drawn?; (4) How do we
reach a summary conclusion when multiple outcome
measures are used?; and (5) What are the implications of
these issues for the design and conduct of future studies? In
discussing these issues, due to space limitations, we choose
specic exemplars from the literature that we nd partic-
ularly useful in illustrating our points rather than
attempting a discussion of each and every study indexed in
Table 1. However, we include a listing of all studies in the
table for the interested reader.
Issue 1: How does Choice of Baseline or Initial
Treatment, Timing of the Addition of Treatments, and/
or their Dosages, Affect Conclusions about Incremental
Benet?
This issue lies at the crux of attempts to evaluate incre-
mental benet of multiple treatments over the effects of
an initial treatment or treatment package. Yet, this issue
has received little attention in the literature. First,
regarding choice of initial treatment, because there is a
nite amount of improvement that can occur in treatment
studies in certain domains, there may be an inverse
relationship between the amount of variance accounted
for by an initial treatment, and the remaining variance
that can be accounted for by a second treatment (Cunn-
ingham 1999). In such a circumstance, the initial
treatment may produce a large improvement, whereas the
added treatment produces a lesser effect or no signicant
effect. In domains where no known treatment comes close
to normalizing functioning (e.g., peer relationships), this
is less of a concern. Nonetheless, the timing of when the
treatments are added relative to one another is very
important, especially in domains in which one of the
treatments does come close to normalizing functioning.
There is greater likelihood of the second treatment
showing improvement either if both treatments are started
together, if the second treatment is added to a lower dose
of the initial treatment, or if the outcome domain is one in
which neither treatment alone comes close to normalizing
functioning.
Examining the studies in Table 1, we see several illus-
trations of this point. For example, the Abikoff Group
1 3
(Abikoff et al. 2004a, b; Hechtman et al. 2004a, b) inclu-
ded only medication responders in their study and titrated
these children to their maximum tolerable effective dose of
methylphenidate (up to 50 mg/day) prior to beginning the
psychosocial intervention (Klein et al. 2004). These
researchers found no incremental effects of the psychoso-
cial intervention, as reected in our averaged effect sizes,
in any of the domains summarized in the table. Similarly,
researchers studying a sample of Turkish children with
ADHD (Ercan et al. 2005), did not begin parent training
until children had been on medication for a full month,
with upward medication dose adjustments occurring during
this month (and subsequently); these researchers found no
evidence of incremental benet of parent training. In
contrast, Klein and Abikoff (1997) began medication and
psychosocial treatments concurrently and found modest
evidence for incremental benet of combined treatment
(medication plus a behavioral intervention) over the effects
of medication alone in a subset of domains. Hence, timing
of the initial treatment relative to the second treatment may
be a contributing factor in a studys outcome.
Perhaps even more illustrative of this point are results
from Pelham et al. (2005a), who crossed placebo and three
doses of methylphenidate (MPH) with two levels of
behavior modication (present vs. absent) in their study
design. Since means and standard deviations were reported
for all conditions, we were able to examine evidence for
incremental benet of combined treatment (intensive
summer treatment program plus medication) as a function
of the dose of medication in both medication-only and
combined treatment conditions (12.5, 25, 37.5 mg
transdermal MPH). As Fig. 1 depicts, higher doses of
medication in the medication-only condition were related
to lower incremental benet of combined treatment relative
to medication-only treatment (averaged across 13 outcome
variables in the behavior domain); this was true regardless
of medication dose in the combined treatment condition.
More specically, incremental benet of combined treat-
ment was greatest when combined treatment was compared
to the lowest dose in the medication-only condition (Co-
hens ds averaged across the 13 outcome measures ranged
from 0.9 to 1.2). Incremental benet of combined treatment
was lowest when the dose of medication in the medication-
only treatment condition was highest (averaged Cohens ds
ranged from 0.2 to 0.8). Hence, it is critically important
that issues such as these be routinely discussed in inter-
preting the results of clinical trials.
Thus far, we have discussed the incremental benet of
adding behavioral treatment to medication as being largely
dependent on the dose of medication being employed when
behavioral treatment is added. However, the reverse phe-
nomenon holds as well, as Pelham et al. (2000) aptly
demonstrated. Specically, using a subset of subjects from
the MTA study, Pelham and colleagues compared children
receiving a maximum intensity multi-component
behavioral intervention package in the context of a com-
prehensive summer treatment program to those receiving
this same intensive behavioral treatment package combined
with medication. Importantly, they reported no incremental
benet resulting from also receiving medication on 30 of
35 dependent variables reported, despite the fact that
medication is an intervention of known efcacy for ADHD.
This study, together with those summarized immediately
above, nicely illustrates the potential impact of dosage
of initial treatments, whether pharmacological or behav-
ioral, on comparisons evaluating incremental benet.
Fortunately, discussions of the effects of treatment
sequencing and dosages on outcomes are emerging in the
literature with increasing frequency. For example, in regard
to the primary analyses from the Multimodal Treatment
Study of Children with ADHD (MTA; MTA Cooperative
Group 1999), Pelham (1999) argued that the superiority of
Medication Management over Behavioral Treatment at
14 months was attributable to differences in dose or
intensity of treatments at the 14-month assessment point,
specically, intensive medication vs. faded (i.e., lower
intensity) behavioral treatment. Similar arguments have
been made to explain the limited differences reported
between Combined Treatment and Medication Manage-
ment at 14 months in the MTA study (Cunningham 1999).
Follow-up articles directly examining these issues using
data from the MTA study, however, have yielded seem-
ingly conicting ndings. As just noted, Pelham et al.
(2000) argue that comparisons between a subgroup of
MTA participants in an early phase of treatment who were
receiving Behavior Therapy during its most active and
0
0.2
0.4
0.6
0.8
1
1.2
1.4
MED 12.5 MED 25 MED 37.5
Medication Dose in Medication Only Condition (in mg)
E
f
f
e
c
t

S
i
z
e

I
n
d
e
x
i
n
g

I
n
c
r
e
m
e
n
t
a
l

B
e
n
e
f
i
t
COMB 12.5
COMB 25
COMB 37.5
Fig. 1 Cohens d effect sizes (averaged across 13 outcome variables
in the behavioral domain) indicating incremental benet of multi-
modal treatment relative to medication only (transdermal
methylphenidate), according to medication dose, in both combined
and medication-only conditions. (Data taken from: Pelham et al.
(2005a) )
1 3
intensive phase (i.e., during the summer treatment pro-
gram), as compared to children receiving Combined
Treatment, yielded little incremental benet of Combined
Treatment over very intensive Behavior Therapy, arguing
that behavior therapy is a sufcient treatment used alone if
it is intensive enough. However, Arnold et al. (2004),
compared the MTA groups a number of months later at the
9 month assessment point, when subjects were no longer
involved in the intensive summer program, but parent
training and the school-based behavior therapy intervention
were only moderately intensive, and found results conso-
nant with the 14 month ndingsspecically, superiority
of Medication Management over Behavior Therapy alone
for symptom measures.
How do we make sense of these results? Our view is
that these results are not necessarily contradictory if
viewed within the context of the dosages of medication
and behavior therapy being administered, since each of
these treatments can vary widely in dosage levels. How-
ever, dosages of behavior therapy have not been widely
considered in the psychosocial treatment literature. Yet, it
is well known that direct contingency management (such
as was delivered in Pelham et al. 2000) is more intensive
and generally more effective than clinical behavior ther-
apy, or behavior modication delivered by other agents
parents, teacherswho have been taught the skills by
clinicians (Pelham et al. 1998). Hence, comparisons of
behavior therapy to other treatments in the MTA should
yield more favorable results for behavior therapy when
maximum intensity direct contingency management is
employed (as in Pelham et al. 2000) than when less
intensive clinical behavior therapy is employed, albeit
with the support of paraprofessional classroom aides (as
in Arnold et al. 2004). This indeed is the pattern that was
obtained.
1
From our perspective, it is not problematic that results
diverge within the contexts of these differing treatment
designs; indeed, as we just have demonstrated, these
divergences generally can be reconciled through careful
contemplation of methodological and design issues. How-
ever, parents, teachers, health care professionals, and other
consumers of the results may interpret and utilize the
results without careful consideration of these important
design issues, as they may not be immediately apparent to
some consumers, especially those with little or no research
training. Investigators can assist in ameliorating this con-
cern by emphasizing these factors in research reports; and
striving through a variety of media (e.g., question and
answer sheets for parents and teachers) to make results of
complex studies accessible to audiences of all levels of
methodological sophistication. This is a very important
point to which we will return in the discussion.
Issue 2: How do we Interpret Studies Involving
Treatment Packages?
Most of the studies listed in Table 1 involved packages
of psychosocial treatments. For example, in the Abikoff
Groups study (Abikoff et al. 2004a, b; Hechtman et al.
2004a, b; Klein et al. 2004) the multi-component psycho-
social treatments employed included parent training/family
therapy, academic skills training, individual psychotherapy
for the child, and social skills training, as part of a psy-
chosocial treatment package. Similarly, Pelham et al.
(2000) employed a summer treatment program intervention
involving multiple components including an extensive
point system, social skills training and other peer inter-
ventions (cooperative tasks with peers, a friendship
intervention), sports skills training, a classroom interven-
tion modeled after a special education classroom, a daily
report card, and parent training. Indeed, multi-component
treatment packages have, to some extent, become the norm
in psychosocial treatment for childhood ADHD, largely
due to the absence of a single successful treatment that
effectively normalizes the many decits associated with
ADHD. From an historical perspective, then, the emer-
gence of treatment packages makes sense, and represents a
logical response to a disorder comprising impairments in
multiple domains across the lifespan. From a research
perspective, however, available data do not permit ade-
quate evaluation of the individual components of the
packages employed. Whereas evaluating individual com-
ponents was unnecessary to the stated purposes of these
studies, and, as we noted earlier, no single study can
address all pertinent issues, this presents a quandary for
those trying to provide evidence-based, cost-effective ser-
vices, as well as for researchers trying to design future
1
Of course, intensive behavioral therapies, similarly to high doses of
medication, have limitations. Specically, aside from the limited
availability and high cost of intensive, complex, direct contingency
management programs (see Barkley 2000, for a discussion of this
issue in regard to the MTA study), an added difculty for
psychosocial treatment generally is that gains are difcult to maintain
outside the treatment setting (Richters et al. 1995). Indeed, general-
ization of gains and persistence of effects following participation in
an intensive behavioral treatment program (e.g., Pelham and Hoza
1996; Pelham et al. 2005b) has not been adequately studied. In
addition, follow-up studies into adolescence and young adulthood of
children who participated in intensive behavioral treatments as
children are rare, and have only recently begun to appear in the
literature. Consistent with other long-term follow-up studies of
ADHD samples (for a brief review, see Richters et al. 1995), these
studies document continuing adjustment problems and risky behav-
iors into adolescence and adulthood relative to controls (e.g., Flory
et al. 2006; Molina et al. 2007; Thompson et al. 2007). This suggests
that short-term intensive behavioral treatment, like medication, has
limitations.
1 3
studies. It is simply unclear exactly which components of
treatment are most critical to include, or in what manner
critical components are most effectively delivered. Of
course, one could easily argue that until a comprehensive
treatment package that successfully manages the chronic,
varied symptoms and impairments of ADHD across mul-
tiple developmental periods is identied, a dismantling
strategy may be premature. We recognize the value in such
a position, yet, wish to point out the need for such an
approach at the time that our state of knowledge reaches a
point where it becomes feasible.
In a related fashion, another issue that arises in the
context of discussing treatment packages is whether con-
sumers of such treatments view receiving more
components as necessarily more desirable than fewer
components. Indeed, relatively little published work has
examined consumer satisfaction with regard to ADHD
treatments. As the number of components increases, so too
does the implementation burden on parents, teachers, and
others involved in the intervention, consequently perhaps
decreasing compliance with the interventions and limiting
potential gains. Further, costs of these interventions also
generally increase as intervention packages become more
complex. Studies are needed to determine whether and at
what point consumer satisfaction and effectiveness may
begin to diminish as additional components are added or as
treatments are made more complex.
Interestingly, available consumer satisfaction data sug-
gest that parents report greater satisfaction with treatments
involving behavioral components than with those involving
medication alone (Pelham et al. 2007). Teachers in this
same study similarly indicated that they felt better equip-
ped to address ADHD in their classrooms when behavioral
treatments were used, as compared to medication man-
agement alone (Pelham et al. 2007). These results are
impressive given the greater demands of the behavioral
intervention in terms of time required from parents and
teachers, as well as the level of improvement that was
obtained for families receiving medication management.
Yet, the point at which satisfaction begins to drop as
demands increase on participants is an unknown area in
need of further study.
Issue 3: How does Choice of Outcome Measure(s) Affect
Conclusions Drawn?
In performing this review, we were struck by a simple fact:
different outcome measures purported to measure the same
domain often yielded drastically different results for the
same comparisons within a study. The reader is encouraged
to peruse the ranges of effect sizes obtained in each domain
for each study, as presented in Table 1. Often effect sizes
within the same domain were of widely varying magnitude
and frequently also in opposite directions. For example, in
the Klein and Abikoff (1997) study, for the ADHD
symptoms domain, effect sizes for comparisons of com-
bined psychosocial and psychostimulant treatment to
stimulant treatment alone ranged from .10 to +.99,
depending on the measure. In the MTA study (MTA
Cooperative Group 1999), also for the symptoms domain,
comparisons of combined treatment to behavior therapy
alone were similarly variable, with effect sizes ranging
from .90 to +.57. Of importance, this pattern was not
unique to specic studies, but for the most part, charac-
terized this group of studies overall. This simple fact was
surprising to us in preparing this article and may not be
evident to the eld at large, as results are generally not
grouped by domain across sources, as we have done. This
indicates the possibility that our methods, rather than our
purported constructs being measured, may be driving the
sizes of our effects, and relatedly, that the treatment effects
that we obtain reect, to a non-trivial degree, our chosen
measurement methods and analysis strategies.
We had difculty in discerning any pattern in the vari-
ation of effect sizes. This task was complicated by
methodological differences among studies, such as those
already discussed, as well as by the fact that each different
study generally employed different outcome measures and/
or different raters; for example, summer program-based
studies (e.g., Kolko et al. 1999) report ratings from summer
program counselors rather than classroom teachers. No
single study employed all possible types of outcome vari-
ables within any given domain, making comparisons of
effect sizes generated by data collected from one type of
rater vs. another quite difcult.
Nonetheless, there indeed appeared to be considerable
variance both among and within raters. More specically,
in studies examining incremental benet of multimodal
psychosocial and stimulant treatment upon ADHD symp-
toms or behavior vs. either psychosocial or stimulant
treatment alone, averaged effect sizes based on teacher
ratings generally were somewhat larger than were averaged
effect sizes based on parent ratings.
Further, even effect sizes from the same rater within the
same study and purportedly assessing the same construct
(yet based on administration of different measures) pro-
duced differing results. As an example, Klein and Abikoff
(1997) administered three separate measures of parent-rated
hyperactivity, providing a unique opportunity to compare
results of multiple measures of parent-rated hyperactivity.
These measures (and effect sizes indexing incremental
benet of combined psychosocial and psychostimulant
treatment over psychostimulant treatment alone) were as
follows: Parent Home Hyperactivity Scale (Cohens
d = 0.62); Parent CPRS Hyperactivity subscale (Cohens
1 3
d = 0.24); and Mother Hillside Behavior Scale, Gross
Motor Activity subscale (Cohens d = 0.12). Based on these
results, it seems likely that differences in items or in
wording among these measures contributed to the discrep-
ancies in effect size. Had these researchers not employed
this comprehensive multi-measure approach, results would
have depended upon the specic measure chosen.
As mentioned earlier, our strategy for dealing with these
widely varying results even within the same domain was to
average effect sizes across all measures within a domain to
yield one overarching effect size. But, we remind the
reader that using this strategy, our conclusions often dif-
fered from those of the original authors who interpreted
measures individually and may have weighted one measure
more heavily over another. We do not advocate for one
approach (e.g., averaging effect sizes) over another (e.g.,
interpreting individual effect sizes). Instead, we caution
consumers of the treatment literature that choice of out-
come measure will heavily inuence both size and possibly
also direction of effects in treatment outcome studies.
Hence, results cannot and should not be interpreted outside
the context of these factors.
Issue 4: How do we Reach a Summary Conclusion
When Multiple Outcome Measures are Used?
In light of the pressing issue just described, a question of
enormous importance for intervention researchers becomes:
How do we reach a summary conclusion when multiple
outcome measures are used? This is a dicey issue because
use of multiple outcome measures, though consistent with
generally recommended multi-method, multi-informant
approaches, opens the door for pre-selection by investiga-
tors based on any of a variety of factors. These may include
practical issues such as how quickly or easily one measure
vs. another can be made ready for analysis, or likelihood of
publication based on size or strength of obtained effects.
Unfortunately, null ndings remain hard to publish (Ro-
senthal 1979). This appears to be true even when studies are
well-designed and adequately powered; we are hopeful that
this is beginning to change, as it is just as important to know
what does not work as it is to know what does work. These
larger issues fall beyond the scope of this article, but are
raised merely as a backdrop for the ensuing discussion.
In this era of evidence-based treatments, with both best
practices guidelines and treatment manuals widely avail-
able, one might assume that similar guidelines exist to
guide us in choosing assessment instruments, both for
selection of study participants and in reporting and inter-
preting outcomes from treatment studies. Unfortunately,
few guidelines exist either for researchers who are report-
ing research results in scientic journals, or for consumers
of scientic journals, to assist in deciding what to conclude
under these often confusing circumstances. As noted
recently by Mash and Hunsley (2005), even though evi-
dence-based treatments have advanced signicantly over
the past decade, it generally is the case that evidence-based
assessment has not progressed in the same manner in
regards to most childhood disorders. Yet, conclusions
regarding treatment outcomes only are as valid as the
procedures used to select study participants and the reli-
ability and validity of treatment outcome measures
employed. Hence, the status of evidence-based assessment
bears directly on the level of evidence for evidence-based
treatments (EBTs). For example, practices vary widely
across studies in terms of whether parent informants alone,
or parent and teacher informants together, are used to
gather diagnostic information. Despite this variability in
assessment practices, however, and as noted by Mash and
Hunsley (2005): For the most part, treatment-focused task
force statements and guidelines have been silent on the use
of assessment in developing and evaluating EBTs, not-
withstanding general recommendations to use a reliable
and valid core assessment batteryrecommendations that
are presented with little guidance as to what makes an
assessment method or process reliable or valid in a par-
ticular context or for a particular purpose (pp. 363364).
This measurement quandary, however, is not necessarily
cause to throw the metaphorical baby out with the bath
water. Rather, this dilemma begs for the development of
measurement models that are mindful of these challenges,
even if completely eliminating these challenges may be too
ambitious a goal at the present time. Nonetheless, a key
rst step would be to outline the most important domains to
be assessed and the types of measures to be included in
treatment outcome studies for ADHD. Measures should
include not just those most easily obtained, such as parent
and teacher ratings, as ratings such as these are more prone
to various types of bias (particularly if the informants are
not blind to treatment conditions). Instead, these measures
are best balanced to a reasonable degree by more objective
measurese.g., peer sociometrics, achievement testing,
behavior countseven though more objective measures
such as these often are more expensive to obtain in terms of
time and resources. Finally, given our current knowledge
that ADHD is a chronic disorder that emerges early in
childhood and persists into adulthood (for a discussion of
long-term prognosis, see Richters et al. 1995), the appli-
cability of available measures, as well as their reliability
and validity across key developmental periods, is a par-
ticularly important consideration.
In addition to developmental sensitivity, importantly, the
inclusion of measurements thought to be sensitive to the
change produced by each type of treatment being compared
seems important to producing a balanced comparison among
1 3
treatments (Cunningham 1999). For example, whereas
medication might be expected to have a large impact on
symptom reduction, behavioral treatments such as parent
training might be expected to have the largest impact on the
parentchild relationship. If this is the case, results may be
most informative when these diverse measures can be
examined and presented simultaneously. Publishing data in
the order in which it becomes available based on ease of
preparation for analysis ought to be discouraged, in the
interest of presenting the most balanced picture.
In sum, measurement models need to be developed that
guide researchers in the selection of measures and that
promote consistent choice of domains and measurement
strategies across studies and across developmental periods.
Such models do not currently exist in the ADHD literature
or in the child psychological treatment literature at large,
but the eld would benet greatly from their development.
Attempts to produce such models should go hand-in-hand
with developments in evidence-based assessment and
necessarily should proceed in tandem. Indeed, the fact that
evidence-based treatment has marched ahead despite lim-
itations in our evidence-based assessments is a
complicating factor, as evidence-based assessment is a
necessary prerequisite for measuring outcomes of evi-
dence-based treatments.
Issue 5: What are the Implications of these Issues for
the Design and Conduct of Future Studies?
We began this article by indicating that outcomes of
treatment studies may be at least partially a function of
their designs. To this simple premise, we added the notion
that it is impossible for any given study to single-handedly
answer most key treatment questions. These are points that
we wish to reiterate as we discuss the implications of past
work for future studies. Our recommendations for future
investigators conducting research in this area follow from
each of our discussion points. Admittedly, a number of our
recommendations are applicable to research design gener-
ally and may not be unique to ADHD treatment outcome
research. Nonetheless, we believe a specic discussion of
how these points apply in this context is useful, particularly
since few such discussions appear in the ADHD literature
specically, or the child treatment literature more
generally.
Recommendation 1
Use care to insure that your choice of baseline or initial
treatment, the timing of when additional treatments are
added, and the dosages of treatments employed match your
research question. In other words, be clear regarding
whether your goal is to study the benet of adding a second
treatment to one that is already administered at full inten-
sity and considered primary, or whether you wish to
consider dosage effects of both treatments simultaneously
in order to evaluate the best dosage combination of treat-
ments from each modality; these are very different
questions. When the course of action chosen is the former,
be cautious not to draw conclusions about the latter as part
of the same study. In other words, for example, if a child is
titrated to their maximum effective dose of medication and
behavior therapy is thereafter added as a secondary treat-
ment, avoid drawing conclusions about the relative
effectiveness of any other dosage combinations of these
two treatments. For example, based on this design, you
cannot rule out the possibility that the child might respond
just as well to a high intensity behavioral intervention
coupled with a minimal medication dose; such a compar-
ison must be made directly.
Recommendation 2
Use care not to draw conclusions about which compo-
nent(s) are responsible for the success or failure of an
intervention when you employ multiple treatments used
together as a package. Similarly, avoid concluding that
each of the components of a successful intervention are
necessary to create that successful outcome. In like fashion,
you cannot conclude that each of the components of an
unsuccessful intervention are not valuable, as you cannot
rule out the possibility that the various treatments under-
mined one another. In other words, empirical studies
employing a treatment package do not provide an evidence
base for the components of the treatment package. Hence,
if you design a study involving a treatment package, use
care that it is only the package about which you draw
conclusions. All other conclusions require a different
design.
Recommendation 3
This recommendation, derived from issues 3 and 4, has to
do with choice of outcome measures and how to reach a
summary conclusion involving multiple (and often con-
icting) outcome measures. We believe that this is the area
in need of the greatest work, and with the greatest potential
to shape the eld. As previously noted (Pelham et al.
2005c), studies comparing assessment strategies are badly
needed and few have been conducted to date. We see a
particularly great need for development of standardized
approaches to assessing treatment outcome that are
1 3
applicable across diverse developmental periods (e.g., from
preschool-age to adulthood). This is not to say that each
study or each developmental period should employ the
exact same measures, but rather, common principles
regarding domains assessed and types of measures required
for adequate coverage of a domain should guide the eld,
with the goal of increasing comparability across studies
and across the lifespan. A logical approach might include
covering 45 key domains of functioning, and including
several established sources of information for each. These
measures might include both pragmatic measures such as
parent and teacher ratings (despite limitations such as rater
bias), as well as more objective strategies such as behavior
counts, peer nominations, and achievement testing. Based
on the wide range of effect sizes displayed in Table 1, even
for measures that purportedly tap the same domain, we
advise extreme caution in relying on a single measure or
type of measure and suggest that researchers err on the side
of comprehensiveness rather than parsimony.
In making this recommendation, however, we note that
we would hope that any future evidence-based and stan-
dardized approach to outcome assessment does aim for
brevity; as any researcher or clinician knows, administering
fewer measures is logistically easier and more cost-effec-
tive than is administering more. Indeed, an important
recent review (Pelham et al. 2005c) summarizes the state
of the eld in regard to assessment of ADHD and identies
this as one of numerous pressing issues for future research.
Summarizing their conclusions is beyond the scope of this
article and, hence, the reader is referred directly to this
article. Among their key recommendations, however, is an
appeal for simplicity, parsimony, a focus on functional
problems as opposed to symptoms, and a return to greater
use of functional behavior analysis in both initial assess-
ment and in evaluating outcomes. These authors also
suggest the possibility of a briefer assessment approach to
ADHD based on a combination of simple strategies that
have heretofore been used in a variety of separate studies;
for example, a small number of items rated by parents and
teachers, behavior counts of an individualized target
behavior, teacher records of academic work completion
and accuracy, and whether or not a child has their school
supplies in their desk (Pelham et al. 2005c). These
researchers acknowledge: The kind of large-scale
research needed to validate this approach to assessment has
not yet been conducted, but clinicians can be condent
based on extant research that it has some empirical sup-
port (Pelham et al. 2005c, p. 468).
We agree with Pelham et al. (2005c) recommendation
that assessment of functional difculties (as opposed to
DSM-IV symptoms) is most appropriate for clinical
intervention, and we similarly nd their arguments for
brevity in assessment quite compelling. Still, we believe
that precisely the type of large-scale research to which
Pelham et al. refer is a necessary prerequisite that should
occur prior to adoption of this type of assessment strat-
egy. Thus, we do not endorse (at least not at the present
time) combining several simple assessment strategies,
each with some supportive evidence, in lieu of the cur-
rently accepted comprehensive assessment approach. In
our view, taking this approach to assessment in treatment
outcome research may be premature in light of our nd-
ings that choice of outcome measure can dramatically
affect the results of between-treatment comparisons.
Instead, we encourage researchers to pursue validation of
measurement models comprised of carefully selected
combinations of measures, and to directly compare dif-
fering combinations of measures in terms of their
usefulness, reliability and validity, incremental predictive
utility, applicability across diverse developmental periods,
sensitivity to treatment response, and cost. In other words,
we recommend an approach similar to our recommenda-
tion in regard to examination of treatment packages,
whereby we argued that evidence for the package does
not imply evidence for the components and vice versa.
Although not without drawbacks, we believe this
approach has the greatest chance of encouraging
researchers and clinicians to select measures and treat-
ment components carefully, and to avoid adding measures
or treatments in the absence of evidence for their incre-
mental usefulness.
Recommendation 4
Finally, and at the risk of sounding prescriptive, we
encourage researchers to be explicit and emphatic about
what questions their studies are and are not equipped to
answer. No study is perfect, and no study answers all
questions; hence, we encourage open acknowledgement of
design limitations that inuence interpretation of study
results. There are no penultimate studiesonly studies
designed to answer specic questions. Researchers are
encouraged to resist the temptation to go beyond their data
in drawing conclusions and to correct others, including
media representatives, who may portray their study results
in an overly zealous fashion. Most often such misrepre-
sentations are unintentional and a result of
misunderstanding of the nuances of the study designsan
understandable problem among lay consumers. We
encourage researchers to correct these misunderstandings,
even if doing so requires great effort and additional work.
This is critical as our work is read not only by our col-
leagues, but also guides policy issues, affects insurance
coverage and directs best practices guidelines. Interpret
each and every study cautiously, as if your own childs or
1 3
grandchilds future may be affected by it; in fact, it likely
will be.
Limitations
Just as no research study answers all study questions, so
too, no review covers all pertinent issues. Our focus was on
several specic issues, at the expense of others that,
admittedly, may be equally or even more important. Before
closing, we wish to make some of these limitations explicit,
in the hope that other researchers may tackle some of these
issues that we leave unaddressed.
A key unaddressed issue has to do with the effects of
comorbidity on outcomes in incremental benet studies.
Given the overwhelming number of comparisons already
summarized in Table 1, we chose not to include comor-
bidity comparisons in the subset of studies that reported
them. However, as work by Jensen et al. (2001) has dem-
onstrated, different comorbid subgroups of children with
ADHD may respond better or worse to different treatments.
Hence, some of the lack of clarity of the big picture we
describe herein, might be claried by such an analysis of
comorbid subgroups. We leave this task to a future team of
investigators.
Similarly, we did not evaluate the evidence separately
by age, gender, length of treatment, or differences in
sample characteristics (e.g., chosen by comprehensive
clinical assessments vs. DSM-based rating scales).
Although we include descriptive data in regard to some of
these characteristics in Table 1, an in-depth analysis is
needed to clearly elucidate their effects. Again, this
remains a task for future work.
Finally, the issues we chose to highlight are admittedly
controversial ones. Many viewpoints are possible, and
considering differing perspectives is an important part of
the scientic process. We are certain that we have not
captured all possible issues, and even in regard to the issues
we chose to address, additional viewpoints were left un-
explicated. We hope that other researchers will assist in
lling the gaps we leave unaddressed and the perspectives
that we leave unexplicated. Nonetheless, if this article
serves to stimulate discussion regarding the importance of
design issues to study outcomes, and if it promotes con-
structive discussion about how to improve future work,
then we have accomplished our goals.
Summary
The goal of this article was to outline issues critical to
evaluating the literature on incremental benet of multiple
effective treatments used together, vs. a single effective
treatment, for childhood ADHD. These issues included: (1)
sequencing and dosage of treatments being combined and
compared; (2) difculty drawing valid conclusions about
individual components of treatment when treatment pack-
ages are employed; (3) differing results emerging from
measurement tools that purportedly measure the same
domain; and (4) the resultant difculty in reaching a
summary conclusion when multiple outcome measures
yielding conicting results are used. The implications of
these issues for the design and conduct of future studies
were discussed, with a particular emphasis on the need to
develop and validate measurement models that can guide
research.
Acknowledgments During the preparation of this manuscript, all
authors were supported in part by a grant from the National Institute
of Mental Health to the rst author: R01 MH065899. The rst author
also was supported in part by N01 MH12010 to B.S.G. Molina; and
the rst and third authors were supported in part by R43 MH076359
to A. Terrazas. The views represented in this article are solely those of
the authors.
References
Abikoff, H., Hechtman, L., Klein, R. G., Gallagher, R., Fleiss, K.,
Etcovitch, J., et al. (2004a). Social functioning in children with
ADHD treated with long-term methylphenidate and multimodal
psychosocial treatment. Journal of the American Academy of
Child and Adolescent Psychiatry, 43, 820829.
Abikoff, H., Hechtman, L., Klein, R. G., Weiss, G., Fleiss, K.,
Etcovitch, J., et al. (2004b). Symptomatic improvement in
children with ADHD treated with long-term methylphenidate
and multimodal psychosocial treatment. Journal of the American
Academy of Child and Adolescent Psychiatry, 43, 802811.
American Psychiatric Association (1994). Diagnostic and statistical
manual of mental disorders (4th ed.). Washington, DC: Author.
Arnold, L. E., Chuang, S., Davies, M., Abikoff, H. B., Conners, C. K.,
Elliott, G. R., et al. (2004). Nine months of multicomponent
behavioral treatment for ADHD and effectiveness of MTA
fading procedures. Journal of Abnormal Child Psychology, 32,
3951.
Barkley, R. A. (2000). Commentary on the multimodal treatment
study of children with ADHD. Journal of Abnormal Child
Psychology, 28, 595599.
Barkley, R. A., Shelton, T. L., Crosswait, C., Moorehouse, M.,
Fletcher, K., Barrett, S., et al. (2000). Multi-method psycho-
educational intervention for preschool children with disruptive
behavior: Preliminary results at post-treatment. Journal of Child
Psychology and Psychiatry, 41, 319332.
Cohen, J. (1988). Statistical power analysis for the behavioral
sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Cunningham, C. E. (1999). In the wake of the MTA: Charting a new
course for the study and treatment of children with attention-
decit hyperactivity disorder. Canadian Journal of Psychiatry,
44, 9991006.
Ercan, E. S., Varan, A., & Deniz, U. (2005). Effects of combined
treatment on Turkish children diagnosed with attention-decit/
hyperactivity disorder: A preliminary report. Journal of Child
and Adolescent Psychopharmacology, 15, 203219.
Flory, K., Molina, B. S. G., Pelham, W. E., Gnagy, E., & Smith, B.
(2006). Childhood ADHD predicts risky sexual behavior in
1 3
young adulthood. Journal of Clinical Child & Adolescent
Hechtman, L., Abikoff, H., Klein, R. G., Greeneld, B., Etcovitch, J.,
Cousins, L., et al. (2004a). Children with ADHD treated with
long-term methylphenidate and multimodal psychosocial treat-
ment: Impact on parental practices. Journal of the American
Academy of Child and Adolescent Psychiatry, 43, 830838.
Hechtman, L., Abikoff, H., Klein, R. G., Weiss, G., Respitz, C.,
Kouri, J., et al (2004b). Academic achievement and emotional
status of children with ADHD treated with long-term methyl-
phenidate and multimodal psychosocial treatment. Journal of the
American Academy of Child and Adolescent Psychiatry, 43,
812819.
Hoza, B., Gerdes, A. C., Mrug, S., Hinshaw, S. P., Bukowski, W. M.,
Gold, J. A., et al. (2005). Peer-assessed outcomes in the
multimodal treatment study of children with attention decit
hyperactivity disorder. Journal of Clinical Child and Adolescent
Hoza, B., Johnston, C., Pillow, D. R., & Ascough, J. C. (2006).
Predicting treatment response for childhood attention-decit/
hyperactivity disorder: Introduction of a heuristic model to guide
research. Applied and Preventive Psychology, 11, 215229.
Hoza, B., Kaiser, N. M., & Hurt, E. (in press). Evidence-based
treatments for attention-decit/hyperactivity disorder (ADHD).
In M. Roberts, D. Elkin, & R. Steele (Eds.), Handbook of
evidence based therapies for children and adolescents. New
York: Springer.
Hoza, B., Owens, J. S., & Pelham, W. E. (1999). Attention-decit/
hyperactivity disorder. In R.T. Ammerman, M. Hersen, & C. G.
Last (Eds.), Handbook of prescriptive treatments for children
and adolescents (2nd ed., pp. 6383). Boston: Allyn & Bacon.
Jensen, P. S., Hinshaw, S. P., Kraemer, H. C., Lenora N., Newcorn J.
H., Abikoff, H. B., et al. (2001). ADHD comorbidity ndings
from the MTA study: Comparing comorbid subgroups. Journal
of the American Academy of Child & Adolescent Psychiatry, 40,
147158.
Klein, R. G., & Abikoff, H. (1997). Behavior therapy and methyl-
phenidate in the treatment of children with ADHD. Journal of
Attention Disorders, 2, 89114.
Klein, R. G., Abikoff, H., Hechtman, L., & Weiss, G. (2004). Design
and rationale of controlled study of long-term methylphenidate
and multimodal psychosocial treatment in children with ADHD.
Journal of the American Academy of Child and Adolescent
Psychiatry, 43, 792801.
Kolko, D. J., Bukstein, O. G., & Barron, J. (1999). Methylphenidate
and behavior modication in children with ADHD and comorbid
ODD or CD: Main and incremental effects across settings.
Journal of the American Academy of Child and Adolescent
Psychiatry, 38, 578586.
Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of
child and adolescent disorders: Issues and challenges. Journal of
Clinical Child & Adolescent Psychology, 34, 362379.
Molina, B. S. G., Pelham, W. E., Gnagy, E. M., Thompson, A. L., &
Marshal, M. P. (2007). Attention-decit/hyperactivity disorder
risk for heavy drinking and alcohol use disorder is age specic.
Alcoholism: Clinical and Experimental Research, 31, 643654.
Mrug, S., Hoza, B., & Gerdes, A. C. (2001). Children with attention-
decit/hyperactivity disorder: Peer relationships and peer-ori-
ented interventions. In W. Damon (Series Ed.), D. W. Nangle, &
C. A. Erdley (Eds.), New directions for child and adolescent
development: No. 91. The role of friendship in psychological
adjustment (pp. 5177). San Francisco: Jossey-Bass.
MTA Cooperative Group (1999). A 14-month randomized clinical
trial of treatment strategies for attention-decit/hyperactivity
disorder. Archives of General Psychiatry, 56, 10731086.
Owens, J. S., & Hoza, B. (2003). The role of inattention and
hyperactivity/impulsivity in the positive illusory bias. Journal of
Consulting and Clinical Psychology, 71, 680691.
Pelham, W. E. (1999). The NIMH multimodal treatment study for
attention-decit hyperactivity disorder: Just say yes to drugs
alone? Canadian Journal of Psychiatry, 44, 981990.
Pelham, W. E., Burrows-MacLean, L., Gnagy, E. M., Fabiano, G. A.,
Coles, E. K., Tresco, K. E., et al. (2005a). Transdermal
methylphenidate, behavioral, and combined treatment for chil-
dren with ADHD. Experimental and Clinical
Psychopharmacology, 13, 111126.
Pelham, W. E., Erhardt, D., Gnagy, E. M., Greiner, A. R., Arnold, L.
E., Abikoff, H. B., et al. (2007). Parent and teacher evaluation
of treatment in the MTA: Consumer satisfaction and perceived
effectiveness. University at Buffalo (submitted).
Pelham, W. E., Fabiano, G. A., Gnagy, E. M., Greiner, A. R., & Hoza,
B. (2005b). The role of summer treatment programs in the
context of comprehensive treatment for attention-decit/hyper-
activity disorder. In E. D. Hibbs & P. S. Jensen (Eds.),
Psychosocial treatments for child and adolescent disorders:
Empirically based strategies for clinical practice (2nd ed., pp.
377409). Washington, DC: American Psychological
Association.
Pelham, W. E., Fabiano, G. A., & Massetti, G. M. (2005c). Evidence-
based assessment of attention decit hyperactivity disorder in
children and adolescents. Journal of Clinical Child and Adoles-
cent Psychology, 34, 449476.
Pelham, W. E., Gnagy, E. M., Greiner, A. R., Hoza, B., Hinshaw, S.
P., Swanson, J. M., et al. (2000). Behavioral versus behavioral
and pharmacological treatment in ADHD children attending a
summer treatment program. Journal of Abnormal Child Psy-
chology, 28, 507525.
Pelham, W. E., & Hinshaw, S. P. (1992). Behavioral intervention for
attention decit-hyperactivity disorder. In S. M. Turner, K. D.
Calhoun, & H. E. Adams (Eds.), Handbook of clinical behavior
therapy (Vol. 2, pp. 259283). New York: Wiley.
Pelham, W. E., & Hoza, B. (1996). Intensive treatment: A summer
treatment program for children with ADHD. In E. D. Hibbs & P.
S. Jensen (Eds.), Psychosocial treatments for child and adoles-
cent disorders: Empirically based strategies for clinical practice
(pp. 311340). Washington, DC: American Psychological
Association.
Pelham, W. E., Wheeler, T., & Chronis, A. (1998). Empirically
supported psychosocial treatments for attention decit hyperac-
tivity disorder. Journal of Clinical Child Psychology, 27, 190
205.
Pffner, L. J., & McBurnett, K. (1997). Social skills training with
parent generalization: Treatment effects for children with
attention decit disorder. Journal of Consulting and Clinical
Reitman, D., Hupp, S. D. A., OCallaghan, P. M., Gulley, V., &
Northup, J. (2001). The inuence of a token economy and
methylphenidate on attentive and disruptive behavior during
sports with ADHD-diagnosed children. Behavior Modication,
25, 305323.
Richters, J. E., Arnold, L. E., Jensen, P. S., Abikoff, H., Conners, C.
K., Greenhill, L. L., et al. (1995). NIMH collaborative multisite
multimodal treatment study of children with ADHD: I. Back-
ground and rationale. Journal of the American Academy of Child
& Adolescent Psychiatry, 34, 9871000.
Rosenthal, R. (1979). The le drawer problem and tolerance for
null results. Psychological Bulletin, 86, 638641.
Swanson, J. M., Elliott, G. R., Greenhill, L. L., Wigal, T., Arnold, L.
E., Vitiello, B., et al. (2007). Effects of stimulant medication on
growth rates across 3 years in the MTA follow-up. Journal of the
1 3
American Academy of Child and Adolescent Psychiatry, 46,
10151027.
Thompson, A. L., Molina, B. S. G., Pelham, W., & Gnagy, E. M.
(2007). Risky driving in adolescents and young adults with
childhood ADHD. Journal of Pediatric Psychology, 32, 745759.
Tutty, S., Gephart, H., & Wurzbacher, K. (2003). Enhancing
behavioral and social skill functioning in children newly diag-
nosed with attention-decit hyperactivity disorder in a pediatric
setting. Developmental and Behavioral Pediatrics, 24, 5157.
van der Oord, S., Prins, P. J. M., Oosterlaan, J., & Emmelkamp, P. M.
G. (2007). Does brief, clinically based, intensive multimodal
behavior therapy enhance the effects of methylphenidate in
children with ADHD? European Child and Adolescent Psychi-
atry, 16, 4857.
Volkow, N. D., & Insel, T. R. (2003). What are the long-term effects
of methylphenidate treatment? Biological Psychiatry, 54,
13071309.
Wells, K. C., Chi, T. C., Hinshaw, S. P., Epstein, J. N., Pffner, L.,
Nebel-Schwalm, M., et al. (2006). Treatment-related changes in
objectively measured parenting behaviors in the multimodal
treatment study of children with attention-decit/hyperactivity
disorder. Journal of Consulting and Clinical Psychology, 74,
649657.
Wells, K. C., Epstein, J. N., Hinshaw, S. P., Conners, C. K., Klaric, J.,
Abikoff, H. B., et al. (2000). Parenting and family stress
treatment outcomes in attention decit hyperactivity disorder
(ADHD): An empirical analysis in the MTA study. Journal of
Abnormal Child Psychology, 28, 543553.
1 3

Accesare Resursa PDF

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Accesare Resursa PDF

Diunggah oleh

Hak Cipta:

Format Tersedia

Multimodal Treatments for Childhood Attention-decit/

Hyperactivity Disorder: Interpreting Outcomes in the Context

Anda mungkin juga menyukai