Available online at www.sciencedirect.

com

Epilepsy & Behavior 12 (2008) 276–280

www.elsevier.com/locate/yebeh

The validity of the Child Behavior Checklist for children with epilepsy
a,*
U. Gleissner , N.E. Fritz a, M. Von Lehe b, R. Sassen a, C.E. Elger a, C. Helmstaedter a

a
Department of Epileptology, University of Bonn, Bonn, Germany
b
Department of Neurosurgery, University of Bonn, Bonn, Germany

Received 14 September 2007; revised 25 October 2007; accepted 25 October 2007

Available online 11 December 2007

Abstract

The validity of the Child Behavior Checklist (CBCL) for patients with epilepsy has been questioned, because several items may reflect
seizure semiology rather than habitual behavior. This study compared a standard version of the CBCL with an adjusted version that
excluded those ambiguous items. Participants were 58 pediatric patients with epilepsy who were assessed preoperatively and 1 year after
successful surgical treatment. Before surgery, the adjusted version indicated significantly lower values for the scales Attention Problems,
Thought Problems, and Total Problems than the standard version. After surgery, the difference between the standard and adjusted ver-
sions and the scores for the ambiguous items were unchanged, although all patients were completely seizure free at that time. Elevated
scores on the ambiguous items thus probably reflect real behavioral problems and are not due to confusion with seizure semiology. The
results support the CBCL as a valid assessment tool in children with epilepsy.
 2007 Elsevier Inc. All rights reserved.

Keywords: Epilepsy; Child Behavior Checklist; Ambiguous items; Validity; Behavioral problems

1. Introduction (for instance, staring blankly, nervous movements or

Children with neurological diseases are at a higher risk
for psychopathology than children with nonneurological
diseases [1]. Therefore, examining possible psychopathol-
ogy is a standard component of the diagnostic procedure
in children with epilepsy. The Child Behavior Checklist
(CBCL) [2] is the most frequently used instrument for
assessing behavioral adjustment in these patients. In a
meta-analysis by Rodenburg et al. [3], 31 of 46 studies
assessed behavioral problems using the CBCL. However,
the CBCL was not originally designed for use with children
who have chronic physical conditions, and there are con-
cerns about its validity in such patients [4,5]. When applied
to children with epilepsy, it is possible that several items are
rated on the basis of seizure features rather than habitual
behavior and, hence, render behavioral scores ambiguous
*
Corresponding author. Address: Department of Epileptology, Univer-
sity of Bonn, Sigmund-Freud Strasse 25, 53105 Bonn, Germany. Fax: +49
(0)228 287 14486.
E-mail address: ulrike.gleissner@ukb.uni-bonn.de (U. Gleissner).
twitching, wets self, and daydreams). In a review of 46
studies that included 2434 children with epilepsy, Roden-
burg et al. [3] found attention problems, thought problems,
and social problems to be relatively specific to children with
epilepsy compared with children with other chronic dis-
eases. Four of the eleven items that contribute to the Atten-
tion Problems scale and two of the seven items that
contribute to the Thought Problems scale are ambiguous.
Oostrom et al. [5] rescored the seven ambiguous items in
the CBCL as missing values. As a consequence, the per-
centage of patients who exceeded the clinical cutoff score
for at least one of the scales, to which these items contrib-
uted, fell from 46 to 23%. This decrease in psychopathol-
ogy ratings occurred only in children with epilepsy, not
in healthy controls. The authors concluded that CBCL-
derived psychopathology percentages should therefore be
interpreted cautiously.
Eliminating critical items increases the risk that poten-
tially useful information about psychological functioning
might be omitted. However, including the critical items
raises another problem: the frequency and the extent of

1525-5050/$ - see front matter  2007 Elsevier Inc. All rights reserved.
doi:10.1016/j.yebeh.2007.10.013
 U. Gleissner et al. / Epilepsy & Behavior 12 (2008) 276–280 277

behavioral problems may be overestimated. The distortion Table 1

takes effect only in one direction (overestimation of prob- Group characteristics
lems), and a result that is below the critical cutoff can be Number of patients 58
regarded as correct. Nevertheless, incorrectly classifying a Gender (male/female) 27/31
Age at presurgical examination (years) 9.8 (3.4)a
child without behavioral problems as being behaviorally Age at onset of epilepsy (years) 4.5 (3.9)a
impaired would be serious and should be avoided. Preoperative IQ 82.8 (16.4) [44–125]b
The reported studies used the old version of the CBCL Side of surgery (left/right) 36/22
(4–18 years) [2]. Yet the same problems also arise for the Site of surgery (number of patients)
new version of the CBCL (6–18 years) [6], because it still Temporal 28
contains all the ambiguous items. The recently developed Frontal 10
preschool version of the CBCL (1.5–5 years) [7] also Parietal/occipital 8
Functional hemispherectomy 7
includes several ambiguous items (daydreams or gets lost
Multilobular resection 5
in his/her thoughts, nervous movements or twitching, fidg- a
ets, and strange behavior). Mean (SD).
b
Mean (SD) [range].
The present study further evaluates the degree of distor-
tion created by the ambiguous items in the CBCL by exam- 2.2. Child Behavior Checklist
ining the pre- and postoperative CBCL results of pediatric
patients who became completely seizure free after epilepsy The CBCL, which yields profiles of children’s problems as observed by
surgery. Because the patients had had their last seizure parents and other caregivers, was used to assess behavioral problems pre-
operatively and 1 year after surgery. It contains descriptions of 118 behav-
more than 1 year before the postoperative assessment, the
ioral and emotional problems. The respondent is asked to rate the child’s
estimates given by the parents should no longer reflect behavior over the previous 6 months on a graded scale: 0 = item is not
the seizures. The postoperative assessment was performed true; 1 = somewhat or sometimes true; 2 = very or often true. The result-
1 year after surgery, and the CBCL asks for behavioral ing profile consists of eight subordinate problem scales (Anxiety/Depres-
problems during a time interval of the previous 6 months. sion, Withdrawal, Somatic Complaints, Aggression, Delinquent
Behaviour, Attention Problems, Thought Problems, and Social Problems)
Thus, the patients have been seizure free for the entire time
that are grouped into superordinate scales: Internalizing Problems (subor-
interval as dictated by the CBCL. Comparing the standard dinate scales: Withdrawal, Somatic Complaints, Anxiety/Depression),
version and the results obtained by an adjusted version, Externalizing Problems (subordinate scales: Delinquent Behavior, Aggres-
which was corrected for the epilepsy-critical items, should sion) and total problems (sum of all items besides two items that assess
result in a smaller deviation after surgery if the ambiguous asthma and allergy). The scores of the items that constitute a scale are
summarized (summed raw scores) and transformed into standardized
items reflect mainly seizure semiology. In this case,
scores (T scores) according to the norms provided in the test manual.
the scores on the ambiguous items should decline According to expert ratings provided in another study [5], there are seven
postoperatively. items in the CBCL that give rise to ambiguity. This is because the wording
Lendt et al. [8] described an early improvement in could reflect aspects of habitual behavior as well as aspects of a seizure:
behavioral problems 3 months after epilepsy surgery in
the CBCL. The behavioral improvements were assumed • Is confused or seems to be in a fog (item 13)
• Has daydreams or gets lost in his/her thoughts (item 17)
to result directly from removal of the epileptic focus,
• Has nervous movements or twitching (item 46)
because they were dependent on seizure outcome. How- • Stares blankly (item 80)
ever, the critical items were included in the CBCL evalua- • Shows strange behavior (item 84)
tion, and therefore, it is not possible to tell whether the • Wets self during the day (item 107)
improvement indeed reflected behavioral improvement or • Wets the bed (item 108)
if it was due to the fact that the seizure symptoms were
These ambiguous items contribute to three subordinate and two super-
no longer incorrectly reported as behavioral problems in
ordinate scales:
seizure-free patients. Therefore, a second purpose of the
present study was to check the degree to which postopera- • Attention Problems (4 of 11 items)
tive improvement is due to a change in the ambiguous • Withdrawal (1 of 9 items)
items and to investigate if postoperative improvement is • Thought Problems (2 of 7 items)
also observed in the adjusted CBCL. • Internalizing Problems (1 of 32 items)
• Total Problems (7 of 118 items).

2. Methods
2.1. Patients
The neuropsychological database of the Department of Epileptology
at the University of Bonn was retrospectively screened for pediatric
patients (age <17) who were completely seizure free 1 year after a surgical
treatment and had a pre- and postoperative assessment with the CBCL.
The search yielded 58 patients ranging between 4 and 17 years in age.
Patient characteristics are provided in Table 1.
We used two sets of behavioral scores for these scales: the first set was
obtained according to the manual [9] and is referred to as CBCL. For the sec-
ond set, the ambiguous items were treated as missing values; that is, they were
not included in the summed raw scores. This set is referred to as CBCLadj..
3. Results
Table 2 provides the pre- and postoperative results
obtained with the standard version and the adjusted
278 U. Gleissner et al. / Epilepsy & Behavior 12 (2008) 276–280

Table 2
Comparison of the standard and adjusted versions of the CBCL
Attention Problems Thought Problems Withdrawal Internalizing Problems Total Problems
Presurgery
CBCL 62.5 (9.3) 57.3 (7.9) 60.2 (9.1) 58.7 (10.1) 59.3 (9.1)
CBCLadj. 59.5 (6.8) 55.1 (6.6) 60.1 (8.7) 58.7 (9.9) 58.6 (8.9)
*** *** ***
T test for paired samples n.s. n.s.
Postsurgery
CBCL 57.1 (7.9) 52.8 (5.4) 56.1 (7.9) 50.9 (11.2) 50.5 (11.1)
CBLCadj. 55.5 (6.5) 52.0 (4.8) 56.5 (8.2) 51.1 (11.3) 49.9 (10.9)
** ** **
T test for paired samples n.s. n.s.
Note. T scores are cutoff scores for clinically significant impairment: T score P 68 for the subordinate scales, T score P 60 for the superordinate scales.
Values are means (SD). CBCL, Child Behavior Checklist; adj., adjusted version; n.s., not significant (P > 0.05).
***
P < 0.001.
**
P < 0.01.

version of the CBCL. A comparison with t tests for paired
samples with the standard and adjusted versions indicates,
for the adjusted version, significantly lower results for the
scales Attention Problems, Thought Problems, and Total
Problems before surgery (t tests for paired samples, all t
> 4.7 with P < 0.001) as well as after surgery (all t > 2.9
with P < 0.007). For the superordinate scales, we compared
the number of patients scoring above the critical cutoff
score (T score P 60) for the standard and adjusted CBCL
(Fig. 1) by computing Cohen j coefficients. The results
show a high congruence pre- and postoperatively for the
scales Total Problems and Internalizing Problems (Cohen
j coefficients before surgery > 0.93, after surgery > 0.91,
a coefficient of 1 indicates maximal congruence).
T tests for paired samples with the pre- and postopera-
tive group average scores as pairs indicated significant
improvements for all scales independent of whether the
standard CBCL or the adjusted CBCL was used (CBCL:
all t > 3.8 with P < 0.001; CBCLadj.: all t > 2.7 with
P < 0.01). Postoperative changes in the ambiguous items
were evaluated by univariate analyses of variance, with
the time of examination (pre- and postoperative) as an
independent variable and the items as dependent variables.
A significant difference between the pre- and postoperative
values was evident only for the item stares blankly,
for which the postoperative score was lower than the
preoperative score (preoperative mean: 0.3, postoperative
mean: 0.1, F = 6.0, P = 0.015). All other ambiguous items
showed no significant postoperative changes (F < 3.0 with
P > 0.09).
A discordant classification means that a child is classi-
fied as impaired only in the standard CBCL and not in
the adjusted version (T score P 68 for the subordinate
scales, T score P 60 for the superordinate scales). The dis-
cordant classification percentages are reported in Table 3.
They were observed mainly in the subordinate scales, but
rarely in the superordinate scales. After surgery, the per-
centage of discordant classifications tended to be higher
for Attention Problems and lower for the Withdrawal
scale. For the other scales, the percentage of discordant
classifications remained largely unchanged.
Elevated levels of behavior problems have been reported
in children with low IQ [10,11], and our sample included
many patients with intellectual disability or borderline
intelligence (21% of patients had an IQ < 70, 35% of
patients had an IQ between 70 and 85). Correlational anal-
yses, however, indicated no significant associations
between IQ and the superordinate scales of the CBCL in
our sample (r < 0.11 with P > 0.4). A group comparison

60 Table 3
Before surgery After surgery
% behavioural problems

Discordant classifications
50
Percentage of discordant classifications
40 Before surgery After surgery
Attention Problems 40 (6/15) 62 (5/8)
30
Thought Problems 57 (4/7) 50 (1/2)
Withdrawal 25 (3/12) 0 (0/4)
20
Internalizing Problems 4 (1/27) 7 (1/15)
Total Problems 3 (1 of 29) 12 (2/17)
10
Total Internalizing Total Internalizing Note. In a discordant classification, a child is classified as impaired
problems problems problems problems (T score P 68 for the subordinate scales, T score P 60 for the superor-
Standard CBCL Adjusted CBCL dinate scales) in the standard version, but not in the adjusted version. For
instance, on the Attention Problems scale, 15 children were preoperatively
Fig. 1. Percentage of patients with problem scores above the cutoff. A classified as being impaired in the standard version, and of those 15
comparison of the standard and adjusted versions of the CBCL before and children, only 6 (=40%) were still classified as impaired when the adjusted
after surgery is illustrated. version was used.
 U. Gleissner et al. / Epilepsy & Behavior 12 (2008) 276–280
between patients with intellectual disability (IQ < 70),
borderline intelligence (IQ between 70 and 85), and average
or above-average intelligence (IQ > 85) indicated no
differences in the superordinate scales of the CBCL preop-
eratively (ANOVAs, F < 0.2 with P > 0.8) and postopera-
tively (ANOVAs, F < 2.2 with P > 0.1). Hence, it can be
assumed that our results are not significantly influenced
by elevated levels of behavior problems in patients with
low IQ.
4. Discussion
Our findings support the validity of the CBCL for the
assessment of behavioral adjustment in paediatric patients
with epilepsy. Less psychopathology was indicated by
using the adjusted version versus the standard version.
However, this was still true after surgery when the patients
had been free of seizures for the 12 months prior to the
postoperative evaluation. It is therefore unlikely that the
ratings of the parents led to false classifications by merely
reflecting seizure semiology. This is supported by the find-
ing that the ratings of the ambiguous items remained
largely unchanged after surgery. Discordant classifications
(i.e., a child was classified as impaired only in the standard
version but not in the adjusted version) were very rare for
the superordinate scales. In a carefully designed study,
Oostrom et al. [5] evaluated the CBCL in children with
recently diagnosed epilepsy. As in our study, the difference
between the standard and adjusted scores was evident
mainly for the narrowband scales (Attention Problems,
Withdrawal, and Thought Problems), but negligible for
the broadband scales (Internalizing Problems and Total
Problems). Their study and ours differ with respect to the
date of examination. At the time of preoperative examina-
tion, our patients had had epilepsy for several years,
whereas Oostrom and colleagues’ patients were newly diag-
nosed. Directly after diagnosis, the effect of ambiguity was
prominent in that study and decreased over time. The
authors explained that this was due to: (1) the relatively
high frequency of seizures (antiepileptic drug treatment
had not yet begun) and (2) the increasing knowledge of
parents, which, over time, helped them to differentiate
between epilepsy symptoms and habitual behavior. In our
study, more than 50% of patients had intellectual disability
or borderline intelligence. Elevated levels of behavior prob-
lems have been reported in children with low IQ in the lit-
erature [10,11]. In our sample, however, behavior problems
were equally frequent in patients with average intelligence
and patients with low IQ. Therefore, it can be regarded
unlikely that our results are significantly influenced by
the low IQ of our patients.
Discordant classifications were found in our study, par-
ticularly for the Attention Problems and Thought Prob-
lems scales. Both scales have a relatively large number of
ambiguous items. After surgery, the discordant classifica-
tion rates remained largely unchanged (Thought Problems:
57% vs 50%) or even increased (Attention Problems: 40%
279
vs 62%). Several studies have described Attention Problems
as frequent behavioral problems in patients with epilepsy.
Dunn et al. [12] reported that 25% of the adolescents with
epilepsy and 37% of the children with epilepsy had scores
above the clinical cutoff for the CBCL Attention Problems
scale. Davies et al. [13] found that 12% of children with
uncomplicated epilepsy had attention-deficit/hyperactivity
disorder according to the DSM-IV (Diagnostic and Statis-
tical Manual of Mental Disorders, Fourth Edition). By
using the Attention Deficit Disorder Evaluation Scale—
Home Version (ADDES-HV), Williams et al. [14] found
that children with epilepsy were at an increased risk for
inattentiveness and hyperactivity. They investigated 42
newly diagnosed children with epilepsy at the time of diag-
nosis and again when they were seizure free for at least 3
months under medical treatment. The frequency of inatten-
tive and hyperactive–impulsive symptoms remained largely
unchanged, although the children were seizure free. In our
study, the absolute number of patients with attention prob-
lems decreased after surgery, but the rate of discordant
classification did not. We therefore suggest that diagnosing
attention problems with the standard version of the CBCL
can be regarded as valid and does not appear to be due to
confusion with seizure semiology.
When the CBCL is used in a clinical context, the exam-
iner has, in principle, the opportunity to clarify and to
interpret the meaning of the critical items. To be on the safe
side, it is possible to inform the caregiver that seizure symp-
toms should not be included in the rating. Austin et al. [15]
instructed the parents not to include any behavior that
might belong to seizure semiology or relate to seizures.
Nevertheless, in their study, newly diagnosed children with
epilepsy had higher levels of psychopathology than those
children from a general population.
All in all, our results demonstrate that valid results can be
obtained using the CBCL in children with epilepsy, at least
for those children who have had epilepsy over a longer per-
iod. The number of ambiguous items is small, and the bias
caused by item ambiguity seems to be negligible. The super-
ordinate scales are only marginally influenced by the ambig-
uous items. Most discordant are the results obtained with an
adjusted version for the subscales Attention Problems and
Thought Problems. However, elevated scores on the Atten-
tion Problems scale probably reflect real attention problems
and are not due mainly to confusion with seizures. The scale
Thought Problems should be interpreted with caution any-
way because of its low reliability [9]. The postoperative
improvement in behavioral problems that has been reported
in previous studies (see, for instance, [8]) seems to be real. In
most patients it is not due to disappearance of seizure-related
behavior after complete seizure relief.
References
[1] Rutter M, Graham P, Yule W. A neuropsychiatric study in
childhood. Clinics in Developmental Medicine No. 35/36. Spastics
International Medical Publications. London: Mac Keith Press; 1970.
280 U. Gleissner et al. / Epilepsy & Behavior 12 (2008) 276–280
[2] Achenbach TM, Edelbrock C. Manual for the Child Behavior
Checklist: 4–18 and 1991 profile. Burlington: Univ. of Vermont,
Department of Psychiatry; 1991.
[3] Rodenburg R, Stams GJ, Meijer AM, Aldenkamp AP, Dekovic M.
Psychopathology in children with epilepsy: a meta-analysis. J Pediatr
Psychol 2005;30:453–68.
[4] Perrin EC, Stein RE, Drotar D. Cautions in using the Child Behavior
Checklist: observations based on research about children with a
chronic illness. J Pediatr Psychol 1991;16:411–21.
[5] Oostrom KJ, Schouten A, Kruitwagen CL, Peters AC, Jennekens-
Schinkel A. Epilepsy-related ambiguity in rating the Child Behavior
Checklist and the Teacher’s Report Form. Epileptic Disord
2001;3:39–45.
[6] Achenbach TM, Rescorla L.A. Manual for ASEBA School-Age
Forms & Profiles. Burlington: Univ. of Vermont, Research Center for
Children, Youth, & Families; 2001.
[7] Achenbach TM, Rescorla L. A Manual for ASEBA Preschool Forms
& Profiles. Burlington: Univ. of Vermont, Research Center for
Children, Youth, & Families; 2000.
[8] Lendt M, Helmstaedter C, Kuczaty S, Schramm J, Elger CE.
Behavioural disorders in children with epilepsy: early improvement
after surgery. J Neurol Neurosurg Psychiatry 2000;69:739–44.
[9] Döpfner M, Schmeck K, Berner W. Handbuch: Elternfragebogen
über das Verhalten von Kindern und Jugendlichen. For-
schungsergebnisse zur deutschen Fassung der Child Behavior Check-
list (CBCL/4-18). Cologne: Arbeitsgruppe Kinder-, Jugend- und
Familiendiagnostik (KJFD); 1994.
[10] Dekker MC, Koot HM, van der Ende J, Verhulst FC. Emotional and
behavioral problems in children and adolescents with and without
intellectual disability. J Child Psychol Psychiatry 2002;43:1087–98.
[11] Buelow JM, Austin JK, Perkins SM, Shen J, Dunn DW, Fastenau
PS. Behavior and mental health problems in children with epilepsy
and low IQ. Dev Med Child Neurol 2003;45:683–92.
[12] Dunn DW, Austin JK, Harezlak J, Ambrosius WT. ADHD and
epilepsy in childhood. Dev Med Child Neurol 2003;45:50–4.
[13] Davies S, Heyman I, Goodman R. A population survey of mental
health problems in children with epilepsy. Dev Med Child Neurol
2003;45:292–5.
[14] Williams J, Lange B, Phillips T, et al. The course of inattentive and
hyperactive–impulsive symptoms in children with new onset seizures.
Epilepsy Behav 2002;3:517–21.
[15] Austin JK, Harezlak J, Dunn DW, Huster GA, Rose DF, Ambrosius
WT. Behavior problems in children before first recognized seizures.
Pediatrics 2001;107:115–22.