Anda di halaman 1dari 9

Multiplehttp://msj.sagepub.

com/
Sclerosis Journal

Assessing the validity of a computer-generated cognitive screening instrument for patients with
multiple sclerosis
Helen Lapshin, Krista L Lanctôt, Paul O'Connor and Anthony Feinstein
Mult Scler published online 7 May 2013
DOI: 10.1177/1352458513488841

The online version of this article can be found at:


http://msj.sagepub.com/content/early/2013/05/07/1352458513488841

Published by:

http://www.sagepublications.com

Additional services and information for Multiple Sclerosis Journal can be found at:

Email Alerts: http://msj.sagepub.com/cgi/alerts

Subscriptions: http://msj.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

>> OnlineFirst Version of Record - May 7, 2013

What is This?

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


488841
2013
MSJ0010.1177/1352458513488841Multiple Sclerosis JournalLapshin et al.

MULTIPLE
SCLEROSIS MSJ
Research Paper JOURNAL

Multiple Sclerosis Journal

Assessing the validity of a computer- 0(0) 1­–8


© The Author(s) 2013
Reprints and permissions:
generated cognitive screening instrument sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1352458513488841
for patients with multiple sclerosis msj.sagepub.com

Helen Lapshin1,2, Krista L Lanctôt1,2, Paul O’Connor2,3


and Anthony Feinstein1,2

Abstract
Background: Neuropsychological testing requires considerable time, expense, and expertise to administer. These
factors can limit patient access. Computerized cognitive testing has been proposed as an alternative.
Objectives: The objective of this paper is to validate a brief, simple-to-use computer-generated cognitive assessment
screening battery for multiple sclerosis (MS) patients that has minimal motor involvement.
Methods: A sample of 96 MS patients and 98 healthy controls completed a computer-generated battery that included
the Stroop, Symbol Digit Modalities Test (C-SDMT), a two- and four-second visual analog of the Paced Auditory Serial
Addition Test (PVSAT-2, PVSAT-4), and simple and choice reaction time tests. The Minimal Assessment of Cognitive
Function in MS was used to define cognitive impairment in the MS sample.
Results: Each newly developed test successfully distinguished between cognitively impaired patients and healthy controls
as well as cognitively intact patients. A combination of three computerized tests (C-SDMT, PVSAT-2, PVSAT-4) with
a mean administration time of 10 minutes had a sensitivity of 82.5% and specificity of 87.5% in detecting cognitive
impairment. Good test-retest reliability was obtained for each measure.
Conclusions: Good sensitivity and specificity, brevity, ease of administration, and a limited motor component highlight
the feasibility of introducing this computer-generated cognitive screening instrument in a busy MS clinic.

Keywords
Cognition, multiple sclerosis, neuropsychological tests, cognitive impairment, computerized cognitive testing,
information processing speed
Date received: 14 November 2012; revised: 2 April 2013; accepted: 8 April 2013

Introduction
An estimated 40%–65% of multiple sclerosis (MS) patients was developed. Although it has good validity and reliability,
are cognitively impaired.1 Signs are more subtle than those it does require the presence of an informant and, like all self-
seen in dementia, but impaired patients may nevertheless report psychometric measures, subjectivity may confound
experience significant difficulties in work and social situa- results.
tions.2 The most common deficits are those relating to Another alternative to conventional testing is computer-
information processing speed and memory although prob- driven assessments. A number have been developed but
lems with executive function, visuospatial processing, and their limitations include a motor component that excludes
verbal fluency may also be found.3 more disabled patients, subject selection limited to relaps-
Given the functional implications of cognitive dysfunc- ing–remitting patients, and a choice of tests that do not
tion, detection is important. The most frequently employed
method involves tests administered by a neuropsychologist. 1Department of Psychiatry, Sunnybrook Health Sciences Centre, Canada.
Access to testing may, however, be hindered by factors such 2University
of Toronto, Canada.
as the absence of a neuropsychological service, cost and, in 3Neurology, St. Michael’s Hospital, Canada.

a busy clinical setting, the duration of current screening bat-


Corresponding author:
teries.4 To offset these difficulties, a self-report question- Helen Lapshin, Department of Psychiatry, Sunnybrook Health Sciences
naire for patients and informants, the Multiple Sclerosis Centre, 2075 Bayview Ave., Toronto, ON, M4N 3M5, Canada.
Neuropsychological Screening Questionnaire (MSNQ),5 Email: Helen.Lapshin@sunnybrook.ca

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


2 Multiple Sclerosis Journal 0(0)

reflect the nature of cognitive impairment in this popula- are administered using standard Windows software and
tion.6–11 The aim of the present study was to formulate and require no specific hardware. Instructions for each test are
validate a computerized protocol that was not subject to read aloud by the test administrator, who subsequently begins
these drawbacks. the tests. The administrator is present throughout the assess-
ment. Scores for all the tests, apart from a visual version of
the PASAT, are collected automatically by the computer and
Methods do not require input from the administrator. A description of
MS patients the tests, their administration, and the rationale for test design
are described in a supplemental file. The tests that make up
A sample of 108 patients were enrolled in the study. Twelve the computer-generated battery are as follows:
patients were excluded from the analysis because of one of
the following reasons: unconfirmed diagnosis of MS (N = The Stroop Color-Word Test: An adaptation of the clas-
2), learning disability (N = 2), marked dysarthria (N = 1), sic Stroop paradigm.14
previous head injury (N = 1), severe fatigue (N = 1), severe A Computerized Symbol Digit Modalities Test (C-SDMT):
physical disability (N = 3), visual acuity below 20/100 in Unlike the conventional SDMT,15 the computerized ver-
both eyes (N = 1), and stroke (N = 1). The final sample sion measures speed of response rather than the number of
consisted of 96 MS patients (ages 22 to 63 years) meeting responses. The time is determined automatically by the
the modified McDonald criteria12 for MS. No patient had computer.
dementia or psychosis. Patients were recruited from outpa- The Paced Visual Serial Addition Test (four-second and
tient MS clinics. Demographic and disease-related varia- two-second trials (PVSAT-4, PVSAT-2)): This is a visual
bles (i.e. course and duration of illness, and physical version of the conventional PASAT.16 Given that MS
disability according to Expanded Disability Status Scale patients have historically found the PASAT anxiety pro-
(EDSS))13 were collected. voking, which in turn can compromise performance,17
we hoped to offset this by starting the test with a four-
Healthy controls second, rather than the more conventional three-second,
version. Thereafter, subjects were administered the two-
Ninety-nine healthy controls, comparable in age, gender, second version.
education, and premorbid intelligence quotient (IQ) to the Simple Reaction Time (SRT): The aim of this test is to
MS patients, were recruited with flyers and online adver- measure the speed with which subjects react to a simple
tisements. One healthy control was excluded from the anal- stimulus.
ysis because of dyslexia. Choice Reaction Time (CRT): This test is similar to the
SRT except that it contains a measure of cognitive speed
Informed consent as well.

The study received approval from the ethics committees Choice Reaction Time minus Simple Reaction Time (CRT-
of both Sunnybrook Health Sciences Centre and St. SRT): This index was calculated to determine a measure of
Michael’s Hospital. Informed consent was obtained from cognitive speed. Six subjects (6.3%) of the sample could
all subjects. Patients and healthy controls were paid for not complete any of the reaction time tests because of motor
their participation. impairment.
Failure on each computerized test was defined as a score
Cognitive assessment more than 1.5 SDs below the mean of healthy control-
derived normative scores. No patients were incapable of
The cognitive assessment included two parts. The first was completing the computer-generated tests because of cogni-
a computer-generated collection of cognitive tests adminis- tive impairment.
tered to all participants. The second part was a conven-
tional, neuropsychological battery, the Minimal Assessment
of Cognitive Function in MS (MACFIMS),3 which only the Conventional cognitive testing
MS patients completed. Thirty patients and 19 healthy con- The MACFIMS is a battery of cognitive tests that has been
trols were retested with the computerized battery on aver- developed for use with MS patients by expert consensus.
age 71.7 (standard deviation (SD) = 26.2) days after the Each test has published normative data that are used for
initial assessment. scoring. The MACFIMS consists of seven tests that meas-
ure the following five cognitive domains:
Computer-generated cognitive testing
Speed of information processing: Paced Auditory Serial
The computer-generated battery comprises five tests of speed Addition Test (PASAT) – Trial 1: three second and Trial 2: two
of information processing and working memory. The tests second16 and the Symbol Digit Modalities Test (SDMT).15

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


Lapshin et al. 3

Table 1.  Demographic and disease characteristics of MS patients and healthy controls.

MS patients mean (SD)/ Healthy controls mean (SD)/ t test/x2 p


frequency (%) (N = 96) frequency (%) (N = 98)
Age 48.5 (9.8) 46.4 (9.0) t = −1.5 p = 0.124
Sex (% female) 65 (67.7%) 64 (65.3%) x2 = 0.1 p = 0.723
Premorbid IQ 113.2 (8.1) 114.1 (8.0) t = 0.8 p = 0.436
Years of education 14.8 (2.2) 15.1 (1.6) t = 1.1 p = 0.290
Illness duration (years) 11.3 (8.5)  
EDSS 4.1 (2.6)  
Disease course
RRMS 44 (45.8%)  
SPMS 37 (38.5%)  
PPMS 15 (15.6)  

MS: multiple sclerosis; IQ: intelligence quotient; EDSS: Expanded Disability Status Scale; RRMS: relapsing–remitting MS; SPMS: secondary progressive
MS; PPMS: primary progressive MS.

Verbal and visual memory: California Verbal Learning Test – of MS patients impaired on the MACFIMS to those who
II (CVLT-II)18 and the Brief Visuospatial Memory Test – were not impaired and healthy controls. A receiver-operat-
Revised (BVMT-R).19 ing characteristics (ROC) analysis was undertaken to assess
Executive function: Delis-Kaplan Executive Function System which threshold on the computer-generated battery gave the
(D-KEFS) Sorting Test.20 best yield with respect to cognitive impairment as defined
by MACFIMS global impairment. Pearson correlations
Spatial processing: Judgment of Line Orientation (JLO).21 were conducted to assess the relationship between the com-
Verbal fluency: Controlled Oral Word Association Test puterized tests in the final battery and their conventional
(COWAT).22 counterparts. Finally, a logistic regression analysis was
completed to explore the potential influence of factors such
By convention failure on each test was defined as a score of as anxiety and depression on the computerized screen.
more than 1.5 SDs below the mean normative values.23
Global impairment on the MACFIMS was defined as
Results
impairment on two or more cognitive measures.
Seventy-one MS subjects were administered the com- Demographic data
puter-generated battery first for the following reasons: 1)
Demographic comparisons between the MS patients and
To offset the potentially deleterious effects of fatigue on the
healthy control subjects are shown in Table 1 together with
timed computerized tests, given the greater length of the
the neurological data.
MACFIMS (90 vs. 20 minutes). 2) To minimize practice
effects on the C-SDMT and PVSAT arising from their ana-
logs in the MACFIMS. 3). To take into account that healthy Cognitive data
control subjects completed the computerized battery only
Comparisons between the MS patients and control subjects
and as such their scores were not influenced by the
on the computerized cognitive tests are shown in Table 2
MACFIMS analogs.
and reveal significant differences for all the indices. After
On the other hand, the C-SDMT and PVSAT could have
dichotomizing each computerized test (normal versus
influenced scores on the conventional SDMT and PASAT
impaired) relative to the healthy control data, the C-SDMT
components to the MACFIMS. To address this, 25 patients
emerged as the most frequently impaired index (34.4% of
were administered the MACFIMS first.
subjects) followed by the PVSAT-4 (24%), CRT (24%),
Finally, all subjects completed the American National
PVSAT-2 (21.9%), CRT-SRT (21.1%), SRT (16.7%), and
Adult Reading Test (ANART)24 to determine premorbid IQ
Stroop (10.4%).
and the Hospital Anxiety and Depression Scale (HADS) to
Based on the MACFIMS determined threshold, 40
ascertain depression and anxiety.25
(41.7%) MS patients were deemed cognitively impaired.
The results of a three-way comparison of the computerized
cognitive data are shown in Table 3. They revealed signifi-
Statistical analysis
cant differences in every computer-generated test between
One-way analyses of variance (ANOVAs) and Kruskal- the cognitively impaired MS patients on the one hand, and
Wallis tests were used to compare computerized test results the cognitively intact patients and healthy controls on the

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


4 Multiple Sclerosis Journal 0(0)

Table 2.  Comparison of computer-generated test scores for MS patients and healthy controls.

Computerized test Healthy controls (N=98) MS patients (N = 96) t test p Cohen’s d


Stroop (seconds) 26.6 (8.5) 30.0 (9.4) t = 2.6 0.009 .37
C-SDMT (seconds) 12.7 (2.6) 16.0 (4.6) t = 6.2 < 0.001 .89
PVSAT-4 (correct responses) 28.6 (2.2) 26.2 (5.1) t = 4.1 < 0.001 .61
PVSAT-2 (correct responses) 24.5 (5.2) 21.0 (6.5) t = 4.1 < 0.001 .59
SRT (milliseconds) 393.6 (118.0) 443.3 (141.8) t = 2.6 0.008 .37
CRT (milliseconds) 444.0 (88.8) 541.1 (161.3) t = 5.1 < 0.001 .73
CRT-SRT (milliseconds) 52.5 (86.2) 99.2 (91.1) t = 3.6 <0.001 .52

MS: multiple sclerosis; C-SDMT: Computerized Symbol Digit Modalities Test; PVSAT: Paced Visual Serial Addition Test; SRT: Simple Reaction Time; CRT:
Choice Reaction Time; CRT-SRT: Choice Reaction Time minus Simple Reaction Time.

other. The index CRT-SRT did not distinguish cognitively the outcome variable was impairment on the computer-
intact from cognitively impaired patients, although both the generated screen. The following predictor variables were
CRT and the CRT-SRT showed more impairment in cogni- included in the model: age, gender, EDSS, education, ill-
tively intact patients than healthy controls. ness duration, HADS depression, HADS anxiety, and
ANART. Only the ANART emerged as a significant predic-
tor of cognitive impairment.
Validation
To ascertain which combination of computer-generated
Reliability
tests had the best sensitivity and specificity in predicting
cognitive impairment relative to the MACFIMS, we Test-retest data on a subset of patients (n = 30) and healthy
adopted two approaches. The first looked at the predictive controls (n = 19) revealed significant intra-class correlation
ability of the entire computer-generated battery according coefficients (ICC) for each computer-generated test:
to a threshold determined by the number of tests failed, i.e. C-SDMT (ICC= 0.927, p < 0.001); PVSAT-4 (four sec-
one of seven; two of seven; three of seven, and so on. The onds) (ICC = 0.483, p < 0.001), PVSAT-2 (two seconds)
results are shown in Table 4(a) and revealed that an impair- (ICC = 0.747, p < 0.001), CRT (ICC = 0.849, p < 0.001),
ment threshold of one of seven yielded the highest sensitiv- SRT (ICC = 0.945, p < 0.001), Stroop (ICC = 0.814, p <
ity and specificity combination of 85.5% and 70.4%, 0.001).
respectively.
The second approach was based on individual test selec-
PVSAT vs. PASAT
tion. Here we started with the test most often impaired, the
C-SDMT, to which we added the next most sensitive index, The PVSAT-4 and PVSAT-2 were replaced with the
the PVSAT-4, followed sequentially by the remaining tests PASAT-3 and PASAT-2 to assess whether this improved the
in decreasing order of sensitivity. The results are shown in sensitivity and specificity of the screening battery. It did
Table 4(b) and show that the C-SDMT together with the not, yielding a result of 70.9% and 94.6%, respectively.
PVSAT-4 and PVSAT-2 offered the best combined sensitiv-
ity (82.5%) and specificity (87.5%). A ROC analysis,
Order of testing comparisons
shown in Figure 1, revealed an area under the curve of
0.864, p < 0.001. The order of administration had no effect on the cognitive
results: There was no significant difference in global
impairment on the MACFIMS (x2 = 1.485, p = 0.223), or
Correlations between overlapping tests from the computer-generated screen (x2 = 0.005, p = 0.941)
MACFIMS and computer-generated battery between patients who completed either the computer-gen-
The number of correct responses for the PVSAT-4 corre- erated battery or the MACFIMS first.
lated robustly with scores from the PASAT-3 (r = 0.73, p <
0.001), while a similar result was obtained for the PVSAT-2
and PASAT-2 (r = 0.76, p < 0.001). Performance on the Discussion
C-SDMT correlated significantly with the conventional Our data show that three tests that are administered via
SDMT scores (r = −0.86, p < 0.001). computer with technician supervision, namely the C-SDMT
and two- and four-second PVSAT, have good sensitivity
and specificity in determining the presence of cognitive
Predictors of cognitive impairment
dysfunction as defined by a conventional neuropsychologi-
To explore the effects of anxiety and depression on cogni- cal battery. Good criterion validity was also demonstrated
tion, a logistic regression analysis was undertaken in which as each test was able to differentiate MS patients from a

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


Lapshin et al. 5

Table 4(a).  Sensitivity and specificity table for computer-


generated battery when compared to MACFIMS.

intact vs. healthy

MS: multiple sclerosis; C-SDMT: Computerized Symbol Digit Modalities Test; PVSAT: Paced Visual Serial Addition Test; SRT: Simple Reaction Time; CRT: Choice Reaction Time; CRT-SRT: Choice Reaction
Cognitively Impairment threshold: Sensitivity Specificity

p = 0.973
p = 0.064
p = 1.000

p = 0.995

p = 0.789
p = 0.031

p < 0.001
p < 0.001
0.029
controls
Impaired on at least
1/7 85.7% 70.4%
2/7 60.0% 87.0%
Cognitively impaired

3/7 48.6% 94.4%


vs.healthy controls

4/7 31.4% 96.3%


5/7 17.1% 98.1%
6/7 11.4% 98.1%
p < 0.001
p < 0.001
p < 0.001

p < 0.001

p < 0.001
p < 0.001
0.003
< 0.001
0.003
7/7 2.9% 100.0%

MS: multiple sclerosis; MACFIMS: Minimal Assessment of Cognitive Func-


tion in MS.
cognitively intact

Table 4(b).  Sensitivity and specificity table for computer-


impaired vs.
Cognitively

generated tests when compared to MACFIMS.


<0.001 p < 0.001
<0.001 p < 0.001
<0.001 p < 0.001

<0.001 p < 0.001

<0.001 p = 0.004
<0.001 p < 0.001
0.571
0.686
0.960

Test(s) Impairment Sensitivity Specificity


threshold
<0.001
<0.001
<0.001

C-SDMT 1/1 65.0% 87.5%


C-SDMT, 1/2 80.0% 87.5%
p

PVSAT-4
F = 17.0
F = 50.7
F = 45.4

F = 42.4

F = 22.7

F = 32.9
F = 10.0

C-SDMT, CRT 1/2 72.5% 78.6%


F = 8.9

F = 6.9

C-SDMT, 1/3 80.0% 78.6%


F/x2

PVSAT-4, CRT
C-SDMT, 1/4 82.5% 78.6%
Cognitively impaired MS Cognitively intact MS Healthy controls

PVSAT-4, CRT,
mean (SD) (N

393.6 (118.0)
444.0 (88.8)
52.5 (86.2)

PVSAT-2
26.6 (8.5)
12.7 (2.6)
28.6 (2.2)

24.5 (5.2)

2.4 (3.0)
4.6 (3.1)

C-SDMT, 1/3 82.5% 87.5%


PVSAT-4,
= 98)

PVSAT-2
Table 3.  Computer-generated test scores for MS patients and healthy controls.

C-SDMT, 1/4 86.1% 77.8%


PVSAT-4,
patients mean (SD)

PVSAT-2,
Time minus Simple Reaction Time; HADS: Hospital Anxiety and Depression Scale.

CRT-SRT
407.7 (100.5)
497.6 (128.5)
91.5 (98.6)

C-SDMT, 1/4 82.5% 80.4%


26.2 (6.7)
13.9 (2.3)
28.6 (2.7)

24.4 (4.8)

6.7 (3.6)
7.4 (3.9)
(N = 56)

PVSAT-4,
PVSAT-2, SRT
C-SDMT, 1/4 82.5% 85.7%
PVSAT-4,
PVSAT-2, Stroop
patients mean (SD)

MACFIMS: Minimal Assessment of Cognitive Function in Multiple


Sclerosis; C-SDMT: Computerized Symbol Digit Modalities Test; PVSAT:
496.7 (175.8)
607.5 (184.2)
35.2 (10.3)

CRT-SRT (milliseconds) 110.8 (78.4)

Paced Visual Serial Addition Test; CRT: Choice Reaction Time Test; SRT:
18.9 (5.4)
22.7 (5.8)

15.8 (5.3)

7.4 (4.4)
7.2 (5.1)

Simple Reaction Time Test; CRT-SRT: Choice Reaction Time minus Simple
(N = 40)

Reaction Time.

group of 98 healthy control subjects. In addition, the ability


of premorbid intellect, i.e. the ANART, to predict test per-
formance fits well with current theories relating to cogni-
CRT (milliseconds)
C-SDMT (seconds)

SRT (milliseconds)

HADS Depression
PVSAT-4 (correct

PVSAT-2 (correct
Stroop (seconds)

tive reserve as a determinant of cognitive functioning in


Computerized test

HADS Anxiety

MS patients.26 Our psychometric data should be viewed


within the context of an expanding literature devoted to the
responses)

responses)

development of computerized cognitive batteries for use


with MS patients.6–11 Our study does, however, differ from
those published to date in some important ways.

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


6 Multiple Sclerosis Journal 0(0)

published reports of such an approach involving MS


patients. An intermediate approach is computer adminis-
tration of test stimuli with the presence of a tester. Our
battery, administered by a Master’s student, falls into this
category. While having a fully automated, nonadministra-
tor-dependent battery available for use with MS subjects
would be preferable from a logistics perspective, the obvi-
ous challenge is to avoid the potentially negating effects of
having a cognitively impaired patient in the “driver seat”
compromise the testing process.
In validating our computerized tests we used a cognitive
battery for comparison that was developed specifically for
MS patients. The MACFIMS comprises tests known to be
sensitive to the array of cognitive deficits present in MS
patients. The fact that our brief battery performed well in
relation to this enhances its validity, but is not unexpected.
The three tests, i.e. the C-SDMT and the two PVSATs, are
to varying degrees part of the MACFIMS. While the
method of assessment in the C-SDMT is quite different
from the SDMT as administered in the MACFIMS, the
principles underlying the two are the same. Although the
Figure 1. ROC curve: Computer-generated battery compared SDMT measures the number of correct responses in 90 sec-
to MACFIMS. onds and the C-SDMT measures the average time to com-
ROC: receiver-operating characteristics; MACFIMS: Minimal Assessment plete a set of symbols, both are tests of speed of information
of Cognitive Function in Multiple Sclerosis; ROC curve of the computer- processing. Notwithstanding the different methodologies,
generated screen (Computerized Symbol Digit Modalities Test, Paced
Visual Serial Addition Test 4 second, Paced Serial Visual Addition Test 2
the robust correlations between the computerized and con-
second) compared to MACFIMS impairment. ventional tests speak to their shared construct validity.
Recent research suggests the SDMT is one of the most
sensitive of all cognitive tasks23 in this population and this
First, our sample is representative of the three main finding emerged too in our computerized battery, where
disease types that comprise clinically confirmed MS. The more than a third of patients were found to be impaired.
percentages of patients with relapsing–remitting, second- The PASAT, recently supplanted by the SDMT as the cog-
ary progressive, and primary progressive disease match nitive test of choice in detecting deficits in MS patients,30
community prevalence rates,27–29 suggesting that our test- has nevertheless retained its position as the single cognitive
ing procedure is valid for the progressive forms of the dis- index incorporated into the Multiple Sclerosis Functional
ease as well. Many earlier studies focused exclusively on Composite.31 That test has, however, been criticized as too
relapsing–remitting patients, essentially to exclude those difficult and anxiety provoking for some MS patients,3
with more marked disability.6,7,11 In a large measure this which may explain why the visual version, which is slower
selection bias was introduced because some of the com- and easier to perform, emerged as a more useful screening
puterized tests demanded a degree of motor dexterity that index when placed alongside the C-SDMT.
placed them beyond the functional limits of the more disa- More recently a group of MS behavioral researchers
bled subjects. In developing our battery we were cogni- devised the Brief International Cognitive Assessment for
zant of this and selected our tests accordingly. Of note is Multiple Sclerosis (BICAMS), a triad of tests comprising
that the three tests that were ultimately selected for the the SDMT and two tests of verbal and visuospatial mem-
final screening tool have a limited motor component, the ory.32,33 While addressing some of the concerns that have
oral response. limited routine administration of cognitive testing thus
When it comes to the administration of cognitive tests, far, data are still awaited on sensitivity and specificity
there is a widening array of approaches. The most frequent and instant, automated scoring is not available for all the
method involves patients being tested by neuropsycholo- indices.
gists or trained psychometricians working for a neuropsy- The proposed semi-automated battery is not without
chologist. By introducing computer-based tests into the limitations. MS patients with significant visual impairment
process, this reliance on neuropsychological expertise has would be excluded from testing, although increasing the
been loosened. At one end of this evolving spectrum are font size could help to some extent. Further work on this is
fully automated tests that potentially would require no required before any firm conclusions can be reached.
supervision by an administrator. To date, there are few Marked dysarthria is another impediment although it is

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


Lapshin et al. 7

worth noting that this also applies to conventional versions Cognosci Inc, Wyeth, Daiichi Sankyo, and Roche; and
of the SDMT and PASAT. The absence of a measure of epi- serves as the National Scientific and Clinical Advisor to
sodic memory is also a likely weakness. Here our decision the Multiple Sclerosis Society of Canada.
to omit was based on a desire to limit administration time. Dr Feinstein has served on scientific advisory boards for
As yet, parallel versions of the tests necessary to reduce Merck Serono and Avanir Pharmaceuticals; has received
practice effects are not available. Efforts are, however, speaker honoraria from Merck Serono, Teva Pharmaceutical
under way to develop and validate these. Finally, the pos- Industries Ltd., Bayer Schering Pharma, and Biogen Idec;
sibility of tester bias cannot be ruled out because one tester serves on the editorial boards of Multiple Sclerosis and the
administered both the computer-based battery and the African Journal of Psychiatry; receives publishing royal-
MACFIMS. If present, however, it was mitigated by the ties for The Clinical Neuropsychiatry of Multiple Sclerosis
fact that scores from the computerized battery were not (Cambridge University Press, 2007); chairs the Medical
accessible to the tester until the full test session was over, Advisory Committee for the Multiple Sclerosis Society of
PVSAT apart. Even here what constituted a pass or fail was Canada; conducts neuropsychiatric evaluation, cognitive
not known at the time of testing. testing, brain imaging in neuropsychiatry in his clinical
In advocating for an assessment approach as described practice; and receives research support from the Canadian
above, it is important to ensure that the results are conveyed Institute of Health Research, the Multiple Sclerosis Society
to patients in a sensitive manner with the aim of improving of Canada, and Teva Pharmaceutical Industries Ltd.
clinical care. It is also necessary to reiterate that this battery
is only a screen and not a comprehensive neuropsychologi- Funding
cal evaluation. That said, the results do carry weight and it This study was funded by the Multiple Sclerosis Society of
is advisable that they be given to patients by neurologists or Canada (grant number 256).
allied healthcare providers knowledgeable about cognitive
functioning in MS and how this can be addressed. Here References
options range from the introduction of simple, yet effective, 1. Rao SM, Leo GJ, Bernardin L, et al. Cognitive dysfunction
compensatory strategies34 to, wherever possible, referrals in multiple sclerosis: I. Frequency, patterns, and prediction.
to specialist services providing remedial cognitive rehabili- Neurology 1991; 41: 685–691.
tation. In relation to the latter, recent findings suggest the 2. Rao SM, Leo GJ, Ellington L, et al. Cognitive dysfunction in
potential for promising benefits.35 multiple sclerosis. II. Impact on employment and social func-
tioning. Neurology 1991; 41: 692–696.
Acknowledgement 3. Benedict RH, Fischer JS, Archibald CJ, et al. Minimal neu-
ropsychological assessment of MS patients: A consensus
The authors thank Liesly Lee, MD (Sunnybrook Health Sciences
approach. Clin Neuropsychol 2002; 16: 381–397.
Centre), Kathleen Carr, RN (Sunnybrook Health Sciences Centre),
4. Rao S. A manual for the Brief Repeatable Battery of Neuro-
and Sheryl Clarke, RN, BSN, M.Ed. (St. Michael’s Hospital) for
psychological Tests in Multiple Sclerosis. New York, NY:
assisting with recruitment, Elizabeth Waknine for assisting with
National Multiple Sclerosis Society, 1990.
data collection, and Dr. Alex Kiss (Sunnybrook Health Sciences
5. Benedict RH, Munschauer F, Linn R, et al. Screening for mul-
Centre) for assisting with data analysis.
tiple sclerosis cognitive impairment using a self-administered
15-item questionnaire. Mult Scler 2003; 9: 95–101.
Conflicts of interest 6. Wilken JA, Kane R, Sullivan CL, et al. The utility of comput-
H. Lapshin has received support from an Ontario Graduate erized neuropsychological assessment of cognitive dysfunction
Scholarship grant. in patients with relapsing–remitting multiple sclerosis. Mult
Scler 2003; 9: 119–127.
Dr Lanctôt has received research support and/or speak- 7. Achiron A, Doniger GM, Harel Y, et al. Prolonged response
er’s honoraria from Abbott Laboratories, Lundbeck Canada times characterize cognitive performance in multiple sclero-
Inc, Pfizer Canada Inc, Janssen Ortho, MedImmune, and sis. Eur J Neurol 2007; 14: 1102–1108.
Wyeth. 8. De Sonneville LMJ, Boringa JB, Reuling IEW, et al. Informa-
Dr O’Connor serves on scientific advisory boards for tion processing characteristics in subtypes of multiple sclero-
Novartis, Sanofi-Aventis, Bayer Schering Pharma, sis. Neuropsychologia 2002; 40: 1751–1765.
Genentech Inc, and Roche; has received speaker hono- 9. Lazeron RH, de Sonneville LMJ, Scheltens P, et al. Cognitive
raria from Biogen Idec, Teva Pharmaceutical Industries slowing in multiple sclerosis is strongly associated with brain
volume reduction. Mult Scler 2006; 12: 760–768.
Ltd., Novartis, and Sanofi-Aventis; has served as a con-
10. Younes M, Hill J, Quinless J, et al. Internet-based cog-
sultant for Biogen Idec, Actelion Pharmaceuticals Ltd, nitive testing in multiple sclerosis. Mult Scler 2007; 13:
Bayer Schering Pharma, EMD Serono Inc, Teva 1011–1019.
Pharmaceutical Industries Ltd., Genentech Inc, and 11. Edgar C, Jongen PJ, Sanders E, et al. Cognitive performance
Warburg Pincus; has received research support from in relapsing remitting multiple sclerosis: A longitudinal study
Abbott, Bayer Schering Pharma, Novartis, BioMS in daily practice using a brief computerized cognitive battery.
Medical, Sanofi-Aventis, CIS Pharma, Genmab A/S, BMC Neurol 2011; 11: 68.

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013


8 Multiple Sclerosis Journal 0(0)

12. Polman CH, Reingold SC, Edan GE, et al. Diagnostic criteria: 25. Zigmond AS and Snaith RP. The Hospital Anxiety

2005 revisions to the “McDonald Criteria”. Ann Neurol 2005; and Depression Scale. Acta Psychiatr Scand 1983; 67:
58: 840–846. 361–370.
13. Kurtzke JF. Rating neurologic impairment in multiple scle- 26. Sumowski JF, Chiaravalloti N and DeLuca J. Cognitive

rosis: An expanded disability status scale (EDSS). Neurology reserve protects against cognitive dysfunction in multiple
1983; 33: 1444–1452. sclerosis. J Clin Exp Neuropsychol 2009; 31: 913–926.
14. Stroop JR. Studies of interference in serial verbal reactions. J 27. Noseworthy JH, Lucchinetti C, Rodriguez M, et al. Multiple
Exp Psychol 1935; 18: 643–662. sclerosis. N Engl J Med 2000; 343: 938–952.
15. Smith A. Symbol Digit Modalities Test: Manual. Los Ange- 28. Miller DH and Leary SM. Primary-progressive multiple scle-
les: Western Psychological Services, 1982. rosis. Lancet Neurol 2007; 6: 903–912.
16. Gronwall DM. Paced auditory serial addition task: A measure 29. Tremlett H, Paty D and Devonshire V. Disability progression
of recovery from concussion. Percept Mot Skills 1977; 44: in multiple sclerosis is slower than previously reported. Neu-
367–373. rology 2006; 66: 172–177.
17. Aupperle RL, Beatty WW, Shelton Fde N, et al. Three screen- 30. Drake AS, Weinstock-Guttman B, Morrow SA, et al. Psycho-
ing batteries to detect cognitive impairment in multiple scle- metrics and normative data for the Multiple Sclerosis Func-
rosis. Mult Scler 2002; 8: 382–389. tional Composite: Replacing the PASAT with the Symbol
18. Delis DC, Kramer JH, Kaplan E, et al. California Verbal Digit Modalities Test. Mult Scler 2010; 16: 228–237.
Learning Test manual: Second edition, adult version. San 31. Fischer JS, Rudick RA, Cutter GR, et al. The Multiple Scle-
Antonio, TX: Psychological Corporation, 2000.
rosis Functional Composite measure (MSFC): An integrated
19. Benedict RHB. Brief Visuospatial Memory Test – Revised:
approach to MS clinical outcome assessment. National MS
Professional manual. Odessa, FL: Psychological Assessment
Society Clinical Outcomes Assessment Task Force. Mult
Resources, 1997.
Scler 1999; 5: 244–250.
20. Delis DC, Kaplan E and Kramer JH. California Sorting Test.
San Antonio, TX: Psychological Corporation, 2001. 32. Langdon DW, Amato MP, Boringa J, et al. Recommendations
21. Benton AL, Sivan AB, Hamsher K, et al. Contributions to for a Brief International Cognitive Assessment for Multiple
neuropsychological assessment, 2nd ed. New York: Oxford Sclerosis (BICAMS). Mult Scler 2012; 18: 891–898.
University Press, 1994. 33. Benedict R, Amato MP, Boringa J, et al. Brief International
22. Benton AL and Hamsher K. Multilingual Aphasia Examina- Cognitive Assessment for MS (BICAMS): International stan-
tion. Iowa City, IA: AJA Associates, 1989. dards for validation. BMC Neurol 2012; 12: 55.
23. Benedict RH, Cookfair D, Gavett R, et al. Validity of 34. Shevil E and Finlayson M.. Pilot study of a cognitive interven-
the minimal assessment of cognitive function in multiple tion program for persons with multiple sclerosis. Health Educ
sclerosis (MACFIMS). J Int Neuropsychol Soc 2006; 12: Res 2010; 25: 41–53.
549–558. 35. Goverover Y, Basso M, Wood H, et al. Examining the ben-
24. Grober E and Sliwinski M. Development and validation of efits of combining two learning strategies on recall of func-
a model for estimating premorbid verbal intelligence in the tional information in persons with multiple sclerosis. Mult
elderly. J Clin Exp Neuropsychol 1991; 13: 933–949. Scler 2011; 17: 1488–1497.

Downloaded from msj.sagepub.com at UNIVERSITAS GADJAH MADA on June 12, 2013

Anda mungkin juga menyukai