Multiple Sclerosis http://msj.sagepub.com/
Validity of an Internet version of the Multiple Sclerosis Neuropsychological Questionnaire Nadine Akbar, Kimia Honarmand, Nancy Kou, Brian Levine, Neil Rector and Anthony Feinstein Mult Scler 2010 16: 1500 originally published online 2 September 2010 DOI: 10.1177/1352458510379615 The online version of this article can be found at: http://msj.sagepub.com/content/16/12/1500
Published by: http://www.sagepublications.com
Additional services and information for Multiple Sclerosis can be found at: Email Alerts: http://msj.sagepub.com/cgi/alerts Subscriptions: http://msj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://msj.sagepub.com/content/16/12/1500.refs.html
>> Version of Record - Nov 30, 2010 OnlineFirst Version of Record - Sep 2, 2010 What is This?
Downloaded from msj.sagepub.com by guest on October 11, 2013
Research Paper
Validity of an Internet version of the Multiple Sclerosis Neuropsychological Questionnaire
Multiple Sclerosis 16(12) 1500–1506 ! The Author(s) 2010 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1352458510379615 msj.sagepub.com
Nadine Akbar1,2, Kimia Honarmand1,2, Nancy Kou1, Brian Levine2,3, Neil Rector1,2 and Anthony Feinstein1,2 Abstract Background: Neuropsychological batteries are long and require expertise to administer. For this reason, the Multiple Sclerosis Neuropsychological Questionnaire (MSNQ) was developed as it is quick and easy to complete. The informant version of the scale has proven to be a useful screen for cognitive impairment in multiple sclerosis (MS). Objective: The objective was to validate an Internet version of the MSNQ. Methods: The following psychometric data were collected at home over the Internet in 82 MS patients: (a) patient selfreport version MSNQ (P-MSNQ), (b) informant version MSNQ (I-MSNQ), and (c) Center for Epidemiological Studies Depression Scale (CES-D). Thereafter patients underwent in-office testing with the Brief Repeatable Battery of Neuropsychological Tests (BRB-N). The sensitivity and specificity of the Internet MSNQ to detect cognitive impairment relative to the BRB-N was determined using receiver operating characteristic (ROC) curve analysis. Results: Thirty-five percent of the sample was cognitively impaired. The P-MSNQ was correlated with depression and two tests of the BRB-N. The I-MSNQ was correlated with depression and all five tests of the BRB-N. A cut-off score of 26 on the I-MSNQ gave a sensitivity and specificity of 72% and 60% respectively. Test-retest and internal reliability analyses were strong for both the P-MSNQ and I-MSNQ. Conclusion: This is the first attempt at an Internet validation of the MSNQ. The modest sensitivity and specificity values suggest that further research is needed before either the patient or informant version of the MSNQ can be used for neuropsychological screening purposes over the Internet. Keywords multiple sclerosis, cognition, Internet, Multiple Sclerosis Neuropsychological Questionnaire, MSNQ Date received: 26th April 2010; revised: 18th June 2010; accepted: 30th June 2010
Introduction Cognitive dysfunction is recognized as one of the common behavioral sequelae associated with multiple sclerosis (MS), with the domains of information processing speed and episodic memory most often affected.1–3 Cognitive impairment has a negative impact on quality of life,4 employment,5 overall functional status,6 and rehabilitation outcome.7 Given these ramifications, accurately detecting cognitive impairment plays an important part in patient management. However, cognitive dysfunction in many MS patients goes undetected for a host of reasons. The number of patients with MS often exceeds neuropsychological resources. For example, Canada, which has an estimated 70,000 MS patients, has fewer
than 300 neuropsychologists of whom only a handful work with MS patients.8 There is therefore a major clinical need that cannot be met. A second barrier to testing is the referral process. Many MS patients will be referred for cognitive testing by a neurologist, but not all 1 Department of Psychiatry, Sunnybrook Health Sciences Centre, Toronto, Canada. 2 University of Toronto, Toronto, Canada. 3 Rotman Research Institute, Baycrest Centre for Geriatric Care, Toronto, Canada.
Corresponding author: Nadine Akbar, Sunnybrook Health Sciences Centre, Department of Psychiatry, Room FG08, 2075 Bayview Avenue, Toronto, Ontario, Canada M4N 3M5 Email:
[email protected]
Akbar et al. MS patients have equal access to a neurologist. Between one-quarter and one-third of MS patients are not able to see a neurologist on a yearly basis. These patients are more likely to be poor, live in rural areas, belong to minority groups, have an illness of longer duration (i.e. more than 15 years), have difficulty walking, rely on an assistive device such as a wheelchair/scooter, and be bedridden.9 A third impediment to cognitive testing is cost. Should patients not have access to a state or province funded assessment, the cost of private neuropsychological testing is often beyond the financial means of a patient group with a high unemployment rate.10 With neuropsychological resources stretched they need to be used judiciously. One way to do this is to introduce a triage system that can identify patients most at risk for cognitive impairment. Typical cognitive screening batteries, however, are still time consuming and require a particular expertise to administer. One of the most widely used is the 4-item Neuropsychological Screening Battery for MS (NSBMS) which takes 20–30 min to complete.1 An analog of this screening battery has been developed for clinical trials, namely the Brief Repeatable Battery of Neuropsychological Tests (BRB-N) which includes the Symbol Digit Modalities Test (SDMT) and 15 alternate versions for serial administration.11 While both these scales are widely used for research purposes, their length and complexity of administration have prevented their widespread clinical use. These limitations do not apply to the Multiple Sclerosis Neuropsychological Questionnaire (MSNQ)12 which is quick, reliable, inexpensive, easy to use, and not dependent on the expertise of a neuropsychologist to administer. The MSNQ is a 15-item self-report measure of perceived neuropsychological impairment. It lists common MS-related cognitive complaints that are rated according to their frequency and severity. It comprises two forms, the first completed by the patient (P-MSNQ) and the second by an informant who is well acquainted with the patient (I-MSNQ). Four studies have been published thus far, three of which reported promising results with the I-MSNQ, with sensitivities ranging from 83% to 91%, and specificities ranging from 80% to 97%.12–14 Validation of an Internet version would expand its reach even further, hence the aims of this study. While not meant to replace neuropsychological testing, a validated Internet version of the MSNQ could potentially allow many more MS patients to be screened for cognitive impairment.
1501 Patients were recruited from an outpatient MS clinic and through advertisements. Patients were not paid to participate, but were given feedback on their cognitive test results upon request. Exclusion criteria included age greater than 65 years, any co-morbid neurological disease, drug/alcohol abuse, history of learning disability, traumatic brain injury with loss of consciousness, concurrent physical disease potentially affecting the central nervous system, severe visual problems that would preclude testing, and neuropsychological testing within the past year in order to minimize practice effects. A present or past history of depression or anxiety did not exclude patients from participation.
Demographic and neurological data Demographic (age, education, employment, marital status) and neurological data (Expanded Disability Status Scale [EDSS] score, duration of MS, disease course, and use of any disease-modifying drugs) were collected prior to neuropsychological testing. The EDSS scores were obtained within a month of cognitive testing.
Online cognitive assessment A confidential, password-protected website was established for the study. Instructions for completing the online assessment were given via email or phone call with each patient provided with a unique study ID and password for login. The online assessment comprised the following: (a) the self-report P-MSNQ, (b) the I-MSNQ, and (c) the Centre for Epidemiological Studies Depression Scale (CES-D).16 Patients and their informants were asked to complete their sections separately. Informants had to have known the patient for longer than a year in order to participate. A randomly selected subset of patients with their same informants (n ¼ 23) were asked to complete the online assessment again after a mean duration of 61 days (SD ¼ 43) in order to evaluate test-retest reliability. No patient experienced clinical relapses in the interim time between the two administrations. The time to complete each questionnaire was measurable based on Internet data revealing the time at which subjects answered their first and last questions. These times were recorded. Any subject who took longer than 45 minutes to complete a questionnaire was excluded from the analysis (n ¼ 2).
Neuropsychological testing Methods Patient selection Eighty-four MS patients meeting the modified McDonald criteria15 were enrolled in this study.
All subjects had a neuropsychological assessment with the BRB-N11 within a week after the online assessment. This consisted of: (a) the Selective Reminding Test (SRT),17 (b) the 10/36 Spatial Recall Test (10/36 SPART),18 (c) the SDMT,19 (d) the Paced Auditory
1502 Serial Addition Test (PASAT),20 and (e) the Controlled Oral Word Association Test (COWAT).21 In addition, an estimate of premorbid intelligence quotient (IQ) was obtained with the American National Adult Reading Test (ANART).22 Cognitive impairment was defined as a performance of 1.5 SDs below the healthy control group scores on two or more of five BRB-N variables. These included the SRT consistent long-term retrieval (CLTR) score, the 10/36 SPART total recall score, the total number of correct responses on the SDMT, the total number of correct responses at the 2-s presentation rate of the PASAT, and the total correct words named on the COWAT. The normative cognitive data were derived from a healthy group of 35 subjects carefully screened for the absence of neurological or psychiatric disease. The healthy control subjects were matched to the MS group on age, years of education, gender, and pre-morbid IQ.
Data analysis Pearson’s correlations were computed between the Internet MSNQ, Internet CES-D, and each of the BRB-N variables. Significance was set at p < 0.05. Thereafter, the sample was divided into those subjects with and without cognitive impairment based on the results of the BRB-N. For every possible cut-off score on the MSNQ a 2 2 contingency table of MSNQ responses versus impairment on the BRB-N (yes/no) was drawn. From this, the sensitivity and specificity of the various cut-off scores were calculated. This in turn generated true positive (sensitivity) and false positive (1-specificity) rates. Based on these scores a receiver operator characteristic (ROC) curve was obtained with the area under the curve reflecting the overall performance of an instrument in discriminating between those with and those without cognitive impairment. By definition, an area greater than 0.5 is considered indicative of good discrimination.23 The test-retest reliability of the MSNQ was assessed using intraclass correlation coefficients (ICCs). Cronbach’s alpha coefficients were also obtained to provide measures of internal consistency of the P-MSNQ and the I-MSNQ.
Ethics This study received ethics approval from the Research Ethics Board at Sunnybrook Health Sciences Centre, affiliated with the University of Toronto. All patients gave their written and informed consent to participate in the study.
Multiple Sclerosis 16(12)
Results Patient demographic and illness data The mean age of the sample was 44.5 years (SD ¼ 8.9). Sixty-four (78%) subjects were female, 51 (62%) were married, and 35 (43%) were currently employed. The mean number of years of education was 15 (SD ¼ 2.2). The mean duration since diagnosis was 114.5 months (SD ¼ 89.1). Fifty-one (62%) subjects had relapsing– remitting MS (RRMS), 17 (21%) had secondary progressive MS (SPMS), and six (7%) had primary progressive MS (PPMS). Thirty-four (42%) subjects were on disease-modifying treatment. The median EDSS was 2.0 (Range ¼ 0–8.5) and the mean ANART verbal IQ was 113.2 (SD ¼ 8.0).
Control sample. The mean age of the healthy control group was 40.5 (SD ¼ 12.1), mean years of education was 15.9 (SD ¼ 2.0), the mean ANART verbal IQ was 112.0 (SD ¼ 9.4), and 26 (74%) were female, all of which did not differ significantly from the MS patients.
Neuropsychological results The scores on all cognition and mood measures for patients and healthy controls are shown in Table 1. Patients scored significantly worse than healthy
Table 1. Comparison of cognitive test scores for MS patients (n ¼ 82) and healthy control subjects (n ¼ 35) MS patients HC Subjects (n ¼ 82) (n ¼ 35) t-test p-value SRT-LTS SRT-CLTR SRT-D 10/36 SPART total 10/36 SPART-D SDMT total PASAT 3 s number correct PASAT 2 s number correct COWAT total correct
(8.3) (12.4) (1.9) (5.4) (2.2) (13.0) (9.7)
6.9 6.2 6.7 1.8 1.8 4.9 3.6
< 0.001b
