Accepted Manuscript - Journal of Substance Abuse Treatment

5 downloads 219 Views 247KB Size Report
Mar 4, 2017 - Methadone Maintenance Treatment Programs in Kunming, China ... Yunnan Institute for Drug Abuse, 471 Xi Fu Rd, Xi Shan District, Kunming,.
Journal of Substance Abuse Treatment 77 (2017) 89–94

Contents lists available at ScienceDirect

Journal of Substance Abuse Treatment

Reliability and validity of the Treatment Outcome Profile among patients attending methadone maintenance treatment programs in Kunming, China☆,☆☆ Mei Wang a,⁎, Jiucheng Shen b, Xianling Liu b, Yuan Deng b, Jiahua Li b, Emily Finch c, Kim Wolff d a

Institute of Psychiatry, Psychology and Neuroscience, King's College London, 4 Windsor Walk, London SE5 8BB, UK Yunnan Institute for Drug Abuse, 471 Xi Fu Rd, Xi Shan District, Kunming, 650000, PR China c South London and Maudsley NHS Trust, Addiction, Blackfriars, London, UK d Institute of Pharmaceutical Sciences, King's College London, Rm 5/10 Waterloo Bridge Wing, 150 Stamford St, London SE1 9NH, UK b

a r t i c l e

i n f o

Article history: Received 1 November 2016 Received in revised form 4 March 2017 Accepted 8 March 2017 Keywords: Treatment Outcome Profile Methadone maintenance treatment China Reliability Validity

a b s t r a c t Background & objective: Substance misuse has been a major health and social issue worldwide and has become an important public health issue in China over the past two decades. Methadone maintenance treatment (MMT) has been proved worldwide by large bodies of research to be one of the most effective practices for illicit drug users. The Treatment Outcome Profile (TOP) was developed in 2007 by the UK National Treatment Agency (NTA). It has been proved to be a reliable instrument for outcome measure. This study aim to develop the Chinese version of the Treatment Outcome Profile (TOP), and to assess whether TOP is a reliable outcome measure that can be recommended for use in Chinese MMT program. Methods: The Chinese version of TOP was translated and revised based on the English version of TOP. Psychometric properties of TOP were evaluated through face-to-face interviews in 197 patients who had been attending methadone maintenance treatment clinics in Kunming city, Yunnan Institute for Drug Abuse, for less than three months. Patients were interviewed by 3 trained interviewers. Reliability and validity of the instrument were analyzed by measures including test-retest and inter-rater reliability, concurrent validity and change sensitivity. Concurrent validity was assessed by comparing the scores from TOP with scores obtained from validated clinometric instruments. Self-reported opiate use was compared with results of urine analysis. Change sensitivity was judged by t-tests and chi-square tests. Results & conclusions: About 67% of the 197 interviewers were male and 33% were female. Test-retest reliability of TOP scores (after 10 days interval) were good (K = 0.65 to 0.95), inter-rater correlations (ICC) ranged from 0.7 to 0.9, and the criterion validity ranged from 0.72 to 0.88. TOP covers a large scope of problems encountered by drug users needed for treatment. The Chinese version of TOP is a reliable and valid assessment tool. © 2017 Elsevier Inc. All rights reserved.

1. Introduction Substance misuse has become an important social and public health issues in China over the past two decades. One of the key impacts of illicit drug use is the negative health consequences experienced by drug users. It also puts a heavy burden on society, contributes to crime, instability and insecurity of society and in particular, the spread of HIV/AIDS. By the end of 2014, 501,000 HIV/AIDS cases had been reported in China, and there were 159,000 HIV/AIDS related deaths in the country (China AIDS Response Progress Report, 2015). The Chinese government

☆ Declaration of interest: none ☆☆ This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. ⁎ Corresponding author. E-mail address: [email protected] (M. Wang).

http://dx.doi.org/10.1016/j.jsat.2017.03.004 0740-5472/© 2017 Elsevier Inc. All rights reserved.

has made efforts to address drug use problems and methadone maintenance treatment (MMT) has been acknowledged. The initial 8 pilot MMT clinics were set up in 2004 (Pang, Hao, & Mi, 2007). As preliminary evaluation suggested positive outcomes, a further 120 clinics were opened in 18 provinces by mid-2006 (Yin, Hao, & Sun, 2010). By the end of 2014, a total of 767 methadone maintenance clinics has been established in 28 provinces in the country, 184,000 people had received treatment (China AIDS Response Progress Report, 2015). In Yunnan Province, 13 methadone maintenance clinics were established in 2010, with a total of 7241 registered patients (Lou, 2010). These has increased to 68 methadone maintenance clinics and 39,472 patients registered for treatment in 2014 (Chinese Center for Disease Control and Prevention, 2014). The MMT programs in China have been proved to be successful in reducing illicit opiate use, HIV risk behavior and criminality related to heroin use and improving family and social aspects of life. However issues

90

M. Wang et al. / Journal of Substance Abuse Treatment 77 (2017) 89–94

have arisen associated with MMT programs, including management and evaluation of treatment. With the increasing emphasis on aftercare and relapse prevention, people have become more concerned with evaluating services related to treatment outcome. Effective clinical and research instruments that assess the multiple aspects of substance abuse as well as the need for treatment are urgently needed. Outcome monitoring measures the effectiveness of treatment interventions delivered in practice, and it is important to have agreement on how to measure outcomes. Over the past decades, standard instruments have been developed and widely used in addiction fields. These include The Addiction Severity Index (ASI) (McLellan, Luborsky, O'Brien, & Woody, 1980; McLellan, Kushner, & Metzger, 1992). The Opiate Treatment Index (Darke, Hall, Wodak, Heather, & Ward, 1992) and the Maudsley Addiction Profile (MAP) (Marsden et al., 1998). However, with the changes and emerge of new knowledge and research in substance misuse field, limitations have been revealed such as lengthy completion time (Cacciola, Alterman, Habing, & McLellan, 2011; Makela, 2004). Other outcome measures of dependency such as the Severity of Opiate Dependence Scale (Gossop et al., 1995), the Leeds Dependence Questionnaire (Raistrick et al., 1994), Brief treatment Outcome Measure (Lawrinson et al., 2005) and the Brief Addiction Monitor (Cacciola et al., 2013) have been developed for methadone treatment with considerably shorter administration time and less training for administer. The need to assess substance abuse patients through structured and validated instruments has been emphasized in China. However, multidimensional instruments developed and validated in the Chinese addiction field are limited. The Addiction Severity Index (ASI) (McLellan et al., 1992, 1980) was validated and has been widely used in Chinese substance misuse field (Lou, Wu, & Wei, 2010). Other standardized outcome measures such as Rating Scale for Protracted Withdrawal Symptoms of Heroin Addicts (Shi et al., 2009) has also been used in Chinese drug treatment field. The Treatment Outcome Profile (TOP) was developed in 2007 by the UK National Treatment Agency (NTA) for monitoring changes in drug users across different treatment episodes (National Treatment Agency, 2010). It has been tested by structured drug treatment services and presents a standardized way of asking questions and recording answers that will show how well patients are doing and how effective the treatment services are and has been validated in 1021 UK samples (Marsden et al., 2008) as well as in Australia (Ryan et al., 2014) and Chile (CastilloCarniglia et al., 2015) samples and has been proved to be a reliable instrument for outcome measure. Compared with other assessment tools used in the drug addiction field which are lengthy, TOP is a brief instrument with the average completion time between 11 and 13 min in different clinical settings across the world. In addition, unlike other instruments, TOP is a multidimensional tool consisting of four items across the domains of substance use, injecting risk behaviors, crime, as well as health and social functioning. This study is designed to assess the psychometric properties of a Chinese version of TOP among a sample of drug users attending MMT clinics in Kunming, China, to assess the cross-cultural validity of TOP and to assess whether TOP is a reliable outcome measure and instrument that can be recommended for use in the Chinese substance misuse field. In the interest of international comparability, minor modifications were made to the original questionnaire based on Chinese social and cultural context and to better reflect Chinese programme settings.

Hp = 0.80 was set against Hp0 = 0.50 (no association), requiring 31 participants to reject the null-hypothesis (power = 0.9, α = 0.05, one-tailed). For dichotomous items, Cohen's kappa was set at 0.61 (threshold for ‘substantial’ agreement), requiring 33 participants for each comparison (power = 0.8, α = 0.05, one tailed) (Landis & Koch, 1977; Marsden et al., 2008). Prior to start this study, we had discussion with TOP original author Professor John Marsden of the National Addiction Center, U.K. for his permission to conduct our study. This study was conducted in 2 MMT clinics in Yunnan Institute for Drug Abuse (YIDA), China. Kunming, the capital city of Yunnan province, borders the Golden Triangle and has severe drug trafficking and abuse problems. In response to global drug and AIDS problems, with the approval of Ministry of Health, YIDA was set up in May 1993. The institute is currently running 2 MMT clinics which were set up in 2005. YIDA MMT clinics are outpatient clinics, N500 patients attend the clinics for daily methadone prescription per day. Patients frequently drop out, enroll and drop out again. However, most of our patients stay in treatment for N 3 years and with considerable amount of patients stay in treatment for N5 years. MMT programs in China are funded by the Chinese Central Government via the Ministry of Health, along with the Ministry of Public Security and the State Food and Drug Administration and local provincial governments. The maximum daily methadone fee for patients is CNY10 (US$1.6), irrespective of dosage. Clinics using this money to pay rent, water and electricity bills and other expenses. Participants had been in MMT for b3 months during the study time period between August 2013 and May 2014. 197 subjects were included in the study. Exclusion criteria were: (1) individuals presenting with intoxication or acute withdrawal symptoms; (2) patients with severe cognitive or communication impairment; (3) patients attended treatment b10 days per month in the past 6 months and those who told us that they couldn't or were not sure if they could attend for the second interview. Three interviewers (the investigator and two doctors) participated in this study. The two doctors involved each had N 10 years of psychiatric clinical experience. Each interviewer completed a 2-day training session, including reviewing the TOP manual, and followed by thorough discussions and practice sessions. The interviewers familiarized themselves by reading the Chinese translation of TOP and the instruction manual, as well as the original English training manual produced by NTA. However, patients might give different responses if they already knew the interviewer in the treatment team.

2. Material and methods

2.3. Content validity

2.1. Participants

In psychometrics studies, content validity has been referred to the extent to which an instrument covers all areas of an underlined construct. Content validity is a critical pre-requirement prior to the development of a new instrument (Streiner, Norman, & Cairney, 1989). Treatment Outcome Profile has been tested to be a reliable instrument

Kraemer and Thiemann's critical effect size (Δ) was used to estimate the number of participants required for interrater, test-retest reliability testing (Kraemer & Thiemann, 1987). For scaled items, correlation of

2.2. Instruments TOP assesses treatment outcomes in twenty items grouped into four outcome domains: (1) substance use; (2) injecting risk behavior; (3) crime; (4) health; and social functioning. The Chinese version of TOP was based on the above four domains. Minimal changes were made to the original questionnaire for the consideration of the Chinese social and cultural background in the substance use and crime domains. Benzodiazepine and ketamine were added in the substance use domain as drugs used in China, and crack cocaine was combined with cocaine. In the crime domain, “theft of vehicle” was changed to “theft of motorbike or bicycles”. The Chinese version of TOP was translated by two Chinese native speakers specializing in addiction research. The result was then translated back into English and compared with the English version by a native English speaker.

M. Wang et al. / Journal of Substance Abuse Treatment 77 (2017) 89–94

for outcome measure and has gained international reputation over the past years (Marsden et al., 2008). The Chinese version TOP was translated from the English version TOP. It has enough items to adequately cover the topic. In this study, we evaluated content validity of the Chinese version TOP by using experts and subjects judgements. Formal discussions were held among members of the research team. Meetings were organized, professionals and experts known to be actively involved in MMT clinics in Kunming were invited and consulted about whether the Chinese version TOP has adequately covered the topic and whether items were phrased as clearly and succinctly as possible in order to ensure patients' and clients' understanding. In addition, as Streiner et al. (1989) stated that research subjects are the best judges for subjective elements, 25 MMT patients were also contacted to give comments towards the instrument. It indicated that the content validity of Chinese version TOP is satisfactory. 2.4. Pre-test A pre-test was carried out. Thirty patients who were attending the methadone maintenance treatment and met the inclusion criteria were selected and invited to complete the Chinese translation of TOP. The questionnaires were administered to the patients by 3 interviewers. The average time for the patients to complete the questionnaire was 10.6 min (SD = 4.8). 96% of the patients demonstrated that they fully understood the questionnaire.

91

time interval of 10 days. Inter-rater intra class correlation coefficient (ICC) for scale measures with 95% confidence interval (CI) were computed from two-way, random-effects analysis of variance and those reaching 0.75 were considered as ‘excellent’ agreement (Shrout & Fleiss, 1979; Fleiss, Levin, & Paik, 2003). Prevalence variables with Cohen's kappa reaching 0.61 were considered as satisfactory (Bartko, 1996; Cohen, 1960). 2.6.2. Concurrent validity Concurrent validity was assessed by using correaltion for scale measures and Cohen's Kappa for prevalence measures. Correlation between TOP items and items of other clinometric instruments was computed using Spearman's ρ correlation (r). The concordance of self-reported opiate use and urine analysis was judged using Cohen's kappa, together with sensitivity and specificity (Cicchetti & Feinstein, 1990). 2.6.3. Change sensitivity A validated, reliable and effective instrument must be able to detect the smallest and practical differences change follow a certain period of time. Sensitivity analysis is a useful technique to determine this ability (Marsden et al., 2008). In this study, change sensitivity was judged by comparing the Chinese TOP scores of test and follow-up. A non-parametric test the Wilcoxon Signed-Rank test was used for continuous variables and chi-square tests for categorical variables. Data were analyzed using SPSS 19.0 for windows. P values represented were from 2-tailed tests and P b 0.05 was considered significant.

2.5. Procedures The test-retest and inter-rater reliability indicators were examined. The stability of the measures over time was evaluated by comparing TOP scores between the first and second interviews with an average time interval of 10 days. When did retest, we checked previous test records and informed all participated subjects that the re-test we asked were to recall their drug use 28 days before the test date but not 28 days before that time. We therefore made sure that it was identical period. Concurrent validity was assessed by comparing the scores from TOP with scores obtained from the following clinometric instruments.

3. Results 3.1. Patient demographic characteristics and substance use

4. Psychological health: SF 36 questionnaire for psychological health, with recall over past month.

Participants had been in MMT for b 3 months during the study period between May and December 2013. One hundred and ninety seven patients were approached for the test study while waiting for their methadone prescriptions. The sample comprised 132 males and 65 females, and among them 91.4% were ethnically Han and 17 patients (8.6%) were from ethnic minorities. The majority of the participants were married or living with their partners (48.2%), whereas 32% and 19.8% of the participants were single and divorced or separated respectively. The subject's average age was 41.5 years (SD = 6.7). The mean education level was 9.7 years (SD = 2.6). Sixty seven percent of the subjects (n = 132) were employed (Table 1). Twenty five point 3% of the test participants used alcohol in the past 28 days, 88.5% reported illicit opiate use in the past 28 days, but only 2 people (2.3%) reported using benzodiazepine in the last 28 days. Five people (5.7%) reported using amphetamine and one person used Ketamine. None of the subjects reported using cocaine and cannabis in the past 28 days (Table 2). Following test interviews, 87 patients were selected for the re-test study after 10 days. One hundred and eighteen patients (60%) attended follow up tests after 35 days (Table 1). The average time used to administer the test form was 10.5 min. On average, the retest interview was held 9.3 days after test (SD = 7.7). There is consistent demographical homogeneity among patients attended test, retest and follow up studies.

5. Quality of life: Life Satisfaction Index B (LSI B), with recall over past 4 weeks.

3.2. Inter-rater and test-retest reliability

1. Heroin use: self-reported heroin use for the past four weeks, where subsequent concordance analysis compared declarations for the same week with urine test results done by clinic staffs recorded in the medical records. 2. Injecting risk behavior: two items were taken from the AIDS related risk behavior of the Chinese version of ASI (‘How many times did you inject drugs in the past 30 days? and ‘Did you share injecting paraphernalia in last 30 days?’) (Lou et al., 2010). 3. Physical health: the 36-item Short Form survey from the Rand Medical Outcome Study (SF 36) (Rand Corporation, 2016) for general health, an assessment of somatic symptoms in the past month.

(Wang & Fan, 1993). Follow up study was carried out after an average of 35 days to assess change sensitivity of this instrument. 2.6. Data analysis 2.6.1. Test–retest reliability and inter-rater reliability The stability of the measures over time was evaluated by comparing TOP scores between the first and second interviews with an average

Inter-rater and test-retest reliability were computed. ICC and K coefficients for all items ranged from 0.70 to 1.00 and 0.65 to 0.94 respectively. 3.3. Concurrent validity Tables 3 and 4 present the correlations between the TOP scores and the criteria measurements. - Physical and Psychological health

92

M. Wang et al. / Journal of Substance Abuse Treatment 77 (2017) 89–94

Table 1 Demographics of study subjects (N = 197). Characteristics of subjects Gender Male: no. (%) Female: no. (%) Ethnic group Han people: no. (%) Minority: no. (%) Marrital status Single: no. (%) Married/cohabiting: no. (%) Divorced/separated: no. (%) Widowed: no. (%) Employment status Unemployed: no. (%) Employed: no. (%) Peasant: no. (%) Education level (years): mean ± SD Age in years: mean ± SD

Test (n = 197)

Retest (n = 87)

Follow-up (n = 118)

P value

132 (67.0%) 65 (33.0%)

60 (69.0%)

81 (68.7%)

0.93

27 (31.0%)

37 (31.4%)

180 (91.4%) 17 (8.6%)

80 (92.0%)

109 (92.4%)

7 (8.1%)

9 (7.6%)

61 (32.0%) 95 (48.2%)

27 (31.0%) 37 (42.5%)

30 (25.4%) 61 (51.7%)

39 (19.8%)

22 (25.3%)

26 (22.0%)

2 (1.0%)

1 (1.2%)

1 (0.9%)

63 (32.0%) 132 (67.0%) 2 (1.0%) 9.7 ± 2.6

34 (39.1%) 52 (59.8%)

40 (33.9%) 77 (65.3%)

1 (1.1%) 9.8 ± 1.9

1 (0.8%) 9.9 ± 2.3

0.84

41.5 ± 6.7

41.0 ± 6.9

42.1 ± 6.7

0.49

0.95

0.85

opiate use days significantly decreased (17.58–3.08, P ≤ 0.000), as well as amphetamine use days (0.19–0.02, P ≤ 0.05). On a typical day, alcohol used and illicit opiate use were significantly reduced (8.93–7.12 & 0.30– 0.10, P ≤ 0.00 respectively), as well as days injected (6.90–1.75, P ≤ 0.00). In health and social function section, participants' psychological and physical health were significantly increased (11.31–13.58 & 11.02– 12.78, P ≤ 0.00 respectively), however, the number of school/training days reduced (1.26–0.50, P ≤ 0.00) (Table 5). Findings of prevalence measures on direct and indirect receptive sharing, theft from or of a vehicle, property theft/burglary, and fraud/forgery handling, were not significant. For substances such as benzodiazepine, cocaine, ketamine and cannabis, follow up studies on days used and quantity used on typical days were not significant. Participants' quality of life did not show any improvement. 4. Discussion

0.84

SD: standard deviation. p values represented 2 tailed were from Pearson chi-suqare test for categorical variables and from non-parameter test for continuous variables.

There were statistically significant correlations between TOP physical and health items and SF36 questionnaires for general health and psychological health (r = 0.72, P b 0.00 & r = 0.88, P b 0.00 respectively) (Table 3). - Quality of Life The correlation of Chinese TOP Quality of life score with Life Satisfaction Index B (LSIB) was satisfactory (r = 0.88, P b 0.00) (Table 3). - Injecting risk behavior & injecting paraphernalia sharing The correlations of Chinese TOP injecting risk and injecting paraphernalia sharing behaviors with the two items from AIDS related risk behavior of Chinese version ASI was satisfactory (r = 0.79 & r = 0.91, P b 0.00) (Table 3) - Urine concordance We take urine sample to check illicit opiate use on a random schedule every month and on selective patients (not every patient) attended clinics. TOP require patients to recall past 4 weeks' drug use and therefore we checked patients past 4 weeks' medical records and a total of 40 participants forms were randomly selected to compare with self-reported heroin use for the past week and the subsequent concordance with urine analysis results were recorded in the clinical record. There were acceptable concordance, sensitivity and, specificity (0.97 and 0.78) (Table 4). 3.4. Change sensitivity One hundred and eighteen patients (60%) completed the follow up study. On average, interviews were held 35 days after the test (SD = 5.1) and administered in 9.3 min (SD = 4.2). Among patients in treatment for 3 months or less, there were statistically significant reductions on the following measures: shop theft (0.50%–0.00%, P ≤ 0.05), no source of income (14%–1%, P ≤ 0.05), and at risk of eviction (24%–3%, P ≤ 0.05) (Table 5). With respect for changes in substance use days, illicit

There is increasing recognition of the importance of monitoring the outcome of substance misuse treatment. The Chinese MMT program is a large centralized drug treatment program and had made progress in reducing HIV infection, crime and drug use among clients who remained in treatment. A need for reliable and valid instruments for the effective measurement of multi-dimensional outcomes in drug treatment services in China is urgently needed. The Chinese version of TOP was psychometrically evaluated in this study and the results were demonstrated to have acceptable levels of reliability and validity and are consistent with the original findings. The average time for completing the Chinese version of TOP in this study is 8.9 min (SD = 3.9). Test-retest reliabilities and inter-rater reliabilities of all scales in the instrument are high and these correspond to the values found in the original study (Marsden et al., 2008). The concurrent validity of TOP was acceptable (Table 3). There were acceptable concordances, sensitivity and specificity between self-report and urine analysis records for heroin use (Table 4). A follow up study showed that change sensitivity was acceptable. These data indicated the substantial validity of this instrument. The relationship between drug use and criminality is well documented (Harrison & Gfroerer, 1992; McBridge, Joe, & Simpson, 1991). However, the Crime domain assessed in our study showed that 90% of the interviewees reported no problems in the past 28 days. This may due to the fact that our patients have records in the Public Security Department, so even if confidentiality is assured, it logically follows that criminal activities may either denied or under-reported. Yet it is agreed that in criminal justice the fear of punishment could be an international problem (Harrell, Kapsak, Cisin, & Wirtz, 1997; Harrison, 1997). This can be considered as one limitation of this study and it would have effect on the evaluation of the reliability and validity of the scale. Another limitation of this study is that none of our patients reported used cocaine and cannabis and only one person reported using Ketamine. Cocaine and cannabis are rarely used substances in Yunnan Province, yet the results may affect the reliability and validity of the substance use scale. Hence a follow up study showed that change sensitivity of cocaine, ketamine and cannabis on days used and quantity used on a typical days were not significant. Participants' quality of life did not show any improvement in this study, but this may due to the fact that 3 months of time is not long enough to detect the significant change. At present, most drug treatment services in hospitals and drug treatment centers in China focus on patients' clinical symptoms when evaluating treatment outcomes. If psychological evaluations are needed, additional instruments will be used. The validated Chinese version TOP appears to be a valid and reliable multidimensional instrument for assessing broad drug related problems in China that across the domains of substance use, injecting risk behaviors, crime, health and social functioning. In addition to assess severity of the problems of drug users who attending MMT clinics for treatment, it can also be applied to different settings and to wider populations such as mental health settings

M. Wang et al. / Journal of Substance Abuse Treatment 77 (2017) 89–94

93

Table 2 Items inter-rater test-retest reliabilities (N = 87). Items (past 4 weeks)

Test

Retest

K

22 (25.3%) 2.1 ± 5.7 9.6 ± 35.1

19(21.8%) 1.7 ± 4.9 8.2 ± 32.8

0.65

77 (88.5%) 19.2 ± 11.3 0.3 ± 0.3

76 (87.4%) 18.1 ± 11.4 0.3 ± 0.2

0.73

2 (2.3%) 0.4 ± 3.2 0.1 ± 0.4

3 (3.4%) 0.5 ± 3.2 0.1 ± 0.4

0.80

0 (0.0%) 0 ± 0.0 0 ± 0.0

0 (0.0%) 0 ± 0.0 0 ± 0.0



5 (5.7%) 0.1 ± 0.6 0.0 ± 0.1

4 (4.6%) 0.1 ± 0.6 0.0 ± 0.0

0.88

1 (1.1%) 0.0 ± 0.3 0.0 ± 0.0

2 (2.3%) 0.1 ± 0.3 0.0 ± 0.0

0.66

0 (0.0%) 0 ± 0.0 0 ± 0.0

0 (0.0%) 0 ± 0.0 0 ± 0.0



25 (28.7%) 6.6 ± 11.2 7 (8.0%) 6 (6.9%)

25 (28.7%) 6.1 ± 10.6 8 (9.2%) 7 (8.0%)

0.88

6 (6.9%) 1.3 ± 5.2 2 (2.3%) 0.6 ± 4.2 0 (0.0%) 0 (0.0%) 0 (0.0%) 2 (2.3%)

6 (6.9%) 0.9 ± 4.0 3 (3.4%) 1.0 ± 5.1 0 (0.0%) 0 (0.0%) 0 (0.0%) 1 (1.1%)

0.82

10.8 ± 5.4

10.4 ± 4.6

Days of paid work (mean ± SD)

17.0 ± 11.6

17.4 ± 11.2

Days of school/Training (mean ± SD) Physical health (mean ± SD)

2.0 ± 6.0 10.5 ± 6.1

1.7 ± 5.7

No any source of income(%) At risk of eviction (%) Quality of life (mean ± SD)

7 (8.0%) 9 (10.3%) 12.8 ± 5.3

Alcohol Used: n (%) Days used (mean ± SD Units used (g) (mean ± SD) Illicit opiate Used: n (%) Days used (mean ± SD) Units used (g) (mean ± SD) Benzodiazepines Used: n (%) Days used (mean ± SD) Units used (g) (mean ± SD) Cocaine Used: n (%) Days used (mean ± SD) Units used (g) (mean ± SD) Amphetamine Used: n (%) Days used (mean ± SD) Units used (g) (mean ± SD) Ketamine Used: n (%) Days used (mean ± SD) Units used (g): (mean ± SD) Cannabis Used: n (%) Days used (mean ± SD) Units used (volume) (mean ± SD) Injection drug use behaviors Injected: n (%) Injected days (mean ± SD) Direct receptive sharing: n (%) Indirect receptive sharing: n (%) Crime behaviors Shop theft: n (%) Shop theft days (mean ± SD) Drug selling: n (%) Drug sellingdays: (mean ± SD) Theft from or of a vehicle: n (%) Property theft/burglary: n (%) Fraud/forgery/handling: n (%) Assault/violence: n (%) Health & social function Psychological health (mean ± SD)

Mean difference (95% CI)

ICC (95% CI)

0.4 (−0.3, 1.1) 1.3 (−1.3, 4.0)

0.80 (0.71, 0.87) 0.93 (0.90, 0.96)

1.3 (0.01, 2.54) 0.00(−0.04,0.04)

0.86 (0.80, 0.91) 0.76 (0.66, 0.84)

-0.01 (−0.03, 0.01) -0.01 (−0.03, 0.01)

1.00 (1.00, 1.00) 0.97 (0.95, 0.98)

– –

– –

0.01 (−0.01, 0.03) 0.00 (−0.00, 0.01)

0.98 (0.98, 0.99) 0.96 (0.94, 0.98)

-0.01 (−0.03,0.01) -0.00 (−0.00,0.00)

0.95 (0.92, 0.97) 0.98 (0.96, 0.98)

– –

– –

0.5 (−0.9, 1.9)

0.82 (0.73, 0.88)

0.40 (−0.37, 1.17)

0.70 (0.57, 0.79)

-0.32 (−0.96, 0.0.32)

0.80 (0.70, 0.86)

0.41 (−0.26, 1.09) -0.15 (−2.0, 1.1) 0.31 (−0.07, 0.69) 0.16 (−0.57, 0.88)

0.80 (0.71, 0.87)

0.93 0.92

0.79 – – – 0.66

0.81 (0.72, 0.87)

0.75

10.3 ± 6.3 6 (6.9%) 8 (9.2%) 12.2 ± 5.3

0.95 (0.93, 0.97) 0.85 (0.78, 0.90)

0.94 0.59 (0.05, 1.12)

0.89 (0.84, 0.93)

ICC: Inter-rater intra class correlation coefficient CI: confidence interval.

and prisons and to homeless people. It is a comprehensive assessment tool that can be used in both clinical and research applications. To our knowledge, our study is the first study that carried out in China to validate TOP among Chinese drug addicts. We hope Chinese version TOP will be recognized and incorporated into the drug treatment systems with appropriate reporting system available for clinical

evaluation and for research purposes in Yunnan and/or China once it is officially published.

Ethics approval This study was approved by King's College London Ethics Committee (approval number: PNM/11/12-134) and by Yunnan Institute for Drug Abuse Ethics Committee. Informed consent was given by all patients who participated in the study.

Table 3 Concurrent validity of TOP with SF/LSIB/ASI (N = 130). Items in TOP

Spearman rho (r)

P value (2tailed)

Psychological health Physical health Quality of life Injecting risk behaviors Injecting paraphernalia sharing

0.72 0.80 0.88 0.79 0.91

b0.00000 b0.00000 b0.00000 b0.00000 b0.00000

Table 4 Validity of self-report (SR) heroin use with urine test (UT) (N = 40). TOP Item

Sensitivity

SR+/UT+

Specificity

SR−/UT−

K (95%CI)

Heroin use

0.97

32/31

0.78

9/8

0.78 (0.48, 1.00)

94

M. Wang et al. / Journal of Substance Abuse Treatment 77 (2017) 89–94

Table 5 Change sensitivity for scaled measures among subjects in treatment for or b3 months (N = 118). Measure

Test

Follow-up

Z value

2.89 ± 7.15 8.93 ± 34.34

3.69 ± 8.33 7.12 ± 34.39

−0.31 0.76 2.91 0.00⁎⁎

17.58 ± 12.06 0.30 ± 0.28

3.08 ± 6.10

−8.43 0.00⁎⁎⁎

0.10 ± 0.28

−7.21 0.00⁎⁎⁎

0.54 ± 3.33 0.04 ± 0.30

0.34 ± 2.80 0.02 ± 0.21

−1.00 0.32 −1.34 0.18

0.05 ± 0.42 0.00 ± 0.02

0.00 ± 0.00 0.00 ± 0.00

0.00 0.00

0.19 ± 1.16 0.03 ± 0.17

0.02 ± 0.18 0.05 ± 0.55

−1.98 0.05 −1.40 0.16

0.02 ± 0.22 0.00 ± 0.00

0.00 ± 0.00 0.00 ± 0.00

−1.00 0.32 −1.00 0.32

0.00 ± 0.00 0.00 ± 0.00

0.00 ± 0.00 0.00 ± 0.00

0.00 0.00

6.90 ± 11.20 8 (4.02%) 0 (0.00%)

1.75 ± 5.36

−4.70 0.00⁎⁎⁎

0.50 ± 3.43 0.28 ± 2.8 0 (0.00%) 0 (0.00%) 0 (0.00%) 3 (1.51%)

0.00 ± 0.00 0.00 ± 0.00 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%)

−2.04 0.04⁎ −1.41 0.16 – – – 0.25

11.31 ± 5.40 18.00 ± 11.36 School/training: mean days ± SD 1.26 ± 4.62 Physical health: mean score ± SD 11.02 ± 5.60 No any source of income: n (%) 14 (7.04%) At risk of eviction: n (%) 24 (12.06%) Quality of life: mean score ± SD 13.33 ± 5.24

13.58 ± 4.43 15.00 ± 11.66 0.50 ± 3.17 12.78 ± 5.44 1 (1.15%) 3 (3.4 5%) 13.22 ± 4.90

−4.14 0.00⁎⁎⁎

Alcohol Mean days used ± SD Mean units (g) used ± SD Illicit opiate Mean days used ± SD Mean units (g) used ± SD Benzodiazepines Mean days used ± SD Mean units (mg) used ± SD Cocaine Mean days used ± SD Mean units (g) used ± SD Amphetamine Mean days used ± SD Mean units (g) used ± SD Ketamine Mean days used ± SD Mean units (g) used ± SD Cannabis Mean days used ± SD Mean units (volume) used ± SD Injecting behaviors Mean days injected ± SD Direct receptive sharing: n (%) Indirect receptive sharing: n (%) Crime Shop theft: mean days ± SD Drug selling: mean days ± SD Theft from or of a vehicle: n (%) Property theft/burglary: n (%) Fraud/forgery handling: n (%) Assault/violence: n (%) Health & social function Psychological health: mean score ± SD Paid work: mean days ± SD

8 (9.20%) 0 (0.00%)

P value

1.00 1.00

1.00 1.00

0.08 –

−1.69 0.09 −3.02 0.00⁎⁎ −5.02 0.00 0.04⁎ 0.02⁎ −0.02 0.98

SD: Standard deviation. ⁎p b0.01 ⁎⁎p b0.001 ⁎⁎⁎Pb0.00001. P values represented 2 tailed were from Pearson chi-square test for categorical variables and from non-parameter test for continuous variables.

Acknowledgements The authors would like to thank the staff in Yunnan Institute for Drug Abuse clinics for their support during data collection for this study. References Bartko, J. J. (1996). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 3–11. Cacciola, J. S., Alterman, A. I., Habing, B., & McLellan, A. T. (2011). Recent status scores for version 6 of the Addiction Severity Index (ASI-6). Addiction, 106, 1588–1602. Cacciola, J. S., Alterman, A. I., DePhilippis, D., Drapkin, M. L., Jr, V. C., Fala, N. C., ... McKay, J. R. (2013). Development and initial evaluation of the Brief Addiction Monitor (BAM). Journal of Substance Abuse Treatment, 44, 256–263. Castillo-Carniglia, A., Marin, J. D., Soto-Brandt, G., Donoso, M. P., Piñol, D., San Martín, J., ... Portilla Huidobro, R. (2015). Adaptation and validation of the instrument treatment outcomes profile to the Chilean population. Journal of Substance Abuse Treatment, 56, 39–47. Chinese Center for Disease Control and Prevention (2014). http://www.chinacdc.cn/dfdt/ 201408/t20140807_100952.htm accessed 29.12.16

Cicchetti, D. V., & Feinstein, A. (1990). R. High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43, 551–558. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. Darke, S., Hall, W., Wodak, A., Heather, N., & Ward, J. (1992). Development and validation of a multi-dimensional instrument for assessing outcome of treatment among opiate users: The Opiate Treatment Index. British Journal of Addiction, 87, 733–742. Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). New York: Wiley. Gossop, M., Darke, S., Griffiths, P., Hando, J., Powis, B., Hall, W., & Strang, J. (1995). The severity of dependence scale (SDS): Psychometric properties of the SDS in English and Australian samples of heroin, cocaine and amphetamine users. Addiction, 90, 607–614. Harrell, A. V., Kapsak, K., Cisin, I., & Wirtz, P. (1997). The validity of self-reported drug use data: The accuracy of responses on confidential self-administered answered sheets. NIDA Research Monograph, 167, 37–58. Harrison, L., & Gfroerer, J. (1992). The intersection of drug use and criminal behaviour: Results from the national household survey on drug abuse. Crime and Delinquency, 38, 4422–4443. Harrison, L. (1997). The validity of self-reported drug use in survey research: An overview and critique of research methods. In L. Harrison, & A. Hughes (Eds.), NIDA Research Monographs. 167. (pp. 17–36). Kraemer, H. C., & Thiemann, S. (1987). How many subjects? Statistic power analysis in research. Newbury, CA: Sage Publications. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. Lawrinson, P., Copeland, J., & Indig, D. (2005). Development and validation of a brief instrument for routine outcome monitoring in opioid maintenance pharmacotherapy services: The brief treatment outcome measure (BTOM). Drug and Alcohol Dependence, 80, 125–133. Lou (2010). The existing challenges and prospects in preventing AIDS in methadone maintenance treatment programs – Yunnan Province as an example. Journal of Yunnan Police College, 5, 59–62. Lou, W., Wu, Z. Y., & Wei, X. L. (2010). Reliability and validity of the Chinese version of the Addiction Severity Index. Journal of Acquired Immune Deficiency Syndromes, 53, 121–125. Makela, K. (2004). Studies of the reliability and validity of the Addiction Severity Index. Addiction, 99, 398–410. Marsden, J., Farrell, M., Bradbury, C., Dale-Perera, A., Eastwood, B., Roxburgh, M., & Taylor, S. (2008). Development of the Treatment Outcome Profile. Addiction, 103, 1450–1460. Marsden, J., Gossop, M., Stewart, D., Best, D., Farrell, M., Lehmann, P., ... Strang, J. (1998). The Maudsley Addiction Profile (MAP): A brief instrument for assessing treatment outcome. Addiction, 93, 1857–1868. McBridge, A. A., Joe, G. W., & Simpson, D. D. (1991). Prediction of long-term alcohol use, drug use and criminality among inhalant users. Hispanic Journal of Behavioral Sciences, 13, 315–323. McLellan, A. T., Kushner, H., & Metzger, D. (1992). The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment, 9, 199–213. McLellan, A. T., Luborsky, L., O'Brien, C. P., & Woody, G. E. (1980). An improved evaluation instrument for substance abuse patients: The Addiction Severity Index. Journal of Nervous and Mental Disease, 168, 26–33. National Health and Family Planning Commission of the People's Republic of China (2015). China AIDS Response Progress Report. (http://www.unaids.org/sites/default/ files/country/documents/CHN_narrative_report_2015.pdf Accessed 18.06.16.). National Treatment Agency, Public Health England (2010). Treatment Outcome Profile. (http://www.nta.nhs.uk/healthcare-top.aspx Accessed 18.06.16). Pang, L., Hao, Y., & Mi, G. (2007). Effectiveness of first eight methadone maintenance treatment clinics in China. AIDS, 21, 103–107. Raistrick, D., Bradshaw, J., Tober, G., Weiner, J., Allison, J., & Healey, C. (1994). Development of the Leeds Dependence Questionnaire (LDQ): A questionnaire to measure alcohol and opiate dependence in the context of a treatment evaluation package. Addiction, 89, 563–572. Rand Corporation Corporation (2016). Rand Medical Outcomes Study, 36-Item short form survey. http://www.rand.org/health/surveys_tools/mos/mos_core_36item.html Accessed 18.06.16 Ryan, A., Holmes, J., Hunt, V., Dunlop, A., Mammen, K., Holland, R., ... Lintzeris, N. (2014). Validation and implementation of the Australian treatment outcomes profile in specialist drug and alcohol settings. Drug and Alcohol Review, 33, 33–42. Shi, J., Wang, J., Bao, Y., Liu, T., Zhang, X. L., Hao, Y., ... Sheng, L. (2009). The reliability and validity evaluation of rating scale for protracted withdrawal symptoms of heroin addicts. Chinese Journal of Drug Dependence, 18, 107–113. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. Streiner, D. L., Norman, G. R., & Cairney, J. (1989). Health measurement scales: A practical guide to their development and use (5th ed.). Oxford: Oxford University Press. Wang, X. D., & Fan, X. D. (1993). Life Life Satisfaction Index B (LSI B). Chinese Mental Health Journal, 77, 76–89. Yin, W., Hao, Y., & Sun, X. (2010). Scaling up the national methadone maintenance treatment program in China: Achievements and challenges. International Journal of Epidemiology, 39, 29–37.