MethodologicalIssuesinAssessingtheEvidence for ... - Ingenta Connect

1 downloads 0 Views 144KB Size Report
1Departments of Community Health & Epidemiology and Psychiatry, ... 2Health Outcomes Research Unit, Capital District Health Authority, Halifax, Nova Scotia, ...
Current Psychiatry Reviews, 2007, 3, 51-56

51

Methodological Issues in Assessing the Evidence for Compulsory Community Treatment Stephen Kisely*,1,2 and Leslie Anne Campbell1,2 1

Departments of Community Health & Epidemiology and Psychiatry, Dalhousie University, Halifax, Nova Scotia, Canada 2

Health Outcomes Research Unit, Capital District Health Authority, Halifax, Nova Scotia, Canada Abstract: There is controversy as to whether compulsory community treatment (CCT) reduces health service use, or improves clinical outcome and social functioning. Given the widespread use of such powers in North America, Europe and Australasia, it is important to assess the benefits and potential harms of this type of legislation. Unfortunately, evidence for their effectiveness remains weak. In our Cochrane systematic review we were only able to identify two randomised controlled trials, and neither study showed any significant reductions in readmission rates, bed-days or arrests for cases on compulsory community treatment compared to appropriate controls. In terms of the number needed to treat (NNT), it would take up to 100 treatment orders to prevent one readmission, 25 to prevent one episode of homelessness and 500 to prevent one arrest. Including other study designs such as controlled before and after (CBA) studies made little difference to the results (total n=1108). There are other concerns for patients. This legislation stigmatises individuals with a severe mental illness, as many of these initiatives are named after a high profile victim of someone who happened to have a psychiatric illness. New York has a 'Kendra's Law', California a 'Laura's Law', and Ontario a 'Brian's Law’. Compulsory community treatment appears to be immune from evidence-based practice, possibly because of the convenience of legislative as opposed to evidence-based solutions such as assertive community treatment.

Keywords: Compulsory community treatment, research methodology. INTRODUCTION

HOW DO WE MEASURE SUCCESS?

Compulsory treatment in the community covers a wide range of interventions including Community Treatment Orders (CTOs), Involuntary Outpatient Treatment (IOT), Involuntary Outpatient Commitment (IOC), Outpatient Commitment (OPC) and Supervised Discharge (SD). Involuntary outpatient commitment is permitted in most of the United States, and conditional leave or community treatment orders have also been introduced in Canada, Scotland, New Zealand, Australia, Israel, Norway, Switzerland, Portugal, Sweden and the Benelux countries [1-4]. In England and Wales, the existing Mental Health Act can impose certain requirements on people living in the community through guardianship or supervised discharge; these are under very limited circumstances and the provisions are rarely used [2]. Extended leave from hospital under the Mental Health Act was also used as a form of compulsory community treatment but was subsequently judged illegal and the practice stopped. However, the introduction of a Non-Resident Order is under consideration, although there will still be no powers to give medication forcibly outside a clinical setting [2].

One problem in assessing the effectiveness of compulsory community treatment (CCT) is the uncertainty in how to measure success following their use. Does CCT reduce admission rates, allowing individuals to remain in their communities during treatment, or are rates increased as a result of earlier identification of relapse? Would length of stay be a more appropriate measure on the basis that increased admission would still be the least restrictive alternative if individuals spent less time in hospital? Is diversion from the criminal justice system in the form of reduced arrests or imprisonment another possible outcome?

In light of such widespread use, what is the evidence for the effectiveness of compulsory community treatment? We critically examine several methodological issues to consider when assessing their impact on patients in the light of our Cochrane review of randomised control trial (RCT) evidence and two controlled before and after (CBA) studies [3, 5]. *Address correspondence to this author at Room 425, Centre for Clinical Research, 5790 University Avenue, Halifax, Nova Scotia, Canada B3H 1V7; Tel +1 902-494-7075; Fax: +1 902-494-1597; E-mail: [email protected] 1573-4005/07 $50.00+.00

ARE WE ASSESSING THE SAME INTERVENTION? The range of different interventions makes it even harder to interpret the literature. These include community treatment orders, involuntary outpatient treatment, involuntary outpatient commitment, extended leave, extended release or supervised discharge. Extended leave provisions or supervised discharge are applied at the time of discharge from compulsory inpatient treatment. They are used in Canada [6], Great Britain [7] and New Hampshire, USA [8]. They give mental health professionals the right to return a patient to hospital against their wishes if they do not comply with treatment. Community treatment orders (CTOs) are used in Australia [9, 10] and Canada [6] and give mental health professionals the right to place an individual on an order, whether they are in hospital or not. This is in contrast to extended leave or supervised discharge, which only applies to patients who are being discharged from inpatient care [6]. Community treatment orders are designed to divert people from possibly hav© 2007 Bentham Science Publishers Ltd.

52 Current Psychiatry Reviews, 2007, Vol. 3, No. 1

ing to be admitted as inpatients. In addition, unlike leave, the individual may not have to meet the same criteria for treatment as an inpatient [6]. In Australia, it can include the power to force medication in the community [10]. By contrast, Canadian jurisdictions such as Ontario require patients, or their substitute decision-makers (usually a family member), to consent to compulsory community treatment [11]. These are not generally found elsewhere. Involuntary outpatient treatment or commitment is the preferred term in the United States and covers court-ordered community treatment [12]. In this case, a judge, not a health care professional, decides on the appropriateness of the order. The situation is complicated by the fact that in some jurisdictions different forms of community treatment such as extended release and involuntary outpatient treatment exist in parallel. Irrespective of classification, compulsory community treatment does not include interventions for forensic populations following criminal proceedings such as therapy for sex offenders. ARE WE ASSESSING THE SAME PATIENTS? There are also variations in the type of patients being placed on compulsory community treatment. Patients in jurisdictions where consent is needed for compulsory community treatment may be different from those who are placed on orders where consent is not required [11]. Some jurisdictions allow the use of community treatment orders in early episode presentations while others require prior evidence of treatment recidivism [11]. These are clearly very different patient groups. EFFICACY OR EFFECTIVENESS? Another issue is the trade-off between efficacy and effectiveness [13]. By efficacy we mean the evaluation of an intervention under ideal conditions such as rigorously conducted randomised controlled clinical trials. By effectiveness we mean an evaluation of practice as it happens in the clinical setting. This might include less restrictive inclusion criteria for subjects. Systematic studies to investigate the effects of CTOs are rare due to the inherent ethical, legal, and political conflicts involved, and we were only able to find two randomised controlled trials [14, 15], when we undertook a Cochrane systematic review in the area [3]. Both randomised controlled trials (RCTs) were of court-ordered outpatient commitment in the United States (New York and North Carolina) (Table 1). The results of these are equivocal, without definitive evidence of the effectiveness of the intervention in terms of health service use, quality of life or satisfaction for patients or caregivers [3]. In terms of the primary outcome measures, there were no significant differences between the intervention and control groups in admissions, bed-days or outpatient contacts in the 11-12 months following the intervention (Table 1). Neither did study subjects and controls differ significantly on medication adherence, functioning, overall arrest rate, arrests for violence, homelessness, or symptomatology over the 11-12 month follow-up period [3]. In the North Carolina study, patients on CTOs were less likely to have been victimized over the 12 month follow-up than control patients, but they were also more likely to report perceived coercion [16, 17]. The authors of the North Caro-

Kisely and Campbell

lina study did find that patients who had been on a CTO for more than 180 days had better outcomes than the control group, but this was not a randomly selected population. Rather, it may reflect a bias where the order was electively extended when it seemed to be helping the patient [18]. Furthermore, the study design meant that the subjects may not have been typical of patients who might be placed on this treatment. For obvious reasons, both excluded patients with a history of violence [14, 15, 19]. Additionally, both the RCTs were of court-ordered outpatient commitment in the United States, which is very different from clinicianinitiated treatment orders found in countries such as Scotland, Australia, New Zealand or Canada. Both studies had small numbers of participants, which was compounded by attrition rates that reached 55% in the New York study [3, 15]. In the study from North Carolina, the authors included a non-random sample of violent patients on compulsory community treatment in some of the analyses (Fig. 1) [19]. This further diluted the benefits of randomisation as these subjects had been excluded from the RCT, but were then compared with controls who did meet inclusion criteria. We therefore cannot exclude the possibility that the two groups were very different and that like was not being compared with like. ARE THERE ANY STUDIES OF EFFECTIVENESS? At the other extreme there are many naturalistic studies of subjects without controls who might be more typical of the type of patient who would be placed on compulsory community treatment. The weakness of uncontrolled designs is that it is difficult to determine the reason for any change in outcome. Aside from the potential effect of the intervention of interest, other possible explanations include regression to the mean, other interventions, life events or changes in social circumstances. These often overestimate the effect of the intervention of interest. Controlled studies without randomisation also have shortcomings, although guidelines have been developed to help address these [20]. One design is the controlled beforeand-after trial where subjects and controls are compared in terms of pre- and post-intervention data. Good quality CBAs have contemporaneous data collection and assess the comparability of intervention and control groups with appropriate statistical tests [20]. They ensure that both groups are assessed in the same way at baseline and follow-up using standardised measures and have high follow up rates [20]. We have attempted to meet these criteria in two CBA studies using administrative databases covering an entire jurisdiction [10, 21, 22] (Table 1). This has several advantages. All patients who are placed on such treatment can be studied, including patients with a history of dangerousness who are commonly excluded from other research designs. It is also possible to follow up almost all the intervention group and matched controls, other than those who die or leave the jurisdiction. Therefore, these studies are less vulnerable to selection and follow-up bias. In the first study, we assessed all patients placed on a community treatment order in Western Australia [10]. The introduction of community treatment orders in 1997 allowed

Compulsory Community Treatment

Current Psychiatry Reviews, 2007, Vol. 3, No. 1

53

Fig. (1). Attrition rates and consequent bias.

us to undertake a controlled before-and-after analysis of their effectiveness. Of the 313 patients who were preliminarily selected, seven died during the one year observation period after their index dates and 32 had not had a full year's contact with the mental health registry before their index dates, leaving 274 subjects. Depending on the variables for which we matched, we were able to find controls for between 83% [n=228] and 96% [n=265] of subjects. We selected a control group matched on gender and Aboriginal ethnicity, and with an age difference of less than 2

years. We also matched on diagnosis and inpatient status in the year prior to the index date. This included the date of previous discharge and the number of bed-days. We controlled for those factors that we did not match for in the selection of our controls with multivariate analyses using multiple, logistic or Cox regression models. These factors included rural versus metropolitan residence, outpatient contacts in the year preceding the index date, as well as the presence of psychiatric history beyond that time. We controlled for a past history of involuntary care including

54 Current Psychiatry Reviews, 2007, Vol. 3, No. 1

Table 1.

Kisely and Campbell

Evaluations of Compulsory Community Treatment Using RCT or CBA Designs 12 Month Outcome Study

Swartz et al. 1999* Wagner et al. 2003*

Type

RCT

Intervention

OPC

N

264

Admissions

Bed-Days

Outpatient Contacts

NS

NS

NS

Steadman et al. 2001

RCT

OPC

152

NS

NS**

NS

Preston et al. 2002

CBA

CTO

456

NS

NS

Significant increase

Kisely et al. 2005***

CBA

CTO

392

NS

NS

Not assessed

*Reporting one of several outcomes from the North Carolina RCT. **Data available on n=142. ***Subset of the intervention sample in Preston et al. 2002. RCT, randomised controlled trial; CBA, controlled before and after trial; OPC, court-ordered outpatient commitment; CTO, clinician initiated community treatment order.

suspended or conditional discharge [‘aftercare’]. We also considered psychiatric comorbidity, including substance use, in the year prior to the index date. Later analyses also controlled for forensic history as recorded in the Offenders’ database of Western Australia [21]. We found that although orders reduced admission rates and bed-days in the following year, the effect was no greater than that seen in the controls after adjustment for possible confounders. Fig. 2 shows the results of survival analyses to admission comparing CTO cases to controls matched on demographic characteristics (the matched control group (MCG)) or date of discharge (the consecutive control group (CCG)). The only area where they had an effect was to increase outpatient contacts in the subsequent 12 months.

of mental health services was free at the point of delivery, and services had similar characteristics in terms of staffing, as well as the balance of inpatient and outpatient care [22]. Importantly, neither had jurisdiction-wide assertive community treatment that could act as a confounding variable in assessing health service use. We again matched or controlled for most patient characteristics associated with CTO placement that could act as confounders. We found that while CTO placement was associated with more frequent and earlier readmission on survival analysis, the same CTO cases were less likely to have long hospital stays (100 days or longer) in the year after the index date than the Nova Scotian controls. One explanation for this difference might be that patients on a CTO are admitted earlier on in a relapse of their illness so avoiding the need for long admissions. This may be a threshold effect for admissions of over 100 days, as our Western Australian study did not show a reduction on mean lengths of stay compared to controls from within the State. However, we do not know if this is due to the increased intensity or coercive nature of the intervention. WHAT DOES THIS MEAN FOR PATIENTS?

Fig. (2). Cumulative survival rates in the community of the three cohorts (n=265 for the community treatment order (CTO) and matched control groups (MCG) groups, n=224 for the consecutive control groups (CCG) (source: Kisely et al., 2004 [21]).

We undertook a second study to address the issue of matching within a jurisdiction, by comparing two jurisdictions, one with CTOs (Western Australia), the other without (Nova Scotia) [22]. Although in different countries, the two jurisdictions had similar health services. In both, the delivery

The number needed to treat (NNT) is the number of patients who need to be treated to prevent one adverse outcome. In the case of compulsory community treatment, this might be readmission to hospital or an arrest. NNTs for effective treatments are usually between two and four [23]. If the results of the two RCTs were taken together, it would take up to 100 orders to prevent one readmission, 25 to prevent one episode of homelessness and 500 to prevent one arrest [14-16, 19, 24]. Using an intention to treat analysis produced lower, but still unacceptable NNTs [3]. These negative findings are not due to insufficient study power but are derived from our meta-analyses of over 400 subjects from both RCTs [3]. In a subsequent review, which combined both RCT and non-randomised trials [n=1108], we found that the evidence for involuntary outpatient treatment in reducing either admissions or bed-days remained very limited [5]. Even if it could be argued that admission rates are not the most appropriate outcome, the evidence on outcomes such as symptomatology, homelessness, quality of life, or perceived coerciveness is no better [3]. There are other concerns for patients. This legislation may stigmatise individuals with a severe mental illness, as many of these initiatives are named after a high profile victim of someone who happened to have a psychiatric illness.

Compulsory Community Treatment

New York has a 'Kendra's Law', California a 'Laura's Law', and Ontario a 'Brian's Law'. It could therefore be argued that involuntary outpatient treatment arises from, and propagates, the erroneous belief that people with mental illness are somehow more dangerous than the rest of society [26]. ARE THERE ALTERNATIVES? There are two alternatives to community treatment orders, both of which are less coercive and for which there is more evidence- advance directives and assertive community treatment. Advance directives are documents that convey a person’s preferences for treatment should the person become incompetent in the future and unable to do so for themselves. These appear to be most effective when the plan is developed at a meeting, lasting at least half an hour, and includes the treating team, patient and an invited relative [27, 28]. Less intensive forms, such as patient information booklets with statements for completion, appear less effective [29]. Assertive Community Treatment (ACT) is a team-based approach aimed at keeping ill people in contact with services, reducing hospital admissions and improving outcomes, especially social functioning and quality of life. A Cochrane review concluded that ACT, if correctly targeted, can substantially reduce the costs of hospital care whilst improving outcomes and patient satisfaction in previously high users of inpatient care [30]. As with advance directives, it is the intensity of care rather than its coercive nature that determines good outcome. Preliminary results from an Ontarian study of patients receiving ACT who are placed on a CTO, compared to those on a CTO without ACT indicate that it was the presence of ACT, not CTO placement, which reduced subsequent bed-days over the subsequent six months [31]. There was no statistical difference between the groups in terms of diagnosis, concurrent substance abuse disorder or legal involvement. However, these are preliminary results and the study would have been strengthened with the inclusion of an ACT only arm. Fig. 3 illustrates our hypothesis for a coercioneffectiveness curve for community treatment, which could be tested by future studies. Interventions that involve consultation with patients on their views on treatment, such as advanced directives for care involving detailed discussions with patients and their families on their treatment preferences should they relapse, may be the most effective. We suggest that with increasing levels of coercion, effectiveness diminishes. Examples of intervention in order of increasing coercion would include assertive community treatment, community treatment orders with consent as found in Ontario, and compulsory community treatment without patient agreement. Trials designed to directly compare these interventions could establish whether there are essential components common to all successful treatments, such as intensity of contact or patient participation in treatment planning, and if there is indeed a trade off between coercion and effectiveness (Fig. 3).

Current Psychiatry Reviews, 2007, Vol. 3, No. 1

However, there is debate whether an indicator that relates to the order itself (i.e. mandating patients to attend for followup) is sufficient evidence for the intervention in the absence of effects on symptomatology, homelessness, quality of life or time as an inpatient. More research is needed to establish the relative merits of the different interventions available, such as advance directives, assertive community treatment or the various forms of compulsory community treatment, rather than comparisons of individual approaches against treatment as usual. Ideally, research should be targeted to specific types of interventions for specific types of patients and take into account the characteristics of the laws involved.

Fig. (3). A possible coercion-effectiveness curve for community treatment.

Research could also assess whether the presence of compulsory community treatment might confer additional benefit when combined with other interventions through improved take up and compliance. Two studies have indicated that compulsory community treatment may enhance the effectiveness of depot medication, but the effect of confounders was not completely controlled for through either matching or randomisation [8, 31]. This finding merits further study. CBA comparisons of jurisdictions where new initiatives are introduced in one, but not the other, could offer an alternative when RCTs are impossible either ethically or practically. Clearly, all other aspects of service delivery in both jurisdictions would need to remain unchanged and comparable. Any new intervention including advance directives, assertive community treatment or compulsory community treatment could be evaluated in this way. REFERENCES [1]

CONCLUSIONS This paper has highlighted the methodological problems in assessing the impact of compulsory community treatment. Evidence for significant changes in admission or bed-days when compared to controls is very limited. Compulsory community treatment does increase outpatient contacts.

55

[2]

Salize HJ, Dressing H. Compulsory admission and involuntary treatment of mentally ill patients - legislation and practice in EU member states. Health and Consumer Protection DirectorateGeneral, European Commission. Brussels: European Commission, 2002. Available at: URL: http://www.mentalhealthalliance.org.uk/ resources/documents/salize.pdf (Accessed 20 September 2005). Lawton-Smith S. A Question of Numbers: The potential impact of community-based treatment orders in England and Wales. London: King’s Fund, 2005. Available at URL: http://www.kingsfund.org.

56 Current Psychiatry Reviews, 2007, Vol. 3, No. 1

[3]

[4] [5]

[6] [7]

[8] [9]

[10] [11] [12]

[13] [14]

[15] [16]

[17]

Kisely and Campbell

uk/resources/publications/a_question_of_1.html (Accessed 3 August 2006). Kisely S, Campbell LA, Preston N. Compulsory community and involuntary outpatient treatment for people with severe mental disorders. Cochrane Database Syst Rev 2005; Jul 20;3: CD004408. Dawson J, Romans S, Gibbs A, Ratter N. Ambivalence about community treatment orders. Int J Law Psychiatry 2003; 26: 243255. Kisely S, Campbell L, Scott A, Preston N, Xiao J. Randomised And Non-Randomised Evidence For The Effect Of Compulsory Community And Involuntary Outpatient Treatment On Health Service Use. Psychol Med (in press). Gray J, O’Reilly R. Clinically significant differences among Canadian mental health acts. Can J Psychiatry 2001; 46: 315-21. Sensky T, Hughes T, Hirsch S. Compulsory psychiatric treatment in the community I: A controlled study of compulsory community treatment with extended leave under the Mental Health Act: Special characteristics of patients treated and impact of treatment. Br J Psychiatry 1991; 158: 792-9. Torrey EF, Kaplan RJ. A national survey of the use of outpatient commitment. Psychiatr Serv 1995; 46: 779-84. Vaughan K, McConaghy N, Wolf C. Community treatment orders: relationship to clinical care, medication compliance, behavioural disturbance and readmission. Aust N Z J Psychiatry; 2000; 34: 801808. Preston NJ, Kisely S, Xiao J. Assessing the outcome of compulsory psychiatry treatment in the community: epidemiological study in Western Australia. BMJ 2002; 324: 1244-6. O’Reilly R. Does involuntary out-patient treatment work? Psychiatr Bull R Coll Psychiatr 2001; 25: 371-4. O’Reilly RL, Brooks SA, Chaimowitz GA, et al. Mandatory outpatient treatment. CPA Position Paper 2003-43. Ottawa: Canadian Psychiatric Association; 2003. Pittler MH, White A. Efficacy and effectiveness. Focus Altern Complement Ther 1999; 4: 109-10. Swartz MS, Swanson JW, Wagner HR, Burns BJ, Hiday VA, Borum R. Can involuntary outpatient commitment reduce hospital recidivism? Findings from a randomised trial with severely mentally ill individuals. Am J Psychiatry 1999; 156: 1968-75. Steadman HJ, Gounis K, Dennis D, et al. Assessing the New York City Involuntary Outpatient Commitment Pilot Program. Psychiatr Serv 2001; 52: 330-6. Hiday VA, Swartz MS, Swanson JW, Borum R, Wagner HR. Impact of outpatient commitment on victimization of people with severe mental illness. Am J Psychiatry 2002; 159: 1403-11. Swartz MS, Wagner HR, Swanson JW, Hiday VA, Burns BJ. The perceived coerciveness of involuntary outpatient commitment: findings from an experimental study. J Am Acad Psychiatry Law 2002; 30: 207-17.

Received: August 16, 2006

[18] [19]

[20] [21]

[22] [23]

[24]

[25]

[26] [27]

[28]

[29]

[30]

[31]

Szmukler G, Hotopf M. Effectiveness of involuntary outpatient commitment. Am J Psychiatry 2001; 158: 653-4. Swanson JW, Swartz MS, Wagner HR, Burns BJ. Involuntary outpatient commitment and reduction of violent behaviour in persons with severe mental illness. Br J Psychiatry 2000; 174: 324-31. EPOC- Effective Practice and Organisation of Care Group of the Cochrane Collaboration. Available at URL: http://www.epoc.uottawa.ca/. (Accessed 3 August 2006). Kisely SR, Xiao J, Preston NJ. Impact of compulsory community treatment on admission rates: survival analysis using linked mental health and offender databases. Br J Psychiatry 2004; 84: 432-8. Kisely S, Smith M, Preston N, Xiao J, An international comparison of health service use in two jurisdictions with and without compulsory community treatment Psychol Med 2005; 35: 1357-67. Bandolier Library. Number needed to treat (NNT). Bandolier Journal 1999; 59: 1 Available at URL: http://www.jr2.ox.ac.uk/bandolier/band59/NNT1.html#box3 (Accessed 3 August 2006). Compton SN, Swanson, JW, Wagner HR, Swartz MS, Burns BJ, Elbogen EB. Involuntary outpatient commitment and homelessness in persons with severe mental illness. Ment Health Serv Res 2003; 5: 27-38. Steadman HJ, Mulvey EP, Monahan J, et al. Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Arch Gen Psychiatry 1998; 55: 393401. Henderson C, Flood C, Leese M, et al. Effect of joint crisis plans on use of compulsory treatment in psychiatry: single blind randomised controlled trial. BMJ 2004; 329: 136. Kisely S. A joint crisis plan negotiated with mental health staff significantly reduces compulsory admission and treatment in people with severe mental illness, commentary. Evid Based Ment Health 2005; 8: 17. Papageorgiou A, King M, Janmohamed A, Davidson O, Dawson J. Advance directives for patients compulsorily admitted to hospital with serious mental illness: randomised controlled trial. Br J Psychiatry 2002; 181: 513-519. Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. Cochrane Database Syst Rev 2000; 2: CD001089. Swartz MS, Swanson JW, Wagner HR, Burns BJ, Hiday VA. Effects of involuntary outpatient commitment and depot antipsychotics on treatment adherence in persons with severe mental illness. J Nerv Ment Dis 2001; 189: 583-92. Freeland A, O’Brien A, Farrell S. The Impact of Community Treatment Orders on Patients with Assertive Community Treatment and Without. Proceedings of the 29th International Congress on Law and Mental Health, Paris, France, July, 2005, 86. Available at URL: http://www.ialmh.org/Paris2005/Abstract_Final.pdf (Accessed 1 September 2005).

Revised: October 18, 2006

Accepted: October 20, 2006