Copyright 2001 by the American Psychological Association, Inc. 0022-006X/01/S5.00 DOI: 10.1037//0022-006X.69.3.489
Journal of Consulting and Clinical Psychology 2001, Vol. 69, No. 3, 489-501
Does Competitive Employment Improve Nonvocational Outcomes for People With Severe Mental Illness? Robert E. Drake, Haiyi Xie, and Gregory J. McHugo
Gary R. Bond and Sandra G. Resnick Indiana University-Purdue University Indianapolis
Dartmouth Medical School
Richard R. Bebout Community Connections
The authors examined the cumulative effects of work on symptoms, quality of life, and self-esteem for 149 unemployed clients with severe mental illness receiving vocational rehabilitation. Nonvocational measures were assessed at 6-month intervals throughout the 18-month study period, and vocational activity was tracked continuously. On the basis of their predominant work activity over the study period, participants were classified into 4 groups: competitive work, sheltered work, minimal work, and no work. The groups did not differ at baseline on any of the nonvocational measures. Using mixed effects regression analysis to examine rates of change over time, the authors found that the competitive work group showed higher rates of improvement in symptoms; in satisfaction with vocational services, leisure, and finances; and in self-esteem than did participants in a combined minimal work-no work group. The sheltered work group showed no such advantage.
proven elusive with traditional vocational approaches (Bond, 1992; Bond, Drake, Becker, & Mueser, 1999). Given the modest improvement in employment outcomes, the secondary goal of improving nonvocational outcomes through employment has been largely a moot point. Recently, a new vocational approach, known as supported employment, has been developed. It involves a rapid, individualized search for community jobs tailored to each client's strengths and preferences, ongoing support on a time-unlimited basis, and close coordination between vocational and mental health treatment staff (Drake & Becker, 1996). For clients with SMI, supported employment has yielded substantially higher competitive employment rates than have traditional vocational services (Bond, Drake, Mueser, & Becker, 1997). Given these findings, the question of the impact on nonvocational outcomes is especially timely. Nevertheless, controlled research has failed to show significant improvement in nonvocational domains for supported employment clients, compared with clients receiving traditional vocational services (Bond et al., 1997). Two confounding factors may explain the lack of findings: First, even in the most successful supported employment programs, a significant proportion of clients are not working at any given time, and second, even without the assistance of effective vocational programs, some clients with SMI achieve competitive employment. If employment is the active ingredient for achieving better nonvocational outcomes, our analysis of nonvocational outcomes should focus on comparisons between employed and unemployed clients, rather than on enrollment in a supported employment program. Nonexperimental studies regarding the relationship between vocational and nonvocational outcomes have been ambiguous. Longitudinal studies of the course of schizophrenia have found only
Rates of competitive employment among people with severe mental illness (SMI) are abysmally low, with most studies reporting rates less than 15% (Anthony & Blanch, 1987). Over the past 5 decades, a variety of vocational rehabilitation approaches (e.g., preparation for employment through skills training, prevocational work crews, sheltered employment, and temporary jobs) has been developed to improve vocational functioning for this population (Bond, 1992). Beyond the obvious goal of increasing employment outcomes, most approaches also have aimed at the broader goal of community integration, including functioning outside the work place (Lehman, 1999). The underlying assumption has been that "work is therapy" (Black, 1988). In other words, working is assumed to benefit clients in nonvocational domains, for example, by increasing self-esteem, better controlling psychiatric symptoms, and improving quality of life. Unfortunately, the goal of increasing employment rates, particularly competitive employment, has
Gary R. Bond and Sandra G. Resnick, Department of Psychology, Indiana University-Purdue University Indianapolis; Robert E. Drake, Department of Psychiatry and Department of Community and Family Medicine, Dartmouth Medical School; Haiyi Xie, New Hampshire-Dartmouth Psychiatric Research Center, Dartmouth Medical School; Gregory J. McHugo, Department of Community and Family Medicine, Dartmouth Medical School; Richard R. Bebout, Community Connections, Washington, DC. This article was supported by Grant MH51346 from the Substance Abuse and Mental Health Services Administration and by Grants MH00439 and MH00842 from the National Institute of Mental Health. Correspondence concerning this article should be addressed to Gary R. Bond, Department of Psychology, Indiana University-Purdue University Indianapolis, 402 North Blackford Street, Indianapolis, Indiana 462023275. Electronic mail may be sent to
[email protected].
489
490
BOND ET AL.
modest correlations between different domains of functioning— including work and symptoms (Cook & Razzano, 2000; Gaebel & Pietzcker, 1987; McGlashan, 1988; Moller, von Zerssen, & Wuschner-Stockheim, 1982; Strauss & Carpenter, 1977). Other studies have examined the relationship between vocational and nonvocational outcomes cross-sectionally or with very brief follow-up periods (Arns & Linney, 1993, 1995; Brekke, Ansel, Long, Slade, & Weinstein, 1999; Brekke, Levin, Wolkon, & Sobel, 1993; Holzner, Kemmler, & Meise, 1998; Kemmler, Holzner, Neudorfer, Meise, & Hinterhuber, 1997; Scheid, 1993; Solinski, Jackson, & Bell, 1992). These studies have variously concluded that clinical functioning (usually assessed by severity of symptoms) predicts success in obtaining and holding a job or, conversely, that employment leads to better nonvocational outcomes, such as higher self-esteem, better quality of life, and reduced symptoms. Psychiatric symptoms have predicted poorer employment outcomes for clients enrolled in vocational programs (Anthony, Rogers, Cohen, & Davies, 1995; Hoffmann & Kupper, 1997; Lysaker & Bell, 1995). Some evaluations of vocational programs also have found that improvement in vocational functioning was associated with improvement in symptoms (Anthony et al., 1995; Bell & Lysaker, 1997; Bell, Lysaker, & Milstein, 1996). Finally, it is often noted that work can be stressful, and in fact, clinicians are often leery of encouraging clients to seek employment, for fear that they will be adversely affected (Blankertz & Robinson, 1996; Braitman et al., 1995; Marrone & Golowka, 1999). However, studies evaluating the conversion of day treatment programs to supported employment have not found increased rates of adverse clinical outcomes (e.g., hospitalizations, suicide attempts; Drake, 1998; Drake, Becker, Biesanz, Wyzik, & Torrey, 1996). Moreover, other studies have not found increased hospitalization rates for clients participating in employment programs (Bond, 1992; Bond et al., 1997). One major limitation in the literature has been the lack of clarity regarding the active ingredients explaining improved nonvocational outcomes as a result of working. One unanswered question concerns the type of employment that might have such benefits. Is competitive employment more potent than sheltered employment for improving nonvocational outcomes? On the basis of research in nonpsychiatric populations, Lamberti and Herz (1995) concluded that "employment in actual employment settings is better for developing self-esteem and competence in individuals than work in sheltered settings" (p. 725), a conclusion consistent with Estroff s (1981) ethnographic observations. Studies that examined the effects of sheltered work on nonvocational outcomes have been mixed (Bell & Lysaker, 1997; Bell et al., 1996; Dick & Shepherd, 1994; Griffiths, 1974; Kates, Lambrina, Baillie, & Hess, 1997; Whittington, Wilson, & Doherty, 1997). Another question not answered in cross-sectional designs is whether observed differences between unemployed and employed groups are a result of the negative effects of sustained unemployment and inactivity (e.g., demoralization, loss of hope; Hayes & Halford, 1996; Marrone & Golowka, 1999; Scheid, 1993; Warr, 1987) or the result of the positive effects of employment. If unemployment is a negative factor, we should see deterioration with continued unemployment; if employment is a positive influence, we should see improvement with continued employment. A final set of issues concerns the impact of the duration of employment. From longitudinal studies
of schizophrenia, we know change often occurs in small increments (Strauss & Carpenter, 1977). Although obtaining a job may provide an immediate boost in morale, a larger impact may be realized after a sustained period of employment. Thus, we hypothesize that the effects of employment are cumulative, such that the effects will be stronger the longer a person is working. Two randomized controlled trials of supported employment have examined nonvocational outcomes at multiple follow-up points for working and nonworking clients. One found that clients who were working had fewer psychiatric symptoms than those who were not working at each follow-up assessment (McFarlane et al., 2000). The supported employment model implemented in this study, however, required that clients be clinically stable before they entered competitive employment, so the findings may reflect a selective delay of entry into competitive employment for more symptomatic clients. No such restrictive criteria were used in a New Hampshire study comparing two supported employment approaches (Drake, McHugo, Becker, Anthony, & Clark, 1996). In a secondary analysis of this study, Mueser et al. (1997) examined the relationship of work to nonvocational outcomes over an 18-month period for 143 clients with SMI. At follow-up, clients who were working had lower rates of psychiatric symptoms, higher selfesteem, and greater satisfaction with finances and vocational services than unemployed clients did, even after controlling for baseline levels of functioning. The current article is a partial replication of the analysis conducted by Mueser et al. (1997). It is a secondary data analysis of a second randomized study of the Individual Placement and Support (IPS) model of supported employment, conducted in Washington, DC (Drake et al., 1999). Like the earlier New Hampshire study, the DC study showed substantially better competitive employment outcomes for clients assigned to IPS, compared with a control group receiving traditional vocational rehabilitation services. Although the goal of both conditions was to help participants obtain competitive employment in community settings, the control condition offered participants an array of sheltered workshop and work adjustment experiences as a first step toward competitive employment. Thus, the current study afforded an opportunity to examine the differential effects of sheltered and competitive work experiences on nonvocational outcomes. We hypothesized that among baseline characteristics, both work history and severity of symptoms would predict employment outcomes over the follow-up period. However, on the basis of the weak findings for these factors in the literature, we hypothesized that the size of these effects would be small. We did not expect to find differences on other client background variables. Our main hypothesis concerned the effects of work on nonvocational outcomes. We hypothesized that the cumulative effects of competitive employment would have a positive effect on nonvocational outcomes. Specifically, positive effects should be found only for those who achieve a sustained period of working competitively, rather than for those who work minimally or not at all. We predicted no differences between those working minimally and those working not at all. We had two secondary hypotheses regarding sheltered employment: (a) that sustained participation in sheltered work would result in better nonvocational outcomes than would minimal work or not working at all and (b) that competitive work would result in better nonvocational outcomes than would sheltered work. In other words, we presumed that sheltered work
WORKING AND NONVOCATIONAL OUTCOMES would have an intermediate effect between competitive and minimal or no work. Because of the hypothesized intermediate status for sheltered work, we further expected that the size of the effects for these latter two hypotheses would be small. On the basis of theory and empirical research, the prediction of an impact of working on nonvocational outcomes is more compelling in some domains than others. Thus we predicted that employment would improve symptoms (Mueser et al., 1997), quality of life (Fabian, 1989, 1992; Mueser et al., 1997), and self-esteem (Arns & Linney, 1993, 1995; Matthews, 1980; Mueser et al., 1997; Van Dongen, 1996, 1998). Because of assumptions made by many clinicians and because of their practical importance, several possible adverse outcomes were also examined. Working was not predicted to affect these areas, either positively or negatively. The literature on people with SMI suggests that working has no impact on rates of psychiatric hospitalizations (Bond, 1992; Bond et al., 1997; Crowther, Marshall, Bond, & Huxley, 2001) and that there are no associations between work and alcohol and drug use (Sengupta, Drake, & McHugo, 1998). Little is known about the relationship between working and homelessness (Kirszner, McKay, & Tippett, 1991) or between working and involvement in the criminal justice system, although one study found that criminal history was negatively correlated with future employment (Rogers, Anthony, Cohen, & Davies, 1997). Method
Study Participants Study participants were recruited at Community Connections, an intensive case management agency serving clients with SMI in southeast Washington, DC. Participants met the following inclusion criteria: SMI (i.e., a major mental illness, such as schizophrenia or bipolar or depressive disorder, and 2 years of major role dysfunction), unemployed at time of study admission, willingness to give informed consent, lack of medical or cognitive problems that would interfere with completing research interviews, and attendance at four informational groups. All clients at the agency were encouraged to attend informational groups explaining the study procedures and the two vocational approaches (Bebout, Becker, & Drake, 1998). Of the 309 clients receiving case management at the study site during the study enrollment period, 152 (49%) participated in the study. Two participants with incomplete vocational data and 1 participant who could not be classified on his pattern of employment (explained below) were excluded from the current analyses. Characteristics of the final sample of 149 participants are shown in Table 1.
Data Collection Procedures Data collection procedures are described elsewhere (Drake et al., 1999). Briefly, most outcome measures were obtained during client interviews by two bachelor-level research assistants who were independent of the clinical services. The research assistants were initially trained during a 2-day orientation by staff at the New Hampshire-Dartmouth Psychiatric Research Center and subsequently monitored by a clinical psychologist (R. R. Bebout) for the life of the project. For monitoring purposes, 36 clients were administered the same interview twice within a 2-week period. R. R. Bebout conducted all the diagnostic interviews.
Measures Used Background measures. Background measures included demographics, psychiatric diagnosis, which was determined on the basis of a diagnostic
491
interview using the Structured Clinical Interview for DSM-HI-R (Spitzer, Williams, Gibbon, & First, 1988) prior to study entry, and the Global Assessment Scale (GAS; Endicott, Spitzer, Fleiss, & Cohen, 1976). Outcome measures. Nonvocational outcomes were measured at baseline and at 6, 12, and 18 months after study admission. Overall life satisfaction, satisfaction with finances, leisure, vocational services, and financial support adequacy were measured with portions of the Quality of Life Interview (Lehman, 1983). Psychiatric symptoms were measured with the expanded (24-item) Brief Psychiatric Rating Scale (BPRS; Lukoff, Liberman, & Nuechterlein, 1986). In addition to the total BPRS score, we used five subscales derived from an unpublished factor analysis of 601 clients with SMI (McHugo, 1999): Affect, Anergia, Thought Disorder, Activation, and Disorganization. The total BPRS score (sum of all items) ranged from 24 to 168, whereas subscales (average of component items) ranged from 1.0 to 7.0, with higher scores indicating greater severity. A previous study of clients with SMI reported test-retest reliability coefficients as follows: Affect (.78), Anergia (.68), Thought Disorder (.93), Activation (.59), and Disorganization (.78; Drake, McHugo, et al., 1996). The Rosenberg Self-Esteem Scale is a 10-item self-report measure of global self-worth (Rosenberg, 1965). It is the most widely used self-esteem measure (Blascovich & Tomaka, 1991). Scores on the Rosenberg Scale range from 10 to 40, with lower scores indicating greater self-esteem. Reliability coefficients from previous studies of clients with SMI found internal consistency coefficients (Cronbach's alpha) exceeding .80 and test-retest reliability of .87 (Torrey, Mueser, McHugo, & Drake, 2000). Alcohol use and drug use were measured using clinician ratings on the Alcohol Use Scale (AUS) and Drug Use Scale (DUS; Drake et al., 1990). Participants' case managers were trained to complete these scales using multiple data sources to rate alcohol and drug use over the prior 6-month period. Both scales range from a low of 1 (abstinent) to a high of 5 (severe dependence), with a 3 or higher indicating a current substance abuse or dependence problem. Drake, Mueser, and McHugo (1996) reported AUS test-retest reliabilities close to 100%. They also reported agreement between raters (kappa coefficients) for the AUS and DUS ranging from .80 to .95. The number of days clients spent in psychiatric hospitals was tracked by Community Connections' management information system. Days spent in jail, in prison, or as homeless were measured by client self-report through direct questioning. These variables were assessed for the previous 1-year period at study entry and for the preceding 6-month period at each follow-up. Independent measure: Typology of work activity. Employment status and job type were tracked by vocational staff at the participating agencies and then verified through bimonthly research interviews with participants. Jobs were classified as either competitive, defined as regular community jobs, in integrated settings (i.e., with nondisabled coworkers), paying at least minimum wage, or sheltered, defined as paid work activity licensed by the Department of Labor in which workers are paid on a piece-rate basis. Most jobs fit unambiguously into one of these two categories. However, 6 participants were employed for a substantial amount of time in jobs secured through the National Industries for the Severely Handicapped (NISH) program (Black, 1988). NISH jobs have characteristics of both sheltered and competitive work and, therefore, are less clearly classifiable. The jobs are reserved for those with disabilities and, thus, are like sheltered work. However, these jobs were classified as competitive for the purposes of these analyses for the following reasons: (a) NISH positions involve working in community settings (e.g., offices of government agencies); (b) clients working these jobs receive wages commensurate with those in competitive employment, which are always above minimum wage; (c) clients' self-perceptions are that they are working "real" jobs, not protected jobs; and (d) coworkers are often people without disabilities. To test the hypothesis that differences in type of work activity would differentially affect nonvocational outcomes, participants were categorized into one of four groups on the basis of their cumulative employment
492
BOND ET AL.
Table 1 Background Characteristics by Group at Study Entry Characteristic Gender « (%) Female Male Race n (%) African American Other Marital status n (%) Never married Ever married Education n (%} ^ c ^ s
&1
•o
c
^
'c
1
u o o O
iS
1
1
•> ^
-j> ®
1
o- ®
^*~
™^
|
i> ^
•i^
"*
•£
S
^
1-3 g>|1 | 1 = 1 1 1 1 1 1 ° 1 1 t 2 %* 1 1 t 2 1 | %1 I 2 1i« o1
H O
O
u °>
^M
^
u 7
§
>• - • r
u '^
? •" ^ p rO
§ 5 ' 5 - ^
£
o
'
^
;
>• •— ii p rO ^
' . S
$
;
§ - ^
o
S -p ^
' " 5
t3
;
p rO j^
s - C
o
> -p
—
p
r^-* •£>
* *p -^i
p
' " 5 > 5 S ° o ° S ^ o
«
C6
r^
495
WORKING AND NONVOCATIONAL OUTCOMES oo
oo
R
00
an
CN
m
v>
-^t
m
2 "2
.2
S c
w^
o
•:- C _*•
C
TO
TO
O
4J
C M
CT3
O H^
^s o;r
^
r
3 s3
*a
("•4 1 G_
K
t
s
p
r--
is
ON (N
r-l in
O NO
ON ON
1
1
1
1
1
1
g N—ON
—;
^iz.
o
—
n
t^
-^t
0
O
"
O
—I
f-
•-i
'll 2 S 1wj _o o - ** "^
1
c
CO
•arNM %^t^ 3 5
3
21 'o^
C
i'lll S
3
1
S
S
3
r^l
r~~ VN
irO
O
—'
1
^ en tn
^ oo
r«
1
1—1
p
p
1
o
CO
in O 1
S3
1
tU
S
o
^
1
1 (N cs —'
1 ON xo —
O 00
o> i5 ^ =a ^ 6 """ O O
*- o
o c/5
„,
i 1«'» ?
1 -g 1 1 1
CQ
CQ
*c Q
CQ
CQ
Wore. Range of ns were work vs. no work; H2 = p comparison of competitiv " Effect size between com support adequacy. d Lov
BPRS total score0 No work Minimal work Sheltered work Competitive work Total BPRS Affecte No work Minimal work Sheltered work Competitive work Total
Outcome
Table 3 (continued)
cd _ aj CX -i.
•o t-
496
BOND ET AL.
of their total earnings for the study period. One participant could not be classified because of a 60-40% split in total earnings between competitive and sheltered employment and was therefore dropped from the analysis, as noted above. This typology resulted in four subgroups: competitive work (n = 31), with mean earnings of $5,799 (SD = $3,751); sheltered work (n = 24), with mean earnings of $3,350 (SD = $2,764); minimal work (n = 50), with mean earnings of $554 (SD = $417); and no work (i.e., no paid work over follow-up; n = 44). Group comparisons are shown in Table 2. Other indicators of duration of employment were consistent with the typology. For example, the competitive work group averaged 32 weeks of competitive employment over the 18-month follow-up period, compared with an average of less than 5 weeks for any of the other groups. Conversely, mean earnings from sheltered employment were 10 times greater for the sheltered work group than was the mean for any other group. Therefore, the typology successfully classified participants into distinct groups. The groups were similar on most background characteristics, as shown in Table 1. However, GAS scores were significantly higher for the competitive work group (M = 47.2) than for the minimal (M = 41.0) or sheltered (M = 39.6) groups, F(3, 148) = 3.87, p = .011. Also, in the 5 years prior to study intake, the groups differed in months competitively employed, F(3, 148) = 5.93, p = .001. Converting these data to an annualized rate, the competitive work group averaged 11.8 weeks per year in competitive employment over the preceding 5 years, compared with 7.6, 4.8, and 4.1 weeks per year, respectively, for the sheltered, minimal, and no work groups. Thus, most participants in all four groups were unemployed for a large proportion of time during the 5-year period prior to the study. We also examined whether the two conceptually distinct subgroups within the minimal work group—those who worked minimally in competitive work and those who worked minimally in sheltered work—differed on background measures or on any nonvocational outcomes during follow-up. Our analyses (not shown) indicated that they did not. Inspection of the response distributions for the outcome measures at baseline suggested that the quality of life measures, the symptom measures, and the self-esteem measure were reasonably well behaved and that these were suited for parametric analyses. However, five measures were highly skewed: those measuring alcohol use, drug use, homelessness, psychiatric hospitalizations, and time in jail. We therefore dichotomized each as either indicating the presence or the absence of the problem. We report the findings for these dichotomized measures first.
Chi-Square Analyses The baseline rates for the dichotomized variables were low, as shown in Table 1. As we previously noted, the groups did not differ at baseline on these variables. We then examined betweengroups differences in these rates at each of the three follow-up periods, using 2 X 2 chi-square analyses, with group comparisons constructed according to the study hypotheses. None of the 12 chi-squares was statistically significant. Overall, there was a slight tendency for the competitive work group to function better on these variables than the no work and minimal work groups did, both at baseline and thereafter. For example, across the 18-month
period, the rate of alcohol abuse ranged from 3% to 7% for the competitive work group and ranged from 11% to 20% for the no work-minimal work group. The sample as a whole did not improve over time on any of these measures, nor was there any evidence that any of the four subgroups improved. Therefore, we conclude that working did not have any impact on functioning in these areas.
Mixed-Effects
Regression Analyses
Means and standard deviations for the remaining nonvocational measures are shown in Table 3. At baseline, there were no significant differences between the groups on any of these variables. All groups showed significant changes over the course of the study on most outcomes, with significant time effects suggesting improvement on all of the quality of life measures and self-esteem, and worsening of symptoms on the total BPRS score and the Anergia subscale. As hypothesized, we found no differences between the no work and minimal work groups for any nonvocational measure, and so these two groups were combined for the remaining analyses. Contrary to prediction, there were no differences between the sheltered work group and the competitive work group (Hypothesis 2), nor were there any differences between the sheltered work group and the no work-minimal work group (Hypothesis 3). However, several measures showed differences between the competitive work group and the no work-minimal work group (Hypothesis 4). All of the differences were in the predicted direction. The strongest differences were seen with those variables most directly related to working. Over the 18-month period, the competitive work group was increasingly more satisfied with vocational services, ?(427) = 3.43, p = .0007; ES = 0.80, with their finances, t(433) = 3.00, p = .0029; ES = 0.80, and with their leisure activities, f(431) = 2.57, p = .011, ES = 0.60, than were those in the combined no work-minimal work group. Those working competitively showed a greater improvement in self-esteem, f(433) = -2.07, p = .039, ES = 0.19, than those not working or working minimally. Differences in the rates of improvement in symptoms favoring the competitive work group over the no workminimal work group were also observed. This was true for total BPRS symptoms, f(432) = -2.17, p = .031, ES = 0.70, as well as the Affect, ?(433) = -2.16, p = .032, ES = 0.48, and Disorganization, f(433) = -1.99, p = .047, ES = 0.54, subscales. To understand the patterns for individual measures, we graphed each of the means of outcome measures for the work groups over time. The graphs for the Rosenberg Self-Esteem Scale and for BPRS total scores are shown in Figures 1 and 2, respectively. Figure 1 suggests that self-esteem improved between baseline and 18 months for the competitive work group (within-group ES = 1.17), with relatively little net change for the other three groups (within-group ESs ranging from 0.14 to 0.47). By contrast, Figure 2 suggests that the competitive work group was unchanged on the BPRS total from baseline to 18 months (within-group ES = 0.21), whereas the no work group deteriorated substantially (ES = —1.12). The patterns for the remaining outcome measures were variable. For example, the competitive work group had large within-group ESs reflecting improvement in self-esteem (ES = 1.17), overall life satisfaction (ES = 0.90), and satisfaction with finances (ES = 1.45), leisure (ES = 0.61), and services
497
WORKING AND NONVOCATIONAL OUTCOMES - * • no work > minimal work A sheltered work —•— competitive work
22.0 21.5 21.0 20.5 20.0 19.5 19.0 18.5 en 18.0 17.5 -
4
17.0 X Baseline
6 months
12 months
18 months
Figure 1. Group means on Rosenberg Self-Esteem Scale over the course of the study. Lower scores indicate greater self-esteem.
(ES = 2.24), whereas the ESs for the symptom subscales were generally smaller: Affect (ES = 0.46), Thought Disorder (ES = 0.27), Activation (ES = 0.00), and Disorganization (ES = 0.59), with one ES in the direction of worsening symptoms, Anergia (ES = —0.44). Averaging across the eight outcome measures listed in Table 3 (using only the BPRS total for the symptom domain), the within-group ESs for the no work, minimal work, sheltered work, and competitive work groups were 0.02, 0.01, 0.19, and 0.74, respectively.
Discussion Clients with SMI who worked in competitive employment for an extended period of time showed a greater rate of improvement in several nonvocational outcomes, which partially replicated the findings in Mueser et al. (1997), who found improvement in some of the same nonvocational domains. Perhaps the most significant area of convergence in the two studies was in reduction in psychiatric symptoms. Because of the lack of baseline differences and
—^ —no work - • - minimal work
46 i
A
sheltered worik
•
competitive work
44 42 H 40
M
34 -
32 1 Baseline Figure 2. severity.
6 months
1 12 months
1 18 months
Group means on Brief Psychiatric Rating Scale (BPRS) total scores. Higher scores indicate greater
498
BOND ET AL.
the statistical model used, we believe the findings are not simply a result of better functioning clients being more successful in achieving and maintaining competitive work. Instead, we conclude that this study offers preliminary support for the view that competitive employment nurtures positive personal changes outside the work domain. However, this conclusion must be qualified. For some of the findings, the improvement in the competitive work group was significant, whereas in other cases, the statistical difference reflected deterioration in the minimal-nonworking group. Studies have frequently not distinguished between these two scenarios. Accordingly, we recommend that future studies routinely indicate whether differences are a result of the improvement with working or of deterioration as a result of not working. The current study also offers a refinement over most previous research by suggesting that the impact of employment on nonvocational outcomes is evident only for an extended period of employment, not merely for exposure to work. Research should examine whether longer periods of employment result in further incremental gains in other life domains, as one would expect intuitively. As has been widely found in studies of people with SMI (Bond et al., 1997), we found most competitively employed clients in the current study worked part time, and some held a series of shortterm jobs. This pattern of partial employment is consistent with the stress tolerance level for this population, as well as with strategies clients use to avoid the disincentives of the disability benefits system, which reduce benefits when earnings exceed specified limits (Walls, Dowler, & Fullmer, 1990; Warner & Polak, 1995). Regarding the widely held assumption among clinicians that employment may cause stress and therefore lead to poorer nonvocational outcomes, there was no evidence that the competitive work group, on average, deteriorated over time in any outcome domain. The single exception was a slight increase in BPRS Anergia. The major thrust of the findings, including those for psychiatric hospitalization, substance use, and total symptoms, is that working competitively generally does not adversely affect people with severe mental illness. Consistent with most prior research (Anthony & Jansen, 1984; Bond, 1992; Mueser et al., 1997; Rogers et al., 1997), client background characteristics mostly did not predict employment outcomes in this study. The fact that symptoms at baseline did not predict later employment outcomes is at variance with some of the literature, although studies are not consistent on this point. We speculate that symptoms are less of a barrier in programs that assertively help clients find jobs, as was true for clients receiving IPS services in this study. We also note that the study participants were generally less symptomatic than clients in many of the published studies examining this relationship. Given the mounting evidence that negative symptoms are especially predictive of poorer vocational functioning in people with schizophrenia (Hoffmann & Kupper, 1997; Lysaker & Bell, 1995), it is of interest that the BPRS factor of Anergia (blunted affect, emotional withdrawal, and motor retardation) did not predict employment outcome. One set of comparisons not previously examined in prior research concerned the differential impact of competitive and sheltered work. In the parent study, Drake et al. (1999) found that the experimental conditions participants had similar earnings from employment, with the difference being that virtually all of the employment outcomes for the IPS condition were in competitive
employment, whereas virtually all of the employment outcomes for the comparison condition were in either sheltered employment or NISH jobs. Thus, one might argue that both conditions were equally successful with regard to work outcomes, because both increased employment, and that the only difference was the type of employment. However, the pattern of results in the current study suggests a distinctive advantage of programs that focus on competitive employment. Although the direct comparisons between the competitive work and sheltered work groups were not significant, the mean within-group ES for the competitive work group was large (0.74), whereas it was negligible for the sheltered work group (0.19). Moreover, there was no evidence that even an extended period of sheltered employment resulted in better nonvocational outcomes than did not working or working very little. Thus, IPS programs may be more likely to yield improved nonvocational outcomes for clients who achieve employment than are programs that provide sheltered work. It is important to note that this study investigated only one type of sheltered employment, namely, piece-rate employment in a workshop setting. In addition to sheltered employment, psychiatric rehabilitation programs offer many different protected employment opportunities, including transitional employment (Macias, Kinney, & Rodican, 1995), agency-run businesses (Chandler, Levin, & Barry, 1999), mobile work crews operated by a rehabilitation agency in which clients work in community settings (Schultheis & Bond, 1993), and many other arrangements that resemble competitive employment in some respects and sheltered employment in other respects (Bond et al., 1999; Shimon & Forman, 1991). The hypothesis forwarded in this study, that sustained employment in competitive work leads to better nonvocational outcomes, does not identify the active ingredients—Is it payment of a decent wage? Is it working in an integrated work environment? Is it the fact that the client "owns" a job? Is it the fact that the job is permanent? Studies should examine whether the apparent advantage for competitive work over sheltered work can be extended to other forms of protected employment, such as agency-run businesses and work enclaves. Such research will test a key premise of diversified placement approaches, which hypothesizes that meaningful activity is more important than the specific type or location of work (Bond et al., 1999). Assuming that working competitively increases self-esteem and improves clinical functioning, what is the mechanism of change for the individual? Is the primary factor the change in role status with employment, so that a client reinterprets his or her identity to incorporate a more positive self-image (Lysaker & France, 1999)? Does the structure provided by the routine of working and by the work environment help combat symptoms (Marrone & Golowka, 1999)? Or is it the socialization that occurs with the integration into the work place that is critical (Gates, Akabas, & Oran-Sabia, 1998; Mank, Cioffi, & Yovanoff, 1997)? Probably a variety of mechanisms account for the findings, with no single explanation holding for all people at all times. Another factor that prevents a simple interpretation of the effects of working is the idiosyncratic course of psychiatric disorders. Strauss, Hafez, Lieberman, and Harding (1985) argued that the course of recovery in schizophrenia is characterized by moratoriums, "change points," and other nonlinear patterns. The group data in Figures 1 and 2 mask individual variation, but it is clear that
499
WORKING AND NONVOCATIONAL OUTCOMES the patterns in this study are not simple linear trajectories. Also, a more long-term perspective than the 18-month follow-up period in our study may be required to see patterns. A final comment concerns the clinical significance of the study findings. It has become increasingly standard practice for psychotherapy researchers to include information on normative comparisons of their outcomes (Jacobson, Roberts, Berns, & McGlinchey, 1999; Kendall, Marrs-Garcia, Nath, & Sheldrick, 1999). In principle, our study could have attempted to document what percentage of clients in each group exceeded some normative criterion. Unfortunately, the psychiatric rehabilitation field has lagged behind psychotherapy research in thinking through the issue of clinical significance and has not yet begun to address the thorny question of what might constitute adequate comparison groups. Given the nonlinear course of psychiatric illnesses, cutoff scores based on general population norms (when they are available) might very well represent a misapplication of the concept of clinical significance that experts in this area have warned us against (Jacobson et al., 1999; Kendall et al., 1999). Further complicating the picture is the suggestion that there may legitimately be instances in which a clinically significant change can be said to have occurred even when there has been no change in symptoms (Kazdin, 1999). Given the lack of conceptualization of clinical significance for the study population, we have opted not to pursue this methodology in the current report, although we urge consideration of it in future work. One limitation of our study concerns the relatively brief follow-up period. Although longer than many studies, 18 months may be shorter than the time needed to observe changes in target domains (Gerden, 1998). The mean duration of employment for the competitive work group was 32 weeks, which is less than optimal for fully testing the study hypothesis. A second limitation is the modest sample sizes for the four groups, limiting statistical power. A third limitation concerns the relatively large number of outcomes examined. Although our findings are bolstered by a priori hypotheses (and our partial replication of the Mueser et al., 1997, study), an inflated Type I error rate (Wilkenson & Task Force, 1999) nonetheless is a concern. A fourth limitation is that statistical significance is not the same as clinical importance. For example, despite significant differences in symptom ratings, most participants experienced subclinical levels of symptoms throughout the follow-up period. Floor effects for some clinical variables may have precluded the detection of any improvement. A fifth limitation concerns the questionable validity of the self-esteem measure (Torrey et al., 2000). A sixth limitation concerns the sampling characteristics. Study participants were inner-city clients, primarily African American, and all indicated an interest in employment and received excellent vocational services. Although drawn from a severely disabled population, the participants' severity of symptoms and level of substance abuse were low. The findings may not generalize beyond this context. A further limitation of our study is the absence of subanalyses for demographic subgroups. For example, given the gender difference in the course of schizophrenia (McGlashan, 1988) and the fact that men and women view employment differently (Cook & Roussel, 1987; Goering, Cochrane, Potasznik, Wasylenki, & Lancee, 1988), the relationships between working and nonvocational outcomes may differ for men and women. Further inquiry is
warranted in samples with sufficient statistical power to address these questions adequately. Finally, as in any correlational study, we must be cautious in imputing causation. There is no experimental way to answer the question of the impact of work on nonvocational outcomes, because not everyone afforded the chance to work actually works. Therefore, the best way to determine the impact of work on nonvocational outcomes may be to examine a pattern of results over a series of studies. The fact that the groups differed little on background characteristics increases our confidence that the findings are not merely an artifact of selection biases in which higher functioning participants were more successful in holding competitive jobs as a result of greater control of symptoms prior to study entry.
References Anthony, W. A., & Blanch, A. (1987). Supported employment for persons who are psychiatrically disabled: An historical and conceptual perspective. Psychosocial Rehabilitation Journal, 11(2), 5-23. Anthony, W. A., & Jansen, M. A. (1984). Predicting the vocational capacity of the chronically mentally ill: Research and implications. American Psychologist, 39, 537-544. Anthony, W. A., Rogers, E. S., Cohen, M., & Davies, R. R. (1995). Relationships between psychiatric symptomatology, work skills, and future vocational performance. Psychiatric Services, 46, 353-358. Arns, P. G., & Linney, J. A. (1993). Work, self, and life satisfaction for persons with severe and persistent mental disorders. Psychosocial Rehabilitation Journal, 17(2), 63-79. Arns, P. G., & Linney, J. A. (1995). Relating functional skills of severely mentally ill clients to subjective and societal benefits. Psychiatric Services, 46, 260-265. Bebout, R. R., Becker, D. R., & Drake, R. E. (1998). A research induction group for clients entering a mental health research project: A replication study. Community Mental Health Journal, 34, 289-295. Becker, B. J. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology, 41, 257278. Bell, M. D., & Lysaker, P. H. (1997). Clinical benefits of paid work activity in schizophrenia: 1-year followup. Schizophrenia Bulletin, 23, 317-328. Bell, M. D., Lysaker, P. H., & Milstein, R. M. (1996). Clinical benefits of paid work activity in schizophrenia. Schizophrenia Bulletin, 22, 51-67. Black, B. J. (1988). Work and mental illness: Transitions to employment. Baltimore: Johns Hopkins Press. Blankertz, L., & Robinson, S. (1996). Adding a vocational focus to mental health rehabilitation. Psychiatric Services, 47, 1216-1222. Blascovich, J., & Tomaka, J. (1991). Measures of self-esteem. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 115-160). San Diego, CA: Academic Press. Bond, G. R. (1992). Vocational rehabilitation. In R. P. Liberman (Ed.), Handbook of psychiatric rehabilitation (pp. 244-275). New York: Macmillan. Bond, G. R., Drake, R. E., Becker, D. R., & Mueser, K. T. (1999). Effectiveness of psychiatric rehabilitation approaches for employment of people with severe mental illness. Journal of Disability Policy Studies, 10(1), 18-52. Bond, G. R., Drake, R. E., Mueser, K. T., & Becker, D. R. (1997). An update on supported employment for people with severe mental illness. Psychiatric Services, 48, 335-346. Braitman, A., Counts, P., Davenport, R., Zurlinden, B., Rogers, M., Clauss, J., Kulkarni, A., Kymla, J., & Montgomery, L. (1995). Comparison of
500
BOND ET AL.
barriers to employment for unemployed and employed clients in a case management program: An exploratory study. Psychiatric Rehabilitation Journal, 19(1), 3-18. Brekke, J. S., Ansel, M., Long, J., Slade, E., & Weinstein, M. (1999). Intensity and continuity of services and functional outcomes in the rehabilitation of persons with schizophrenia. Psychiatric Services, 50, 248-256. Brekke, J. S., Levin, S., Wolkon, G. H., & Sobel, E. (1993). Psychosocial functioning and subjective experience in schizophrenia. Schizophrenia Bulletin, 19, 599-608. Chandler, D., Levin, S., & Barry, P. (1999). The menu approach to employment services: Philosophy and five-year outcomes. Psychiatric Rehabilitation Journal, 23(1), 24-33. Cook, J., & Razzano, L. (2000). Vocational rehabilitation for persons with schizophrenia: Recent research and implications for practice. Schizophrenia Bulletin, 26, 87-103. Cook, J. A., & Roussel, A. E. (1987, August). Who works and what works: Effects of race, class, age, and gender on employment among the psychiatrically disabled. Paper presented at the meeting of the American Sociological Association, Chicago. Crowther, R. E., Marshall, M., Bond, G. R., & Huxley, P. (2001). Helping people with severe mental illness to obtain work: Systematic review. British Medical Journal, 322, 204-208. Dick, N., & Shepherd, G. (1994). Work and mental health: A preliminary test of Warr's model in sheltered workshops for the mentally ill. Journal of Mental Health, 3, 387-400. Drake, R. E. (1998). A brief history of the Individual Placement and Support Model. Psychiatric Rehabilitation Journal, 22(1), 3-7. Drake, R. E., & Becker, D. R. (1996). The Individual Placement and Support Model of Supported Employment. Psychiatric Services, 47, 473-475. Drake, R. E., Becker, D. R., Biesanz, J. C., Wyzik, P. F., & Torrey, W. C. (1996). Day treatment versus supported employment for persons with severe mental illness: A replication study. Psychiatric Services, 47, 1125-1127. Drake, R. E., McHugo, G. J., Bebout, R. R., Becker, D. R., Harris, M., Bond, G. R., & Quimby, E. (1999). A randomized clinical trial of supported employment for inner-city patients with severe mental illness. Archives of General Psychiatry, 56, 627-633. Drake, R. E., McHugo, G. J., Becker, D. R., Anthony, W. A., & Clark, R. E. (1996). The New Hampshire study of supported employment for people with severe mental illness: Vocational outcomes. Journal of Consulting and Clinical Psychology, 64, 391-399. Drake, R. E., Mueser, K. T., & McHugo, G. J. (1996). Clinical rating scales: Alcohol Use Scale (AUS), Drug Use Scale (DUS), and Substance Abuse Treatment Scale (SATS). In L. I. Sederer & B. Dickey (Eds.), Outcome assessment in clinical practice (pp. 113-116). Baltimore: Williams & Wilkins. Drake, R. E., Osher, F. C., Noordsy, D. L., Hurlbut, S. C., Teague, G. B., & Beaudett, M. S. (1990). Diagnosis of alcohol use disorders in schizophrenia. Schizophrenia Bulletin, 16, 57-67. Endicott. J., Spitzer, R. L., Fleiss, J. L., & Cohen, J. (1976). The Global Assessment Scale: A procedure for measuring overall severity of psychiatric disturbance. Archives of General Psychiatry, 33, 766-771. Estroff, S. (1981). Making it crazy. Berkeley: University of California Press. Fabian, E. S. (1989). Work and the quality of life. Psychosocial Rehabilitation Journal, 12(4), 39-49. Fabian, E. S. (1992). Supported employment and the quality of life: Does a job make a difference? Rehabilitation Counseling Bulletin, 36, 84-97. Gaebel, W., & Pietzcker, A. (1987). Prospective study of course of illness in schizophrenia: Part II. Prediction of outcome. Schizophrenia Bulletin, 13, 299-306. Gates, L. B., Akabas, S. H., & Oran-Sabia, V. (1998). Relationship
accommodations involving the work group: Improving work prognosis for persons with mental health conditions. Psychiatric Rehabilitation Journal, 27(3), 264-272. Gerden, J. F. (1998). Do long-term treatments alter lifetime course? Lessons learned, actions needed. Journal of Psychiatric Research, 32, 197-199. Goering, P., Cochrane, J., Potasznik, H., Wasylenki, D., & Lancee, W. (1988). Women and work: After psychiatric hospitalization. In L. L. Bachrach & C. Nadelson (Eds.), Treating chronically mentally ill women, (pp. 45-61). Washington, DC: American Psychiatric Press. Griffiths, R. D. (1974). Rehabilitation of chronic psychotic patients. Psychological Medicine, 4, 316-325. Hayes, R. L., & Halford, W. K. (1996). Time use of unemployed and employed single male schizophrenic subjects. Schizophrenia Bulletin, 22, 659-669. Hoffmann, H., & Kupper, Z. (1997). Relationships between social competence, psychopathology and work performance and their predictive value for vocational rehabilitation of schizophrenic outpatients. Schizophrenia Research, 23, 69-79. Holzner, B., Kemmler, G., & Meise, U. (1998). The impact of work-related rehabilitation on the quality of life of patients with schizophrenia. Social Psychiatry & Psychiatric Epidemiology, 33, 624-631. Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, 300-307. Kates, N., Lambrina, N., Baillie, B., & Hess, J. (1997). An in-home employment program for people with mental illness. Psychiatric Rehabilitation Journal, 20(4), 56-60. Kazdin, A. E. (1999). Meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332-339. Kemmler, G., Holzner, B., Neudorfer, C., Meise, U., & Hinterhuber, H. (1997). General life satisfaction and domain-specific quality of life in chronic schizophrenic patients. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 6, 265-273. Kendall, P. C., Marrs-Garcia, A., Nath, S. R., & Sheldrick, R. C. (1999). Normative comparisons for the evaluation of clinical significance. Journal of Consulting and Clinical Psychology, 67, 285-299. Kirszner, M. L., McKay, C. D., & Tippett, M. L. (1991, October). Homelessness and mental health: Replication and adaptation of the PACT model in Delaware, Proceedings from the Second Annual Conference on State Mental Health Agency Services Research (pp. 68-81). Arlington, VA: National Association of State Mental Health Program Directors Research Institute. Lamberti, J. S., & Herz, M. I. (1995). Psychotherapy, social skills training, and vocational rehabilitation in schizophrenia. In C. L. Shriqui & H. A. Nasrallah (Eds.), Contemporary issues in the treatment of schizophrenia (pp. 713-734). Washington, DC: American Psychiatric Press. Lehman, A. F. (1983). The well-being of chronic mental patients. Archives of General Psychiatry, 40, 369-373. Lehman, A. F. (1999). Developing an outcomes-based approach for the treatment of schizophrenia. Journal of Clinical Psychiatry, 6"0(Suppl. 19), 30-35. Lipsey, M. W. (1990). Design sensitivity. Newbury Park, CA: Sage. Lukoff, K., Liberman, R., & Nuechterlein, K. (1986). Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophrenia Bulletin, 12, 578-602. Lysaker, P., & Bell, M. (1995). Negative symptoms and vocational impairment in schizophrenia: Repeated measurements of work performance over six months. Acta Psychiatrica Scandinavica, 91, 205-208. Lysaker, P. H., & France, C. M. (1999). Psychotherapy as an element in supported employment for persons with severe and persistent mental illness. Psychiatry, 62, 209-220.
WORKING AND NONVOCATIONAL OUTCOMES Macias, C., Kinney, R., & Rodican, C. (1995). Transitional employment: An evaluative description of Fountain House practice. Journal of Vocational Rehabilitation, 5, 151-158. Mank, D., Cioffi, A., & Yovanoff, P. (1997). Analysis of the typicalness of supported employment jobs, natural supports, and wage and integration outcomes. Mental Retardation, 35, 185-197. Marrone, J., & Golowka, E. (1999). If work makes people with mental illness sick, what do unemployment, poverty, and social isolation cause? Psychiatric Rehabilitation Journal, 23(2), 187-193. Matthews, W. C. (1980). Effects of a work activity program on the self concept of chronic schizophrenics. Dissertation Abstracts International, 41, 358-359. (University Microfilms No. 816281). McFarlane, W. R., Dushay, R. A., Deakins, S. M., Stastny, P., Lukens, E. P., Toran, J., & Link, B. (2000). Employment outcomes in familyaided assertive community treatment. American Journal of Orthopsychiatry, 70, 203-214. McGlashan, T. H. (1988). A selective review of recent North American long-term followup studies of schizophrenia. Schizophrenia Bulletin, 14, 515-542. McHugo, G. (1999). PRC factors on the 24-item BPRS. Lebanon: New Hampshire-Dartmouth Psychiatric Research Center. Moller, H.-J., von Zerssen, D., & Wuschner-Stockheim, M. (1982). Outcome in schizophrenic and similar paranoid psychoses. Schizophrenia Bulletin, 8, 99-108. Mueser, K. T., Becker, D. R., Torrey, W. C., Xie, H., Bond, G. R., Drake, R. E., & Dain, B. J. (1997). Work and nonvocational domains of functioning in persons with severe mental illness: A longitudinal analysis. Journal of Nervous & Mental Disease, 185, 419-426. Rogers, E. S., Anthony, W. A., Cohen, M., & Davies, R. R. (1997). Prediction of vocational outcome based on clinical and demographic indicators among vocationally ready clients. Community Mental Health Journal, 33, 99-112. Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press. Scheid, T. L. (1993). An investigation of work and unemployment among psychiatric clients. International Journal of Health Services, 23, 763782. Schultheis, A. M., & Bond, G. R. (1993). Situational assessment ratings of work behaviors: Changes across time and between settings. Psychosocial Rehabilitation Journal, 17(2), 107-119. Sengupta, A., Drake, R. E., & McHugo, G. J. (1998). The relationship between substance use disorder and vocational functioning among persons with severe mental illness. Psychiatric Rehabilitation Journal, 22(0,41-45. Shimon, S. M., & Forman, J. D. (1991). A business solution to a rehabilitation problem. Psychosocial Rehabilitation Journal, 14(4), 19-22. Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models,
501
hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323-355. Solinski, S., Jackson, H. J., & Bell, R. C. (1992). Prediction of employability in schizophrenic patients. Schizophrenia Research, 7, 141-148. Spitzer, R., Williams, J., Gibbon, M., & First, M. (1988). Structured Clinical Interview for DSM-HI-R-Patient Version (SCID-P). New York: Biometrics Research Department, New York State Psychiatric Institute. Strauss, J. S., & Carpenter, W. T. (1977). Prediction of outcome in schizophrenia: III. Five-year outcomes and its predictors. Archives of General Psychiatry, 34, 159-163. Strauss, J. S., Hafez, H., Lieberman, P., & Harding, C. M. (1985). The course of psychiatric disorder: III. Longitudinal principles. American Journal of Psychiatry, 142, 289-296. Torrey, W. C., Mueser, K. T., McHugo, G. H., & Drake, R. E. (2000). Self-esteem as an outcome measure in studies of vocational rehabilitation for adults with severe mental illness. Psychiatric Services, 51, 229-233. Van Dongen, C. J. (1996). Quality of life and self-esteem in working and nonworking persons with mental illness. Community Mental Health Journal, 32, 535-548. Van Dongen, C. J. (1998). Self-esteem among persons with severe mental illness. Issues in Mental Health Nursing, 19, 29-40. Walls, R. T., Dowler, D. L., & Fullmer, S. L. (1990). Incentives and disincentives to supported employment. In F. R. Rusch (Ed.), Supported employment: Models, methods, and issues (pp. 251-269). Sycamore, IL: Sycamore Publishing. Warner, R., & Polak, P. (1995). The economic advancement of the mentally ill in the community: 2. Economic choices and disincentives. Community Mental Health Journal, 31, 477-492. Warr, P. (1987). Work, unemployment, and mental health. New York: Oxford University Press. Whittington, D., Wilson, R., & Doherty, H. (1997). The effectiveness of the Industrial Therapy Organization.' The executive report on a three year study. Belfast, Northern Ireland: University of Ulster. Wilkenson, L., & Task Force. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 94604. Wolfinger, R., & Chang, M. (1995). Comparing the SAS GLM and MIXED procedures for repeated measures. SAS Institute Inc. SUGI Proceedings. Available: http://www.sas.com/rnd/app/papers/papers_da.html [1999, September 13].
Received March 2, 2000 Revision received August 7, 2000 Accepted September 3, 2000
