A User-Derived Evaluation Framework for Mental Health Services ...

A User-Derived Evaluation Framework for Mental Health Services: Does routine outcome measurement satisfy the objectives of service users?

Final report to:

The Australian College of Mental Health Nurses

March 2007.

Professor Brenda Happell, Central Queensland University, 2007.

1

Contents Contents .............................................................................................................................. 2 Project Information ............................................................................................................. 5 Executive Summary ............................................................................................................ 6 1.

2

Background ................................................................................................................. 8 1.1

Overview ............................................................................................................. 8

1.2

Project aims ......................................................................................................... 8

1.3

Methodological approach.................................................................................... 9

1.4

Report structure ................................................................................................... 9

Literature Review...................................................................................................... 10 2.1

Introduction ....................................................................................................... 10

2.2

Routine Outcome Measurement in Victorian Mental Health Services ............. 10

2.3

Policy ................................................................................................................ 10

2.4 The adequacy of data collection, outcome measures and quality control for monitoring and evaluating mental health services ........................................................ 11

3

2.4.1

Problems with measures used. .................................................................. 11

2.4.2

The quality of collected data. .................................................................... 15

2.4.3

The use made of collected data. ................................................................ 15

2.5

Australian research into routine outcome assessment ...................................... 15

2.6

Consumer perspectives on consumer self-report measures .............................. 16

2.7

Evaluation of mental heath services ................................................................. 17

2.8

Promoting recovery........................................................................................... 19

2.9

Summary ........................................................................................................... 20

Stage 1 – Formation of the Reference Group and PAT Discussions ........................ 21 3.1

Methodology ..................................................................................................... 21

3.2

Methods – Stage 1 ............................................................................................. 21

3.2.1

Establishment of the Reference Group ..................................................... 21

3.2.2

Ethics......................................................................................................... 22

3.2.3

Recruitment ............................................................................................... 22

3.2.4

Group discussion between PAT group members ...................................... 23

3.2.5

Analysis of the group discussion ............................................................... 23

3.3

4

Findings............................................................................................................. 23

3.3.1

Assumptions behind ROM ......................................................................... 23

3.3.2

Consumer concerns with the use of ROM ................................................. 25

3.3.3

Consumer Perspective: purpose, process and principles ......................... 26

3.3.4

Summary of Findings ................................................................................ 30

Stage 2 -Mental Health Service User Focus Groups ................................................ 32 4.1

Methodology ..................................................................................................... 32

4.2

Method .............................................................................................................. 32

4.2.1

Recruitment of focus group participants................................................... 32

4.2.2

Consumer focus groups............................................................................. 32

4.2.3

Analysis of consumer focus groups ........................................................... 33

4.3

Findings............................................................................................................. 33

4.3.1

Aspects of service delivery that aid recovery ............................................ 33

4.3.2

Aspects of service that hinder recovery .................................................... 38

4.3.3 Principles or values that should underpin the evaluation of mental health services ………………………………………………………………………….. 45 4.4

Summary ........................................................................................................... 48

5

Discussion and Conclusions ..................................................................................... 49

6

Outcomes .................................................................................................................. 54


3

6.1

New Research Directions .................................................................................. 54

6.2

Collaboration with other organisations ............................................................. 55

6.3

Dissemination of Study Findings ...................................................................... 55

6.3.1

The project report ..................................................................................... 55

6.3.2

Refereed publications and conference presentations ............................... 55

7

Acknowledgements ................................................................................................... 57

8

References ................................................................................................................. 58

9

Appendix 1 – Psych Action and Training (PAT)...................................................... 64 9.1

Description of the PAT Group .......................................................................... 64

9.2

PAT Philosophy ................................................................................................ 64

9.3

What are the PAT group’s criteria? .................................................................. 64

9.4

What can PAT do? ............................................................................................ 64

9.5

Education and Training: .................................................................................... 65

9.6

Research: ........................................................................................................... 65

9.7

Evaluation: ........................................................................................................ 66

9.8

Consumer Participation..................................................................................... 66

9.9

First Project – Offering training opportunities to Consumer Educators ........... 66

10

Appendix 2 – Project Reference Group ................................................................ 67 Project summary ............................................................................................... 67

10.2

Aims of the Reference Group ........................................................................... 67

10.3

Operation of the Reference Group .................................................................... 67

11

10.1

Appendix 3 – Discussion Stimulus for PAT Discussions..................................... 68


4

Project Information Project title: A User-Derived Evaluation Framework for Mental Health Services: Does routine outcome measurement (ROM) satisfy the objectives of service users? Year of grant: 2004 Investigator/s: Professor Brenda Happell Ms Cath Roper Ms Karla Gough Organisations:. The Centre for Psychiatric Nursing, School of Nursing, University of Melbourne. School of Nursing and Health Studies Central Queensland University. Address for correspondence: School of Nursing and Health Studies Central Queensland University Bruce Highway Rockhampton, 4701 Queensland. Email: [email protected]


5

Executive Summary Background The introduction of routine outcome measures (ROM) in Victorian public mental health services has been criticised for not including domains of interest to consumers. The aims of this project were to review and evaluate questionnaires currently used by public mental health services throughout Victoria to assess the impact services have on the health outcomes of service users and to identify alternative user-derived strategies for service evaluation that can be used alongside existing methods currently used by public mental health services. Methodology and Method A qualitative framework was adopted for this study to provide consumer participants the opportunity to voice their views and opinions on this subject. Furthermore, a qualitative approach offered flexibility to enable the precise focus of the research to be shaped determined by the participants. Stage 1 Two group discussions were held between consumer members of Psychiatric Action and Training (PAT) group members, moderated by the consumer academic. This discussion was audio-tape recorded, transcribed verbatim, and then content analysed. The findings from this stage influenced a change of focus towards recovery and service evaluation for the second stage of the research. Stage 2 Two focus groups were conducted with consumers as participants and with facilitation provided by a consumer researcher and nurse researcher. One group was conducted in a metropolitan area and one in a rural area of Victoria, Australia. The participants were asked to describe the aspects of service delivery that assisted with recovery; aspects that hindered recovery; and the values and principles that would ideally underpin the evaluation of mental health services. Findings Stage 1 Three main themes were identified from the data: (1) assumptions behind ROM, for example that positive changes in consumers’ health and well-being are the result of service delivery, (2) concerns with the use of ROM, that they may be used Professor Brenda Happell, Central Queensland University, 2007.

6

(inappropriately) to determine access to services and the treatment provided, and (3) consumer participation: purpose, process and principles as it relates to the evaluation of services. Stage 2 The concept of recovery was identified as important to consumers, and factors were identified as promoting recovery, as were factors that hindered recovery. Prominent themes were social support as helping recovery and poor access to staff and facilities hindering recovery. The principles and processes that should underpin the evaluation of mental health services were identified as consumer involvement in a more rigorous evaluation; peer support; and, more responsive care and treatment Conclusions The findings of this study support the available literature in suggesting some significant limitations to the concept of measuring the outcomes of mental health service delivery in a manner which is useful and meaningful for consumers. ROM were considered as not reflecting or promoting the principles of recovery, which is of greater importance to consumers than the measurement of symptoms and other factors measured by ROM. Rather than seeking an alternative, consumer-driven outcome measure, preference was for more meaningful and genuine opportunities for consumers to participate in the evaluation of mental health services, and the adoption of a recovery focus within the service paradigm.


7

1. Background 1.1

Overview

There is a clear need for local strategies developed by service users to evaluate the utility and effectiveness of mental health services (Oades, Viney, Malins, Strang, & Eman, 2005). Evidence indicates that domains of primary interest to consumers are not assessed by outcome measures currently utilised by Victorian public mental health services, specifically the consumer-rated Basis-32 (Campbell, 1998; Fossey & Harvey, 2001; Graham et al., 2001; Miller, Siggins, Kavanagh, & Donald, 2003), where an outcome is defined as the effect on a patient's health status that is attributable to an intervention (Andrews, Peters, & Teesson, 1994). Moreover, findings from consultancies commissioned by the Victorian government suggested the development of alternative strategies to assess consumer outcomes and emphasised the need for consumer involvement and collaboration in the development of these strategies (Graham et al., 2001; Miller et al., 2003). In response to the need for user-derived strategies, the Centre for Psychiatric Nursing Research and Practice sought and received funding from the Australian and New Zealand College of Mental Health Nurses Research Board to undertake a research project in collaboration with the Psychiatric Action and Training Group (the PAT Group). The PAT Group is a group of consumer and senior psychiatric nurse educators who aim to promote consumer perspectives in the professional development activities of mental health sector (More information about PAT is included in Appendix 1). 1.2

Project aims

The aim of the project was to address the following research questions: 1. To review and evaluate questionnaires currently used by public adult mental health services throughout Victoria to assess the impact services have on the health outcomes of service users. 2. To identify alternative user-derived strategies for service evaluation that can be used alongside existing methods currently used by public mental health services. The aims of this project were achieved across two inter-related stages. The first stage comprised a literature review, including government policy and reports, and a series of group discussions with the consumer members of the PAT group. Findings from the first stage informed the development of the second stage, which specifically addressed the second project aim; that is, to explore alternative user-derived strategies for service evaluation.


8

1.3

Methodological approach

A qualitative methodological approach was adopted for this study for two main reasons. Firstly qualitative exploratory methodology is particularly appropriate for studies where little is known about the phenomena under investigation (Elliot & Hayes, 2003). In this case while there is literature to suggest that ROM do not meet the needs of consumers, there is limited research which focuses specifically on consumer views of ROM or alternative user-derived strategies that would more appropriately reflect what they consider important (research question two). Secondly, a qualitative framework allows the direction of the research to be shaped by the views and opinions of the participants (Beanland, Schneider, LoBiondo- Wood & Haber, 1999). This is particularly important in the case of research involving consumers, as they have traditionally been the subjects of, rather than actively shaping, mental health research (Telford and Faulkner, 2004). Indeed, following the completion and analysis of data from PAT discussions, a change of focus in the research direction became evident. The intention to primarily discuss the assessment of consumers (using outcome measures) moved more towards the evaluation of mental health services and providers and the role of consumer input in evaluation processes. The PAT discussions suggested that this topic was of greater importance to service users than further exploration of outcome measures would be. To examine this issue, two focus group discussions were conducted with service users from one metropolitan and one rural public area mental health service. 1.4

Report structure

The remainder of this report describes the literature review and group discussions. Specifically, salient findings from the literature review are presented in Section 2. Section 3 provides a description of the method and results relevant to the group discussions with the consumer members of the PAT Group. Section 4 provides a description of the method and results relevant to the focus group discussions with service users. Finally, Section 5 summarises the findings from both stages and discusses them in the context of policy and research relevant to routine outcome measurement in Victorian mental health services.


9

2 2.1

Literature Review

Introduction

An extensive review of the literature was conducted in 2003 (Miller, Siggins, Kavanagh, & Donald, 2003) and it was not an aim of this project to replicate this process. Rather the focus for the review specifically related to the literature describing the usefulness and value of routine outcome measurement (ROM) in reflecting the impact mental health services have on the health outcomes of service users, as determined by the service users themselves. At the completion of the first stage of the research, the focus for the second stage moved from ROM specifically to a broader consideration of recovery and service evaluation. A second part to the literature review was subsequently undertaken to provide background information for these issues. 2.2

Routine Outcome Measurement in Victorian Mental Health Services

The introduction of routine standardised outcome measurement to Australian mental health facilities has been controversial (Coombs & Meehan, 2003; 2005; Lakeman, 2004). Proponents of outcome measures suggest that their use may facilitate consumer expression of meanings and experiences of care, may encourage clinicians to assess domains that they commonly overlook, provide useful information to non-ward staff (e.g., educators, managers, researchers), assist consumer-staff communication, and may promote staff self-reflection on practice (Coombs & Meehan, 2005). Coombs and Meehan (2003) suggest the findings of ROM will assist clinicians to monitor changes in the outcomes of individual consumers specifically, and more broadly will assist with monitoring outcomes for certain groups of consumers. Although outcome measures would ideally enable these benefits, those currently in use may lack the relevance, precision and depth to be of real use to practitioners and it is argued that their use has not been shown to improve consumer outcomes (Lakeman, 2004). The literature suggests the outcome measures currently used by Victorian public mental health services do not assess domains that are of primary interest to consumers (Campbell, 1998; Fossey & Harvey, 2001; Graham, Coombs, Buckingham, Eagar, Trauer & Calally, 2001; Miller, et al., 2003). Authors of reports written for the Victorian government suggested the development of alternative strategies to assess consumer outcomes and emphasised the need for consumer involvement and collaboration in the development of these strategies (Graham et al., 2001; Miller et al., 2003). 2.3

Policy

Developing consumer outcome measures is on the agendas of Commonwealth and State Governments. This policy direction arose from the perceived need for the “… provision of sound information to support planning and service delivery” (Pirkis, Burgess, Kirk, Professor Brenda Happell, Central Queensland University, 2007.

10

Dodson & Coombs, T. 2005: 2), to ensure “the continued improvement of the quality and effectiveness of the treatment of people with a mental illness” (Pirkis et al., 2005: 2). The implementation of ROM is considered an important strategy in ensuring this goal is realised. In 1992, Commonwealth, State and Territory Health Ministers agreed on the National Mental Health Strategy. In this strategy, five information-related priorities were set, including the development of consumer outcome measures for routine use in mental health facilities (Department of Health and Aged Care, 1999). The relevant objectives of the Strategy are: •

To undertake regular reviews of outcomes of services provided to people with serious mental health problems and mental health disorders, as a central component of the delivery of mental health services, and

•

To encourage the development of national outcomes standards for mental health services, and systems for assessing whether services are meeting these standards.

To achieve these objectives, routine outcome measures have been introduced throughout Australian public mental health services. Although the measures used to assess outcomes vary from state to state, all Australian states and territories have signed Information Development Agreements that require them to submit outcome and case-mix data, which is gathered on a routine basis, to the Australian Government. 2.4

The adequacy of data collection, outcome measures and quality control for monitoring and evaluating mental health services

Whilst it is important to acknowledge the efforts of the State and Federal Governments with respect to the monitoring of outcomes, there is a range of problems associated with the standardised measures used to assess outcomes and the quality and use made of collected data. 2.4.1

Problems with measures used.

Currently in Victoria, three standardised measures are used in routine clinical practice for monitoring the outcomes of individual consumers. These are the Health of the Nations Scale (HoNOS), the Life Skills Profile (LSP-16) and the Behaviour and Symptom Identification Scale (BASIS-32). When evaluated against 12 principles broadly applicable to outcome assessment (Smith, Manderscheid, Flynn, & Steinwachs, 1997), these measures perform relatively poorly. The following table outlines each of the principles and provides an evaluation of the routine measures against these principles.


11

Table 1 – Evaluation of Routine Outcome Measures against Principles of Patient Outcome Assessment Principle of patient outcomes assessment

Outcome assessments should be appropriate to the application or question being answered

Tools for assessing outcomes should have demonstrated validity and reliability and must be sensitive to clinically important change over time

Outcomes assessments should always include the patient’s perspective; outcomes assessments obtained from providers and family members may enhance what is learned

Evaluation of Australian measures against principle

“To understand the relationship between patients’ health status (outcomes), disease status, and treatment (processes of care) … disorder-specific assessment tools are needed” (Smith et al., 1997: 1034).

Requirements of principle met

No

The tools currently employed by Australian services are generic tools not disorder-specific assessments. A review of ROM found a reasonable level of validity and reliability (although most had problems with specific items), and were sensitive to change over time. It was noted that most studies of the LSP used the LSP-39 and further analysis of the LSP-16 was required (Pirkis et al., 2005). However, the extent to which observed change is the result of service delivery is disputed. For example: “My team has routinely used the HoNOS and the long form of LSP for several years. When viewing the graphed scores over time for some individuals, one can perceive no discernable pattern. It is hard to see how the aggregation of such data if meaningless at the individual level could have any possible usefulness to a service … commonly used outcome measures … fail to capture the significance of particular issues for individuals” (Lakeman, 2004: 211). The BASIS-32 is currently used to garner the patient’s perspective; however, a report on the views of consumers regarding this measure indicated that consumers had strong reservations regarding “the suitability of the current measure to meet their needs and the capacity of mental health services to engage in ‘good process’” (Graham et al., 2001: 1). Most importantly, the BASIS-32 does not assess domains of interest or aspects of mental health care and service provision that are important to service users and their carers (Fossey & Harvey, 2001; Graham et al., 2001; Miller, Siggins, Kavanagh, & Donald, 2003; Noble & Douglas, 2004).

Uncertain

No

No information is routinely gathered from family members. Outcomes assessment systems should place minimal burden on the respondent and have the ability to be adapted to different health care

The administration time for each of the measures currently employed by Australian mental health services is short; hence, they place minimal burden on the respondent.

Yes

Routine outcome assessment tools are highly adaptable to different health care systems


12

Principle of patient outcomes assessment



systems Outcomes assessments should include general health status as well as mental health status

HoNOS includes an item on physical illness and disability, LSP-16 makes reference to neglect of physical health and adequacy of diet. Basis-32 does not refer to general health

Partially

Despite the implementation of routine outcome measures throughout Australian public and private mental health facilities, there are no acceptable, systematic means through which consumers can provide feedback on the services they receive. We acknowledge that some services do employ consumer satisfaction surveys to obtain consumer feedback; however, there are significant doubts regarding the validity and utility of such surveys. Specifically, consumer satisfaction surveys consistently produce positive results irrespective of the quality of the service being rated (Gill, Pratt, & Librera, 1998; Oades, Viney, Malins, Strang, & Eman, 2005).

No

Outcomes assessment tools should quantify the type and extent of treatment the patient receives (the process of care) for the target condition in order to understand the clinical relationship between the outcomes of care and treatment

The outcome assessment tools currently employed in Australia do not quantify the type and extent of treatment the patient receives (the process of care) for the target condition.

No

Outcome assessment tools should include generic and disorderspecific information that is predictive of expected patient outcomes; this prognostic information may include case mix and severity characteristics that are associated with choice of or success of treatment

The outcome assessment tools currently employed in Australia do not include disorder-specific information that is predictive of expected patient outcomes.

No

Outcomes assessments should include areas of personal functioning affected by the condition

ROM include areas of personal functioning but this is not relate to the condition or conditions of interest

No

Outcomes assessments should include measures of the patient’s evaluation of treatment and outcomes


13

Principle of patient outcomes assessment



or conditions of interest

Outcomes should be initially assessed and reassessed at clinically meaningful points in time given the course of the disorder

Outcomes assessments should use an appropriate scientific design and representative sample

Assessing outcomes of patients who prematurely leave treatment is as important as assessing outcomes of those who are still in treatment at the time of follow-up.

The current measures should be completed “on admission, review (every 91 days) and at discharge” (Coombs & Meehan, 2003: 163). These points were chosen because they fit with “current clinical practice (information is already collected at these time points) and “enables the monitoring of consumer progress as individuals move through the different service settings” not because they correspond to clinically meaningful points in time given the course of the disorder.

No

A recent review reported in the British Journal of Psychiatry indicated that “large-scale studies using ‘humongous’ databases are largely achieved at the expense of clinically meaningful outcomes” (Gilbody, House, & Sheldon, 2002: 14). Services assume that changes in the outcomes assessed can be attributed to the services received, but is this a fair assumption? A number of other changes may occur in an individual’s life at the same time that they are receiving services. “There are so many possible factors that may influence ratings that changes can rarely be attributed to any specific intervention” (Lakeman, 2004: 211). Unless services specifically chart the events and circumstances of each individual consumer and map these against noted changes, they cannot state with any certainty the reasons for these changes. The difficulty in accessing people who have left treatment prior to discharge means that their experiences are generally not reflected in routine outcome measurement. This would ultimately skew the results as the results for this group of patients could be significantly different to those who remain in treatment to the point of discharge. While the problems of reaching those no longer in treatment are acknowledged, unless a strategy is put in place to attempt to include these people, this principle has not been met.


No

No

14

Although national and state policy documents (Commonwealth of Australia, 2003; Department of Human Services, 2002) emphasise the importance of consumer participation, particularly in the planning, monitoring and review of mental health services, to date, there is little evidence that consumers are meaningfully and substantially involved in the development or implementation of the methods and procedures used in the monitoring and review of services. 2.4.2

The quality of collected data.

There are widespread problems with the rates of completion, i.e., response rates are poor for a number of these measures. Therefore, there would be significant issues associated with the interpretation of analyses performed on the available data, if that were to occur. 2.4.3

The use made of collected data.

The information being gathered routinely is not being used appropriately and/or to its full extent, although it is acknowledged that this varies between States and Territories. There is a view that simply measuring outcomes, in and of itself, does not assist with the improvement of consumer outcomes (Lakeman, 2004). This view was supported by the results of a randomised controlled trial in the UK (Slade, McCrone, Kuipers, Leese, Cahill, Parabiaghi, Priebe & Thornicroft, 2006). The study found cost savings primarily associated with reduced inpatient admissions for the experimental group, but no difference in outcomes as perceived by the consumers themselves. The information gathered needs to be converted into a digestible format (clear, constructive feedback) before it is of any use to services and the consumers who they aim to assist. At present, the time and resources spent on assessing outcomes does not realise any specific benefits to either consumers or clinicians and this will continue unless governments and/or services are actually prepared to facilitate the analysis and interpretation of routine outcome assessment data. While the analysis of data has commenced (Pirkis et al, 2005) as yet there is no clear pathway identified to ensuring this information is received by consumers and clinicians. 2.5

Australian research into routine outcome assessment

In Australia, research on the consumer development of outcomes measures is in its infancy. Andrews, Peters, and Teesson (1994) measured the attitudes of mental health professionals, consumers and carers throughout Australia towards outcome assessment. The authors conducted a review of existing measures, and developed recommendations regarding the development and implementation of national consumer outcome measures. Based on their consultations and literature review, Andrews et al. recommended five measures for further field testing to determine the feasibility of their use nationwide and the adequacy of their psychometric properties. The five measures were the Resource Associated Functional Level Scale (RAFLS; McPheeters, 1984), Behavior and Symptom Identification Scale 32 (BASIS-32; Eisen, Dill, & Grob, 1994), Health of the Nation Outcome Scales (HoNOS; Wing, 1994), Medical Outcomes Study Short Form (SF-36; Ware & Sherbourne, 1992), and the Mental Health Inventory (MHI; Veit & Ware, 1983). Professor Brenda Happell, Central Queensland University, 2007.

15

An additional measure, the Life Skills Profile (LSP; Rosen & Parker, 1989) met minimum criteria for use, i.e., it was applicable, acceptable, practical, reliable, valid and sensitive to change but was not recommended on the grounds of cost. Field testing of the recommended consumer outcome measures was undertaken in 1996 (Stedman, Yellowlees, Mellsop, Clarke, & Drake, 1997). Although the results of the field tests indicated that all six selected measures were valid, reliable, and useful when used in different settings, Stedman et al. (1997) concluded: We simply do not know enough about the utility of these measures, nor enough about the proposed uses of a ‘national’ measure, to conclusively recommend any of the six selected measures as a national measure for the routine assessment of consumer outcomes in mental health. More work needs to be undertaken to investigate the utility of these measures, as well as alternative measures and systems for outcomes assessment (p. 5). Further to the field testing, Stedman et al. (1997) emphasised that the implementation of consumer outcome measures throughout Australian mental health services should also be subject to review; specifically, the ability of selected measures to meet local needs required further assessment. Moreover, any consultation should include consumers or their representatives. 2.6

Consumer perspectives on consumer self-report measures

Following the implementation of routine outcome measures throughout Australia, the Victorian government commissioned two reports focussing on consumer self-rated instruments. In the first report, Graham et al. (2001) examined consumers views about the BASIS-32 and the overall process of routine outcome measurement (Graham et al., 2001). In the second report, Miller et al. (2003) compared consumer self-rated instruments in use throughout Australia, identified international best practice for outcome measurement and recommended a unified national approach (Miller et al., 2003). Findings from both reports highlighted substantial problems with existing measures. The outcomes that service users “regard as valuable to record include but go far beyond the responses sought by the existing choices” (Miller et al., 2003, p. 12). The authors of both reports recommended the development of one or more additional measures covering domains of relevance to consumers, emphasising the importance of substantial consumer (and carer) involvement and collaboration with researchers in the development of these measures. A recent Scottish study has highlighted the value in collaborating with consumers in such development (Hunter et al., 2004). Hunter et al. found that an outcome measure that service users had devised in partnership with health professionals was more likely to detect problems as assessed by service users and key-workers than the outcome measure that Victorian public mental health services currently employ (Hunter et al., 2004).


16

The literature reviewed and findings from the first stage of the current project indicate the need for substantial consumer involvement in the development of alternative strategies to assess the effect, both positive and negative, that public mental health services have on the health status of service users. As stated by Miller et al. (2003) If the purpose of measuring consumer outcomes is to improve the safety and quality and effectiveness of the treatment system then measures need to be designed that reflect those aspects of outcome most important to the consumers of such treatments (p. 12). In summary, the development of consumer outcome measures has been a priority area for Commonwealth and State Governments for over a decade. Although valid and reliable measures have been developed, consumers have not had an integral part in this process. Evidence suggests that at least one outcome measure that was developed in consultation with consumers (Hunter et al., 2004). In order that they are meaningful for consumers and responsive to their needs, outcome measures need to be developed in collaboration with consumers. If such an instrument is to be developed, it is important to examine the role of consumers in the evaluation of mental health services, and in particular to explore what aspects of care consumers perceive to be important to their recovery. 2.7

Evaluation of mental heath services

The change of focus from the assessment of consumers (using outcome measures) to recovery and the evaluation of mental health services meant that additional literature was now required to inform the second stage. This involves discussion of the evaluation of psychiatric facilities and, in particular, the role of consumers in such evaluations. Secondly, in order to promote consumer input into the evaluation of mental health facilities, an understanding of what helps and hinders recovery from mental health conditions is necessary background information. Over the last two decades there has been an international movement towards the empowerment of consumers and their carers in the provision of mental health services (Goodwin & Happell, 2006; Lammers & Happell, 2004a; Lammers & Happell, 2004b; Weinstein, 2006). Australian Government policy clearly articulates consumer participation as a priority area (Commonwealth of Australia 1992; 1997; 1998). Indeed, one of the 11 National Standards for Mental Health Services comprises a complete section devoted to the promotion and implementation of consumer and carer participation in mental health services. Among the requirements of services in this domain is the inclusion of consumers and carers at every level of Mental Health Services, including evaluation (Commonwealth of Australia, 1997). The increased emphasis on consumer participation has not yet been accompanied by an increase in research to determine the extent to which the goals have been realised in practice. The findings from a qualitative study undertaken by Tobin, Chen and Leathley (2002) found that consumer participants had experienced very limited opportunities for participation at either an individual or systemic level. More varied responses were found in a qualitative study undertaken by Lammers and Happell (2004a). In this study, some Professor Brenda Happell, Central Queensland University, 2007.

17

consumer participants described their opinions being valued by health care professionals. However, many others identified that a number of barriers restricted the degree of effective participation that could occur. The attitudes of health professions towards the inclusion of consumers were identified as a major barrier. In terms of evaluation more specifically, there is mixed evidence as to whether the rhetoric of greater consumer involvement has been matched with their active participation. The review of the literature suggests a degree of involvement is evident (Chapko, Borowsky, Fortney, Hedeen, Hoegle, Maciejewski, 2002; Eales, Callaghan, & Johnson, 2006; Howard & El-Mallakh, 2001; Poulton, 1999; Weinstein, 2006). In the Eales et al. (2002) evaluation of a liaison mental health service, for example, consumers (n = 17) were interviewed, along with professional stakeholders, about what aspects of the service were important to them. Howard and El-Mallakh (2001) described how former consumers were trained to collect data from consumers using a survey tool and face-to-face interviews. The researchers perceived that involving former consumers in the evaluation would increase the validity of the data because participants would be more likely to provide open and honest feedback than would likely be the case if the interviews were conducted by health professionals. A point consistent with the views of Gill, Pratt and Liberera (1998). However, the involvement of the consumers in these examples has been limited. Essentially, they have been used either as sources of data and data-collectors rather than having active involvement in the conception and design of the evaluation, as well as in the analysis and interpretation of the data. This would suggest there is still some considerable way to go before active consumer involvement in mental health service evaluation is achieved. A large-scale project was undertaken in New South Wales to develop a framework to facilitate consumer evaluation of mental health services (Viney, Oades, Strang, Eman, Lambert, Malins, Morland, K., & Tooth, 2004). Consumer involvement was fundamental to this project at all stages as stated in the report: The difference between this project and other projects, and what makes it unique, is that the consumers were not only employed and trained as researchers and collaborators, they actively drove the development of the Evaluation Framework … Such an approach requires privileging the knowledge that comes from the lived experience of mental illness and hence valuing this ‘expert’ knowledge in a similar vain to ‘expert’ professional and academic knowledge (p.3). The consumers involved in the project were trained as researchers and provided with adequate support to ensure that their potential for involvement could be fully realised. A subsequent project also conducted in New South Wales known as MHCopes (NSW Department of Health 2006), developed a four stage framework to facilitate consumer participation in quality improvement within mental health services. The stages included: Professor Brenda Happell, Central Queensland University, 2007.

18

• • • •

Data collection using the MH-CoPES questionnaire Data analysis Reporting and feedback to stakeholders including managers, clinicians, consumers and carers Action and change. In this important stage, consumers and providers work together to enact the changes identified as necessary during the first three stages.

As a cyclical framework it remains ongoing to ensure quality improvement remains responsive to consumer identified needs and issues. The benefits of this approach go beyond the development of the framework. The successful outcomes of the MH-CoPES project, clearly demonstrate the capacity for consumers to have an active role in the evaluation of mental health services. 2.8

Promoting recovery

The concept of recovery has been promulgated by the consumer movement as an alternative way to consider the impact of mental health services to the outcome measurement approach (Ralph, 2000). The importance of recovery for people experiencing a mental illness was described by Deegan (1996) as: … rooted in the simple yet profound realisation that people who have been diagnosed are human beings”. Acknowledging the ‘humanness’ of service users provides a necessary starting point for appreciating their right and capacity to be actively involved in shaping and determining the outcomes of services for themselves as individuals. As part of a review of the recovery literature, Ralph (2000) presented a number of definitions from consumers as to what recovery means to them. Although commonalities were evident, a recurring theme indicated the unique and individual nature of recovery for each person who seeks to attain it. Indeed this is reflected in the following quote by Anthony (1993: 13): … a deeply personal, unique process of changing one’s attitudes, values, feelings, and goals, skills, and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with the limitations caused by illness. Recovery involves the development of new meaning and purpose in one’s life. Recovery has increasingly become an important focus of mental health service policy and delivery (Borg & Kristiansen, 2004; Davidson, O’Connell, Tondora, Styron & Kangas, 2006; Reberio Gruhl, 2005; Resnick, Rosenheck & Lehman, 2004). However despite this change of focus there is limited evidence about what helps and hinders the recovery of consumers with mental health conditions (Jacobson & Greenley, 2001; Mancini, Hardiman, & Lawson, 2005). In their conceptual model of recovery, Jacobson and Greenley described internal and external conditions of consumers that facilitate recovery. Professor Brenda Happell, Central Queensland University, 2007.

19

Conditions internal of consumers include empowerment, connection, healing, and hope, whereas external conditions include a positive culture of healing, implementation of the principle of human rights, and recovery-oriented services. This distinction between internal and external conditions, however, seems imposed and does not appear to be useful to either research or practice. Also, the list of conditions seems short. Mancini et al. (2005) identified several barriers to, and facilitators of, recovery through their interviews with 15 consumer-providers of mental health care. Barriers to recovery included staff paternalism, coercion, indifferent and judgemental professionals, and medication side effects and psychiatric symptoms. Facilitators of recovery were supportive relationships, engagement in meaningful activities, and access to alternative and formal treatments. Factors, such as these, may need to be included in evaluations of mental health services. 2.9

Summary

While policy documents refer to the right and expectation that consumers of mental health services have a significant role in the evaluation of the services they receive, there is a paucity of literature addressing the extent to which this is realised in reality. The literature on this topic suggests that the outcome measures used routinely in Australia do not necessarily measure the aspects of care and treatment considered useful or beneficial by consumers. In particular the ROM currently used were not developed in collaboration with consumers and subsequently do not reflect the process of recovery valued by service users. The current study will build upon the existing work in this area by providing a voice for consumers with a view to: identifying alternative user-derived strategies for service evaluation; articulating aspects of mental health services that promote and hinder recovery, and exploring the principles and values that the consumer participants consider should be implicit within the evaluation of mental health services.


20

3 3.1

Stage 1 – Formation of the Reference Group and PAT Discussions

Methodology

As stated in section 1.3, this study utilised a qualitative approach. The review of the literature highlighted the limited attention afforded to consumer perspectives of ROM and a consideration of potential alternative strategies. Qualitative methodology provides the opportunity to explore a relatively underdeveloped area (Elliot & Hayes, 2003). Furthermore, this style empowers the participants to actively determine the course and outcomes of the research according to what they consider important and valuable, rather than that predetermined by the researcher(s) (Beanland, Schneider, LoBiondo- Wood & Haber, 1999). This philosophy is essential if the inclusion of the consumer view is to be genuine rather than tokenistic. Traditionally, consumers have been ‘researched’ and their participation has been a relatively passive process of providing information within the limitations imposed by the preconceived objectives and concerns of researchers. (Telford & Faulkner, 2004). As the aim of this study was to elicit the views and opinions of service users it was imperative that they be given the opportunity to do more than merely respond. Their active involvement in shaping the study under investigation was actively encouraged and facilitated within this framework. This choice of methodology proved particularly apt. Analysis of the data gained from PAT discussions, required the research team to rethink the aims of the study and adjust the research direction accordingly. This change of focus will be described in Section 4.1 as a prelude to the second stage of this study. 3.2

Methods – Stage 1

The preparatory stage included the identification of reference group members, the conduct of a literature review and the preparation of an ethics proposal relevant to Stage 2. 3.2.1

Establishment of the Reference Group

The reference consisted of a group of collaborative partners (consumers and service providers) with: •

An extensive knowledge of and/or experience with outcome measures used routinely in Victorian mental health services, and

•

A demonstrated interest in consumer perspective research, or

•

Well-developed research knowledge and skills.


21

Membership included: Brenda Happell, CPNRP Director; Cath Roper, Consumer Academic; Karla Gough, Research Fellow; Carolyn Graham, Consumer Research Assistant; Shirley Jennings, senior psychiatric nurse; Stephen Elsom, President ANZCMHN; Mishka McIntosh and Amanda Stella, PAT group members; Viviana Cohn, researcher using consumer perspective; and, Tom Callaly, psychiatrist. The aims of the reference group (see Appendix 2 for Terms of Reference) were to: •

Give advice and feedback on the implementation of the project

•

Give advice and feedback on the development of the project

•

Give feedback on the interpretation of study findings, and

•

Assist with the dissemination of project findings and recommendations

Operation of the Reference Group: •

The Reference Group met three times throughout the duration of the project.

•

Minutes of meetings were distributed to all Reference Group members.

•

Reference Group members were invited to give feedback on the project between formal meetings, where necessary.

•

The Reference Group developed a discussion stimulus for the PAT conversations (See Appendix 3)

•

The reference group reviewed and provided feedback on the draft report including recommendations for further research

3.2.2

Ethics

Ethics approval was obtained by from University of Melbourne for the PAT group discussions. The conduct of this research confirmed to the expectations of this committee as set out in the plain language statement. Participants were informed about the voluntary nature of participation and their right not to be involved or to withdraw their participation at any stage. They were also reassured that the information they provide would remain private and confidential. 3.2.3

Recruitment

PAT is a group convened through the Centre for Psychiatric Nursing Research and Practice. This group had previously indicated their support for, and willingness to participate in, this research.


22

3.2.4

Group discussion between PAT group members

Two group discussions of approximately 90 minutes each were held between the consumer members of PAT. Eight members participated in at least one of the groups, with six participating in both. The discussions were moderated by a consumer researcher and research fellow (non-consumer). The ‘discussion stimulus’ formulated by the reference group was used to promote consideration of the issues. The group discussion between PAT group members was audio-taped. The taped discussions were transcribed for analysis. 3.2.5

Analysis of the group discussion

Data analysis was based on the approach developed by Ritchie and Spencer (1994). This method comprises five key stages: familiarization with content; identifying a thematic framework; indexing themes; charting themes; and, the mapping and interpretation of themes. To enhance research rigour, two researchers conducted data analysis separately. Subsequently they came together to compare themes and interpretations. Consensus was readily achieved following discussions of minor discrepancies. These themes will now be considered and illustrated with indicative quotes from the participants. The data is primarily presented through quotes from the participants themselves. As the aim of this research is to gain the attitudes and opinions of service users, it is important that they are given the voice to do so. As stated by Epstein (2003: i) “… giving each person who speaks a chance to speak as a whole … in a way that made sense to them rather than to an editor (or researcher) arranging people’s meaning to suit their own publishing purpose”. While some analysis and editing has been conducted the level of intrusion by the researchers has been limited. 3.3

Findings

Data analysis revealed that three main themes emerged from the PAT Group data: assumptions behind ROM; consumer concerns with ROM; and Consumer Perspective: purpose, process and principles. These themes will now be presented and illustrated with indicative quotes. 3.3.1

Assumptions behind ROM

Within this theme the PAT members identified three main assumptions underlying the use of ROM: they are primarily designed to benefit the service by providing information to support the benefits gained as the result of service use; they suggest that positive outcomes for consumers are the result of service use and therefore do not consider the many additional factors, external to the service that might have influenced changes; and, they do not provide a basis from which the negative effects of the service system can be identified. The following discussion articulates the PAT members’ views on the inaccuracy of these assumptions. Professor Brenda Happell, Central Queensland University, 2007.

23

There was considerable discussion about the purpose of ROM, and a general sense that ROM are designed for the benefit of services, to demonstrate that they have been effective in improving health outcomes, rather than being a consumer-focused initiatives the following quote suggests: … what usually happens is you’ll receive a service and whether it’s medications, or some kind of group therapy or individual therapy, when they look at evaluation, they want to see whether it has had any benefit for you … As far as governments are concerned, this move toward quality frameworks, so they want to make sure there’s improvements in services and that services are actually doing something so they do currently have tools in place, like the BASIS-32, so that’s what they currently use to say the service has had an impact and more specifically, which is interesting, it is framed in terms of the benefits of the service (G1, p. 10-11). This strategy provided the scope for services to take credit for positive outcomes without necessarily being able to clearly demonstrate what was responsible for the improvements, as the following quotes suggest: The assumption is that if people get better it must be because of the service, because that’s the only thing that makes people get better, all the nice drugs and things, whereas in fact, people get better despite the service (G1, p.5) Furthermore: I don’t think there’s any real link to change with this stuff. Unless at the end of a certain amount of time after people are filling out a properly derived evaluation framework and then it’s “oh, wow, the system’s not working, let’s see what we can do. I don’t see the direct link between change in service delivery and filling out these forms (G1, p. 18) ROM appeared to be based on a clear assumption that quality of services is related to outcomes, without seeking the input of the consumers themselves regarding what might have contributed to improved outcomes, as stated: There’s nothing that says “was your experience with your case manager contributing to your state of well being?” (G1, p. 8). This was particularly apparent because consumers are continually influenced by far more than the existing service system … they [services] can’t make the point that just because someone changes it’s the service because increasingly people are using GPs, NGO’s, families, support services, private therapists, massage (G1, 10) Participants criticised ROM for not reflecting the damaging aspects of service use which were not reflected through the use of ROM, as stated:


24

There are so many of us that have these ongoing scars, open wounds as a direct result of having been treated (I don’t even know the word to use) by services and it’s those things where I feel it seems to be that we need the truth, I feel for me the main principle or guideline would be feeding the truth back and the reality of how the services impact our lives (G2, p. 3).

A concrete example provided by one participant illustrates this point: An outcome for me, after having used services and being locked in seclusion, is now that I have claustrophobia, that’s one of the outcomes for me is that I now can’t be in closed rooms, and those kinds of outcomes are what need to be fed back to the service (G2, p. 1). Where improvements were not evident the service was seen to have an ‘escape clause’ Services generally think that ‘we make things better’, and if things are worse, well then it’s the illness (G1, p. 13) 3.3.2

Consumer concerns with the use of ROM

The PAT members had three main concerns about the potential use of ROM: it could become the basis to decide access to or denial of services; become a formula for the funding of services; and, a sense that the real purpose of ROM was not disclosed to consumers. Some consumer participants expressed concern that ROM might be used to determine eligibility or otherwise for service, as the following quotes suggest: Consumers are concerned that it might mean they don’t get a service (G1, p. 6). Similarly: What happens if I’m doing well, do I lose a service, for those who want it?; The DHS position is it’s not going to be linked in that fashion. But the fear’s there and it could happen down the track (G1, p. 6). And again: One of my fears is that it becomes about how well the service is scoring, and then it will be linked to money and well, we do this score on our HoNOS and BASIS-32 and therefore we should be more able to get better funding than so and so service down the road and that’s my greatest fear with this sort of stuff (G1, p. 15).


25

There was also some concern that ROM might be used for other unsound purposes, as suggested: What’s the information being used for, because people are being told, oh, you’re filling your census data in so we can find out about your income and that means the government will be able to do something about poverty, but what they really use it for is to find out where the poorest people live and that’s where they put the poker machines (G1, p. 19).

A strong sense that ROM were not used to accomplish stated objectives was clearly evident, as demonstrated by the following quotes: …when they’re trying to find out whether a service is helping or hindering, in the past, they give you lots of bloody tests (G1, p. 11). Similarly: If I say, I think the service should spend more time doing this, and nothing ever happens of it, why would you say it again, consumers have been too much queried out. You do this satisfaction survey, and if you’re expecting the change to happen, and don’t see it, you’re not going to fill in another one (G1, p. 12). And again: … when the government says we want to be able to compare one area mental health service to another, is that really true, or do they want it to happen in five years’ time which will always be in five years’ time (G1, p. 12). 3.3.3

Consumer Perspective: purpose, process and principles

The issues raised under this theme include: the focus of outcome measures on symptoms rather than those factors valued by consumers; the need for service evaluation to be linked to accountability and change; the need to respect the rights of consumers; and, the importance of promoting self-determination. Consumer participants felt ROM predominantly focused on measuring symptoms as a way of determining success or failure: By emphasizing service use, we’re making an important point that it isn’t just about individual pathologies, like “how depressed are you?” the focus is on something larger (G1, p. 20). And:


26

When you’re trying to find out if a service is hurting or helping, you should ask a consumer (G1, p. 11). This change of focus to aspects valued by consumers would potentially provide the basis for constructive changes to service delivery, as the following quote indicates: … once you’ve defined what’s important [from services to consumers] once you’ve identified it, you can look then at what service models increase it. And what models do not (G2, p. 4). There was a lengthy discussion alternative language to outcome measurement that more accurately captured the concerns and interests of consumers. Suggestions included progress indication, progress supplement, review and assessment. Ultimately the participants endorsed the use of the term consumer critique, particularly because it: It indicates power. If you can critique something, it indicates you have the authority and capacity and the intelligence (p.26). Service improvement would result more effectively if evaluation was actually linked to change in order to sustain and create improvement: I think basically it’s about seeing how much services contribute to people getting well, and finally to make any modifications necessary, based on that. The only reason we’re looking at services is because they do impact on people’s lives and it’s about looking and seeing how they impact, and trying to make that change, whether it be by increasing the positive things, or decreasing some negative things (G1, p. 18). Indeed the capacity to create change is an important part of consumer perspective work: I think we need to have an intent that we’re [consumers] doing this with the purpose of change, that’s what it’s about, that’s what our philosophy is (G1, p. 19). This led to discussion of the importance of accountability by channelling information gathered back into services as the impetus for change, as a number of consumers stated: That’s part of the accountability and the responsibility that comes out of the stuff that people ask us to do, like it’s fine to fill out a response, but there has to be a response. Because if there’s no response, who wants to fill it out, who wants to go through that? If there’s a promise at the end of it that something’s going to get better, that’s a motivation for filling it out. But to put some hope into it, that’s a potential cost (G1, p. 9). Similarly: It is about there being a commitment on behalf of the services that if they use these things, then there needs to be feedback to consumers, they need to demonstrate to you how this is helping you, whether this is benefiting you (G1, p. 11), Professor Brenda Happell, Central Queensland University, 2007.

27

And: In the essence of accountability they [service providers] need to demonstrate that there is a relationship between what the service has done and any change that’s occurred. Does the service hurt or help, so that means people can comment on how the service has impacted on their recovery or whatever. And do no harm being one of the principles guiding it (G1, p. 16). The participants emphasised the importance that the intent and purpose of the instruments was clearly and honestly articulated: … There should be honesty around why the hell are we filling them [ROM] out (G1, p. 11). Similarly: That’s part of the process too, that you set up an expectation – you don’t just give another piece of paper, you’re setting this up as an expectation that this is supposed to be helpful for all of us, especially you, and if you fill this in, it helps understand more about what we need to do to help you more, so it’s about that engagement (G1, p. 11). And: … if you’re going to do something [and] consumers are going to part take in this process, then it needs to be for a purpose, and that’s about intent and purpose and accountability so it’s not just about filling it out and saying, well I got 80%, but I filled it out, I got 80% and that means that I need to now do this (G1, p. 15). Furthermore: There needs to be an element of accountability built into it and that’s around the service being clear about why they’re doing an evaluation, what they can and can’t accomplish (G1, p. 16). And finally: What’s the intent of why are we filling out the outcome measures, is it to open a dialogue, is it to get funding from DHS, is it to precipitate change for the person, whatever the intent of the tool is, then there’s an honest communication about that (G1, p. 18). The participants felt that any improvement in processes to evaluate service effectiveness should by necessity be consumer driven, as the following quote relates: If we’re evaluating something, it needs to come from us as consumers and it needs to be thinking about what the relationship between the consumers are and the people providing the service (G1, p. 10). Professor Brenda Happell, Central Queensland University, 2007.

28

Similarly: … what do we think the intent or the purpose should be, and it’s about us developing that (G1, p. 18). There was also a sense that genuine consumer participation in evaluation reflected a genuine desire to make services more responsive to consumer needs rather than to encourage consumers to understand the reasons why services run as they do. This point was powerfully made by one participant as illustrated in the following quote: [consumer participation should] not [be] about “Let’s try and make this seclusion and this forced injection less traumatic for the consumer by explaining to them why it was necessary”. That’s the sort of bullshit (excuse me) that’s coming out of this stuff, well, we must debrief, and we must explain to them why we had to do it to them and therefore then they’ll see it was necessary so then it’s less traumatic and that’s the only response I’ve seen to the now well known knowledge that these experiences are traumatic for us (G2, p. 3). Furthermore, consumer involvement in service evaluation needs to be imbued with a suitable level of respect and importance, as suggested: The process has got to be invested with a certain amount of importance (G1, p. 18). It was also considered important that consumer involvement in service evaluation be undertaken in a friendly manner: A friendly process is part of the philosophy (G1, p. 18). By necessity the use of ROM must be respectful of the human rights of consumers: … we need to go a step back and my opinion is that services need to be aiming at treating people without infringing their basic human rights (G1, p.13). Human rights was regarded as a specific issue facing consumers of mental health services to a much greater degree than consumers of other health care specialties, as the following quote demonstrates: I think more than anything, we would need to draw attention to the fact that we have separate legislation what other area is there where there’s separate legislation that does not have a check and balance, an ombudsman, something to offset the removal of human and civil rights. There’s not a mental health ombudsman (p. 34). The voluntary nature of consumer participation in completing ROM needed to be clearly articulated from the outset. Informed consent was recognised as an important part of this process: Professor Brenda Happell, Central Queensland University, 2007.

29

You’ve got to have a real choice. If you don’t even know what’s there to be offered, you don’t have the choice to refuse it (G1, p. 17). Furthermore: It’s [ROM] got to be transparent in a fair and amicable way between each clinician to the consumer (G1, p. 11). Rather than completing forms, clinicians should be encouraging consumers to directly communicate their concerns: Whoever’s putting it [ROM] to the client, should be providing a forum where if the person wants to say to their face – it’s rubbish, you’re rubbish, they can, and also the clinician has to actually listen to it and has to set up a plan of action that says, “yeah, I’m going to respond to that”. Not, “let’s hush hush, let’s not deal with this one, put down the file..” (G1, p. 8-9). Ultimately the aim of ROM should be to promote self-determination, as the following quote suggests: But self determination should be the goal here they say it but the real goal should be to make the clinicians and the service unnecessary. And they say that, but very few people practice it”; “So you can facilitate the self determination of another person”; “By making yourself redundant”; “By making yourself redundant, and by getting people off orders and criteria for involuntary status for example. That should be how people approach the job”; It’s almost like it (self determination) incorporates the notion of recovery”; “But it’s positioning yourself in a different relationship to the person, there’s much less likelihood that you can say: recovery is done like this, because if it’s self determination, it’s explicitly individualistic. And there are already principles around how you do self determination, what it is and how you assist with it – it’s already there”; “For me it does carry on from, and encompass recovery because if people really are in charge of their own lives – people get better and well”; It lets you determine your own recovery, what recovery means to you”; “Individuals, you can’t just have one thing called recovery, and give it away to everyone – there you go, this is the format”; “That’s right “; “The way I look at recovery is just, a good life, and my good life is going to be different from yours, my good life includes a computer, and [name withheld] good life would have nothing to do with a computer “; (group laughs); “And some people’s good life involves regular exercise and other people’s don’t [sic] (G1, p. 17). 3.3.4

Summary of Findings

Analysis of the PAT Discussions revealed significant criticisms of the current use of ROM, in particular three main themes were identified: assumptions behind ROM, Consumer concerns with the use of ROM, and Consumer perspectives: purpose, process and principles. Professor Brenda Happell, Central Queensland University, 2007.

30

The participants considered the use of ROM to reflect an underlying assumption which ultimately benefits the services. ROM suggest that any positive changes in consumers as measured could be attributed to the performance of the mental health services, and therefore did not take account of the external influences that might bring about positive changes (falling in love was an example given). A number of concerns were raised that ROM could be used to determine eligibility to access or continue with services, with some consumers possibly being identified as no longer in need as the result of the HoNOS score. There was also concern that scores on ROM might be used to determine levels of funding, with those services presumed to be performing better provided with additional funds, making it extremely difficult for those performing less well to improve their service delivery. ROM were criticised for determining the success or failure of service delivery with a strong emphasis on an increase or reduction in symptoms. Alternatively, the purpose and methods of evaluation need to be consumer driven to determine the perspectives’ of consumers regarding what assisted or hindered them in recovery. The promotion of selfdetermination was considered the ultimate aim. Given these findings, PAT group members considered it necessary to change the focus of the research towards recovery and the evaluation of mental health services. This change of focus was subsequently implemented in light of the inherent consumer/participant focus of this work.


31

4 4.1

Stage 2 -Mental Health Service User Focus Groups

Methodology

The intention to primarily discuss the assessment of consumers (using outcome measures) moved more towards the concept of recovery, the evaluation of mental health services and providers and the role of consumer input in evaluation processes. The PAT discussions suggested that this topic was of greater importance to service users than further exploration of outcome measures would be. To examine this issue, two focus group discussions were conducted with service users from one metropolitan and one rural public area mental health service. 4.2 4.2.1

Method Recruitment of focus group participants

One rural and one metropolitan mental health service were selected for the research. Recruitment was conducted with the assistance of the consumer consultant and the senior nurses from the participating mental health services. A total of 16 consumers participated in one of two focus groups (nine from the metropolitan service and seven from rural). 4.2.2

Consumer focus groups

Focus groups were conducted in order to create an environment where participants felt sufficiently comfortable and engaged to freely discuss their views about recovery in the context of mental health service delivery. Focus groups potentially promote discussion and interaction between group members (Mansell et al, 2004; Verpeet, de Casterle, Van der Arend & Gastmans, 2005). Two consumer focus groups were facilitated by a consumer researcher and nurse researcher. The groups began by introducing the following three questions as a stimulus for discussion: 1. What aspects of service delivery do you believe help your recovery? How and why do these aspects of service delivery help your recovery? 2. What aspects of service delivery do you believe hinder your recovery? How and why do these aspects of service delivery hinder your recovery? 3. What principles, or values, do you believe should guide the evaluation of public mental health services? The focus group discussions were audio-taped and transcribed in their entirety to enhance data analysis. Professor Brenda Happell, Central Queensland University, 2007.

32

4.2.3

Analysis of consumer focus groups

The focus group data was analysed using the framework of Spencer and Ritchie (1994) as described above. Analysis was undertaken independently by two researchers, who later came together to compare findings. Discussion of minor discrepancies resulted in a consensus view. Data is presented based on the themes identified for each of the three sections outline above (help recovery, hinder recover, guiding principles). Indicative quotes are used to support the main points. Quotes are referenced with the letters MFG to indicate Metropolitan Focus Group, or RFG to indicate Rural Focus Group. The letters are followed with the page number of the transcript. 4.3

Findings

4.3.1

Aspects of service delivery that aid recovery

The following main themes were identified in response to this question: •

Treatment

•

Support and social connectedness

4.3.1.1 Treatment The responses included in this theme include medication, spiritual and counselling therapies, crisis management plans and cigarettes. Medications and other physical treatments were rarely mentioned as a positive strategy to aid recovery. However, one participant indicated that given the right conditions medication could be helpful: I think medication is very important … if you’re on the right medication, the right dose, its very important, its very helpful I’ve found (MFG p.4). The benefits of medication were particularly evident where consumers had input into the medication regime: I was having problems with my medication, the dose was too strong and I was getting severe side effects. I spoke to my doctor about it and she did agree to reduce the dosage which … really helped me recover a lot … cause I was having all sorts of wrong thoughts about it and it helped me when I got a reduction in the medication. So they [staff] listened to me, which was really good (MFG p.5).


33

Participants talked of other forms of treatment they found useful as an aid to recovery, including more spiritual approaches: [Meditation is] something that you yourself will benefit from no matter what your beliefs are. Some people for example believe in religion as a guidance and they pray … other people find that prayer helps them (MFG p.15). The value of counselling therapy was raised by one participant: There is therapy that people need to work on. For example, grief. You need to get over grief. A loved-one in your family dies and people need to find these things, discover what is it that causes it, that’s caused this depression, it could be a number of things (MFG p.15). A crisis management plan was suggested as a positive strategy in promoting recovery: I don't know if anyone else has done this … sat down and written out a crisis management plan with [their] case manager, so that if you came back into emergency, you know, that you have slit your wrists or something, that the case manager that you get to come out to assess you knows the steps to take (RFG p.35). To which another participant responded: Yeah, and it [crisis treatment plan] is kept in the emergency, which is helpful. But it is kept at emergency, so that anybody that comes out to assess you looks that up, and you have already previously agreed to whatever is on there, that you will do (RFG p.35). Although recovery was not specifically mentioned, cigarettes were seen by some participants as important strategies for surviving service use and/or the symptoms of mental illness, as the following quotes suggest: I just can't believe that the government are taking away the right to smoke in a hospital, when they make taxes on it. As far as I am well aware, I was there a couple of months ago. In 2007, from 1 July 2007, if you want a cigarette you have to go out on the [name of highway] (RFG p.2). And: Yeah, cigarettes is a big thing. They take up a lot of my day. As much as it is killing me, it is something to do. You have got to keep busy (RFG p.2). Professor Brenda Happell, Central Queensland University, 2007.

34

Similarly: I have a panic attack and I have a smoke. You know, I suffer the panic attacks, and as soon as I have a panic attack I am straight away for a cigarette. And that is what I am thinking, like, a friend (RFG p.2) 4.3.1.2 Support The issues discussed within this theme included: supportive staff; follow up from services; respect; peer support; promoting social connectedness; and, individual responsibility for recovery. Support was identified as a major facilitator of recovery and was a much greater focal point for discussion than the topic of treatment. . The support of staff was considered essential to recovery. This support was demonstrated through a number of factors. Timely service and follow up were valued, as the following quotes demonstrate: When I was in [name of service withheld] they [staff] were fantastic. They followed up all the time. Like, they ring you once a week and make sure that you are okay and all that (RFG p. 4) Similarly: When I was with [name of service withheld] … if you ring, they ring you straight back (RFG p.3). And: When I got out of hospital, she [case manager] would ring me about every second day. And wanted to come around and check on me (RFG p.19) The simple act of making a phone call was considered very important as one participant articulates: When I get a phone call off psych services, it’s a morale boost … it is like a bonus. Oh shit, someone cares (RFG p.10). One participant referred to the importance of having some say in the extent of the follow up provided: Well my case manager is good, and I find that I get followed-up every fortnight. I got to choose how much I wanted to be followed-up. Because I went through a time when I didn't want to speak to them [case managers] … I saw her today, and she did give me the reassurance that she can come Professor Brenda Happell, Central Queensland University, 2007.

35

and see me every week if she wants to. If I wanted to. So she is good (RFG p.23). In some cases staff support had assisted participants to obtain employment, as stated: They've [staff] helped me get a job, and got me in contact with the job network and things like that (RFG p.3) And: … once they [staff] got me in contact … the job network people have been fantastic. I start a full time job tomorrow, and that, so all that is good (RFG p.3). Staff support for specific initiatives from consumers was recognised as particularly beneficial, as the following participant described: Yeah, they’ve [staff] been really supportive … I’m a musician and I’ve recorded and released my own CD and they’ve bought if off me, which has really helped a lot … That’s helped a lot, yeah (MFG p.5). In particular, participants valued the capacity for staff to assist them with their confidence levels: [staff] give me that confidence, because I am still unsure of myself and that (RFG p.5). Respect for consumers was a quality much admired in mental health staff, as suggested: People [staff] who have respect for me, or us [consumers]. And being independent. And not being treated like a child or girl. Being treated like adults. One participant provides an example of how s/he felt respected and supported by psychiatrists who treated him/her as more than a diagnosis: I remember … a really good psychiatrist I had when I was back at the other clinic and … he’d ask what the voices were saying and I told him, it was mainly friends, the voices I was hearing and in a way it was kind of a positive beacon in amongst all the anxiety I was experiencing and … a lot of people don’t believe that to start with but he did and he asked me is it possible that these voices could just be a thought and that was one thing that he said that really impacted, that really made me think about it … Another psychiatrist said I think you should be these voices outside of you and that again was a really helpful thing rather than – right hearing voices and boom, dose him up, he’s a schizophrenic (MFG p.4). Professor Brenda Happell, Central Queensland University, 2007.

36

Peer support was also identified as a strategy to promote recovery. Consumers appreciated the opportunity to be with others who experienced similar issues and therefore had a similar understanding, as the following quotes illustrate: It was really good just to spend time with people who were suffering the same things, who could understand what you’re going through (MFG p.20). And: [we] get together, and they have all got a mental illness, and just talk (RFG p.8). The support of both staff and peers was identified as important in enhancing social connectedness: There are fun things to do that are available in communities. There are a lot of outlets and groups centres that are run by various different organisations. I attend … … for people that are out in the community that have been hospitalised, that have been institutionalised, that are back running in the community sector and they do fun things, like they have outings for the day that are dirt cheap or nothing. They go out in community buses … they might go fishing or horse-riding. Some things you do contribute to others things are cheap. I was running a soup kitchen there in their facilities and I was feeding thirty or forty people soup but they do it economically, one will do the shopping and they put it together and they do things as a group and they’ll go on outings or they’ll have a game of sport or they’ll see a movie, you know fun things to do. Getting back into the community, letting them live life again, rather than struggling and battling Notwithstanding the identified importance of staff and peer support, participants in the metropolitan group, referred to the importance of taking responsibility for one’s own recovery. The following quotes give voice to this view: Its up to you the initiator to put your feet back on the ground and put the first foot forward (MFG p.14). Similarly: What better experience than life itself? If life has taught you these situations, right, and you can identify them, what better experience and learning than learning it yourself? (MFG p.18). And:


37

The workers are there to guide you but you’re the one who’s got to make the initiation (MFG p.14) 4.3.2

Aspects of service that hinder recovery

Analysis of data revealed the following themes: •

Staffing issues

•

Hearing the person not the illness

•

Lack of safety and security

4.3.2.1 Staffing issues The issues discussed here included: staff shortages; accessing services out of hours; staff attitudes; and trust. Staff shortages were acknowledged as a significant barrier to service delivery in the rural focus group. Participants described many instances of being unable to access staff in a timely fashion, for example: Sometimes you can ring and you wait four days before they [staff] even get back to you (RFG p.4). And: My worker is either sick or she is not there, or there is always some reason that I haven't been able to get in contact (RFG p.5). Similarly: I don't think they [staff] have enough backup … they help me, and then all of a sudden there has been no help at all. So I think they [services] need more follow through (RFG p.3). One rural participant describes his/her experience of accessing staff within the inpatient unit: You couldn't get attention, you know? You would see your nurse, something, and you would want one of your needs attended to. And, "Oh, I will be back in a minute. They were busy, you know? … Basically they were busy with other people (RFG p.33). Accessing mental health triage services was seen as problematic for some of the rural participants, particularly out of working hours, as the following example illustrates: Professor Brenda Happell, Central Queensland University, 2007.

38

Night time help and triage is hopeless. You just get answering machines after answering machines. You're going off your head and all of a sudden you get cut off, then you redial, you are at reception, you ask for triage, you go to triage, you go from triage to [name of inpatient unit], from [Inpatient unit] back to triage. All you are getting is answering machines (RFG p.32) For the metropolitan participants, staff attitudes posed a greater problem than staff availability. Negative experiences and relationships with staff were clearly identified as barriers to recovery. Lack of trust was one problem: I was hindered by a psychologist claiming I smoked dope, drank alcohol and had stopped my medication, and the hospital staff when I was there believed him. I said do a test, you’ll find I don’t have any of those problems. They can do a test (MFG p.2). Failure to take the situation seriously had dire consequences in the words of this participant: They [staff] refused to increase my medication beyond the therapeutic dose, so I had loud halo voices in my head for about three months and I said, look, you can’t do anything for me, they cancelled all my leave, they wouldn’t let me have any day leave, because of this psychologist and what he’d claimed about me, which was all lies. I had to discharge myself. I ended up taking a massive overdose in my sleep and ending up back there [mental health service] again (MFG p.2). Most participants could recount incidents where they felt they were not listened to by staff. One rural participant describes the potential benefits of having staff to talk to when needed: All you want to do is talk to someone who is - I know for a fact, I can talk to someone for five minutes and I can calm down fifty per cent of my anger … just talking to someone I know. A few good words, "Buddy, just settle down" (RFG p. 8). This participant continued to say that unfortunately on many occasions staff were to busy to be available in this way. The issue of trust was particularly emphasised as essential for recovery by the metropolitan participants. In the words of one participant: Unless I’m believed, I’m not going to get proper treatment (MFG p.2). Similarly:


39

If you feel like you’re just talking and no one is listening, you end up just shutting yourself off (MFG p.3). 4.3.2.2 Relating to the person not the illness The participants described being judged and treated according to their symptoms; lack of respect for the consumer experience; and a lack of opportunity for participation. The metropolitan participants projected a clear sense that they tended to be treated according to their symptoms rather than according to their individual needs, as the following quotes illustrate: … just categorise you and … put you in your little box … As long as you’re in your little box, its okay (MFG p.3). Similarly: I really wish that some of the [staff], particularly psychiatrists, made more of an effort to not categorise people, to not just … judge someone without any effort to understand what the real experience is and try to fix it (MFG p.4). Participants described the struggle they experienced in being listened to and heard and how this impacted on the journey to recovery. In one powerful example: I believe you have to go to members of staff … for example, social workers or welfare workers or even psychiatrists and you have to prove a point to get them to listen to … your particular problem is and its almost as if you’ve got to fight against the system to struggle to survive and put your point across: ‘And this is what’s happening to me, do you understand me? do you know what I’m doing? do you know what I’m talking about? … No I don’t need to be pushed into the background … I need to be heard, I need something done about the problem. And a lot of psychiatrists will take it upon themselves … bang … to push in medication, don’t be right out of your brain’. You’re not thinking let alone moving. Why do that to the person? A person’s got to work through emotion. They’ve got to work through stress. They’ve got to be able to work through voices and things that are disturbing and destructive to their lives. They’ve got to bring out an issues and say: ‘Can you hear me? Can you do something to help me? And this is what a lot of mental patients find the struggle and the battle that no one wants to trust in them, no one wants to say that they are humans and they have human rights (MFG p.3). Similarly: Some of them [psychiatrists] do ask you a few simple questions and then make their judgment but not any effort trying to get to know exactly where Professor Brenda Happell, Central Queensland University, 2007.

40

you’re coming from … you know, hearing voices can mean so many different things to different people experiencing it (MFG p.3). For many consumers this meant medication would often be given without much consideration as to whether this was the most appropriate strategy, as stated: Right! hearing voices and boom, dose him up, he’s a schizophrenic (MFG p.4). Furthermore, this meant that doctors would not consider consumers’ experience of medication and side effects as data in making determinations about treatment. If you go to your psychiatrist, for example, and tell them what’s wrong with you and your explaining to him what symptoms you’re getting and the effect its having and you believe that the medication’s not working and they’re going against your decision with what they’re prescribing you and you know its not working for you and they’re not listening to what you have to say about it and … bang … they’ll prescribe a higher dose of the same medication (RFG p.10). One participant describes how the diagnosis and symptoms of mental illness lead to forced treatment and heavy sedation. Subsequently to the event s/he has not encouraged to be an active participant in her treatment: One criticism that I had about [name of unit withheld] was the police brought me in there against my will. And when you get there they - they want to sedate you. I didn't … want to be sedated, but it was either have this tablet or we will hold you down ... So I had the tablet. And then I woke up about 24 hours later, you find out you have been injected. And you ask them [staff] questions, what you have been given, and they won't tell you. And the first assessment they did of me in there was - I think it was about from 24 to 36 hours after I got in there, and I was incoherent on the drugs I was given. So the first assessment of me was made of me in a very drugged up condition, you know? (RGF p.35). This apparent reliance on symptoms detracted from the extent to which doctors were prepared to consider modifications of or alternatives to the more conventional approaches to medical treatment. One client describes this experience: Sometimes I’ve found some treating doctors are not prepared to take any risks to see whether um maybe less medication or no medication, you might be able to recover without it. I mean, ideally, you want to get back to functioning normally, and you know, they should be prepared to try alternatives and not be just consistently negative and not be prepared risks to … especially if that’s something you want to do when you feel it might Professor Brenda Happell, Central Queensland University, 2007.

41

work and it might be okay and its often the case that you know what’s good for you (MFG p.11). Consequently, many participants felt they were not credited with having knowledge about themselves and their illness, as the following quote demonstrates: It’s often the case that you know what’s good for you. It’s not always the case that just because you’re mentally ill that you don’t know what’s good for you (MFG p.11).

This type of approach impacted on the extent of trusted with the staff and the system and the extent to which relationships with treating doctors could be effective: I mean it caused me to react negatively to my treating doctor and everything and the entire service. I didn’t trust anyone. I didn’t want to talk to anyone. I ended up just not wanting to talk to anyone and I think … Sometimes I’ve found some treating doctors are not prepared to take any risks to see whether um maybe less medication or no medication, you might be able to recover without it (MFG p.11). 4.3.2.3 Lack of safety and security Discussion included the following issues: personal safety; lack of support and follow-up, inappropriate treatment; dangerous practices and neglect of physical health. While both groups of participants had a number of concerns about the standard or safety of the treatment they received within mental health services, there were significant differences between the two groups. The rural participants primarily focused on a lack of safety on the inpatient unit, premature discharge and inadequate supports in the system. Participants talked about the set up of the inpatient unit potentially increasing the likelihood of violence, as the following quotations suggest: The way they have got it in [name of unit withheld], all these corners and doors. You could probably easily - especially a bloke who is violent, … drag a woman … the staff can't be everywhere (RFG p.38). And: From my experience of being in [name of unit withheld], I was actually assaulted by one of … the male patients. I was already at a very vulnerable stage, which didn't help my stay in [name withheld] which probably prolonged it a little bit more. … Because it is a mixed ward. … Apart from seclusion, the males and the females can mix around with each other (RFG p.36). Professor Brenda Happell, Central Queensland University, 2007.

42

Similarly: I'd be shitting myself if I was a woman in Flynn (RFG p.38). One participant described a situation to highlight the danger that young women may find themselves in within such an environment: There was a fairly young girl in there [inpatient unit], she was very young and pretty vulnerable, I thought. And she had been there for a long while. And I was in there for about, oh, ten days or so, I was starting to get a bit better. I noticed another person came in there, and he was quite mad, or let us say quite out of it, and stuff, you know? And the next thing I see him … they were sort of holding hands, you know? And I didn't want that, you know? They didn't know each other before hand, and he was like, as soon as in there, he sort of - it was as though it was sort of get straight on to her, you know? And people in there, they are vulnerable. They are sort of looking for a friend, you know? (RFG p.39). Personal safety was also raised as an issue for the metropolitan participants. In one example: The hospital environment in itself … we’re all just thrown in together, which can be a bit harrowing. You know, you’ve got bloody reformed paedophiles, ex-prisoners, your murderers, one of the guys I was in with had stabbed and killed a woman and served time for it. You know, if you’re already paranoid it can get a bit hairy. I’ve met some great people and even had some good times, jeez it would help if they could separate us a little bit (MFG p.12). Early discharge and lack of follow up post discharge presented safety issues for participants, as the following quotes indicate: They [staff] are saying "Well, we are ready to discharge you." And I am like, "Argh, pardon?" Like, I just deal with it myself now. I just - every now and then my case manager will ring me (RFG p.9). And: They [staff] need to follow up more on the patients, and … ring and see if you are okay, and things like that. When I was in [name of service withheld] they rang once a week. You know, like they would ring me once a week and say, "Are you all right? Is everything okay?" And, you know, "Do you need anything?" And things like that. Here - - - [nothing] (RFG p.10). One participant referred to the lack of support groups in the community: Professor Brenda Happell, Central Queensland University, 2007.

43

The only way you get that [support] is you have got to be institutionalised (RFG p.20). The responses of the metropolitan participants more focused on problems they experienced with their actual treatment I’ve had experiences where I’ve been put on the wrong doses and stuff and that just sent me backwards in my recovery compared to helping me to get better (MFG p.4). Similarly: I’ve been getting a lot of restlessness in my legs and my arms and I keep taking the tablets that are called Rivotrol and the doctor keeps saying: “Don’t take it, don’t take it.” And if I don’t take it, then I can’t sleep. So she’s telling me know but I’m going against what she says and then when I told her: “I want to take it. I'm taking it alright.” At the end of the meeting, before I was walking out the door, she said to me: “Stop taking it.” That was the last time and I’m still taking it. I’m not going to go without sleep (MFG p.10). Seclusion was identified as a particularly damaging practice by a number of participants. The following quotes suggest the severe damage this practice is capable of procuring: Yeah, seclusion. I tried to choke myself. I tried to chop my penis off, ‘scuse my language, I tried to smash the television with the metal ashtray, um, I had all these awful thoughts, but I didn’t do any of them. There were nurses there. So that hindered me, but there needed to be more nurses there, because I could have done it and I had to be strong. Half of it, it’s a 50-50 situation. There needs to be more nurses in seclusion to keep the people either occupied or whatever. So they don’t do something to themselves (MFG p.13). And: I was in seclusion and I was just being … I was really paranoid about being victimised and being put in seclusion didn’t help. Yeah, I think it made me worse (MFG p.14). The tendency to neglect physical conditions was raised by one participant: They [staff] won’t consider the fact that you have a physical side, an aspect, its always up here [pointing to her head]. Its your mind, its your mind and yet your body’s shaking, your like this [demonstrates bodily shakes], you’re jittering or you’re twisted and your neck’s turned and then you’re dribbling … (MFG p.10) Professor Brenda Happell, Central Queensland University, 2007.

44

4.3.3

Principles or values that should underpin the evaluation of mental health services

In response to the question of principles or values, participants tended to raise specific issues or initiatives they felt would improve the service. However, the following themes were identified from this data: •

Consumer involvement in a more rigorous evaluation

•

Peer support

•

More responsive care and treatment

4.3.3.1 Consumer involvement in a more rigorous evaluation The participants identified the need for greater consumer involvement at both a systemic and an individual level. Within this theme the participants discussed: seeking opportunities for involvement; the importance of independence; and the evaluation of individual treatment outcomes. The participants (particularly metropolitan) expressed very positive feelings about the opportunity to be involved in the focus group, which they viewed as a form of participation. One participant stated: I knew this meeting was going to happen about four or five months ago. And then, I think there should be more of them planned way in advance so that people have something to look forward to. It sort of sets in your mind (MFG p.6). Similarly: Yeah, I think, having more input and forums like this where people can talk about their experiences, the positives and the negatives, research and stuff like that. I think that all helps (MFG p.16). One participant suggested that evaluations or outcomes could be more appropriate measured for individual clients by comparing them with their state at discharge from previous admissions: Well, for example, with someone who has bi-polar disorder and schizophrenia, who’s had more than one hospitalisation, should at least be discharged at the same level of health after each episode (MFG p.16). One participant referred to the importance of independence as a component of evaluation, as stated: It’s always good to have an external kind of thing, where more of a broader community takes account of things (MFG p.16). Professor Brenda Happell, Central Queensland University, 2007.

45

4.3.3.2 Meaningful activity and peer support More specifically the participants referred to the need for organisational structures to promote support; and the availability of activities, as a principle that should be embedded into the evaluation process. In this context participants discussed the issue of peer support as an intervention that should be embedded systemically within the mental health service delivery framework, particularly to benefit clients once they are discharged into the community: We are sitting in this beautiful brand new mental health community centre. Surely it is possible to have groups where people can - like this [focus group] can come and talk. It doesn't have to take a focus group like this to bring people together to talk, does it? (RFG p.20). And: So maybe it would be helpful to have more kind of programs where even after hospital [you can meet] with people experiencing the same things as you (MFG p.7). Similarly: Having a group like this [focus group] is good, because you don't feel like you are the only one (RFG p.10). However, the opportunity for peer support and socialisation was considered an important part of inpatient care: [A] discussion group, would be good to have. The mind-set these days has kind of moved away from any type of … counselling and I never really did [get counselling]. I mean there are some fantastic people working in the industry, there are really good people, but there are not enough resources. Like I remember in [name of hospital withheld] –– and early on we had a guy who would come in about two or three times a week and then talks to us all [consumers] and puts some music on and would break out the crayons and badge making and just the energy that it … and the therapeutic benefit of that was amazing, but that got phased out. In the last few times I went back that wasn’t paid for, so there needs to be more money (MFG p.7). Furthermore: When you’re in there for a long stay, it gets pretty dull. That’s not helpful when you’re just bored, especially if you’re suffering depression. Things like that, little things that wouldn’t even cost that much, discussion groups, that sort of thing, would be good (MFG p.8). Professor Brenda Happell, Central Queensland University, 2007.

46

And: I think what they should do is provide an entertainment centre where people can go and listen to music and do painting or sit in the corner and do their tapestry or read or whatever it is they want to do and with a gym provided as well (MFG p.9). 4.3.3.3 More responsive care and treatment This theme was considered to represent an essential component of mental health service delivery that should be clearly reflected in any evaluation. Within this theme participants discussed: promoting optimal recovery; collaborative relationships between services and consumers; and specific activities and treatments. One participant expressed the view that the underlying principle of service delivery should be towards optimal recovery: I think they [services] should have a belief in optimum recovery. I mean, they should look for the best possible outcome. When I had this situation in January, I’ve had an illness for 30 years. I have never ever ever had to live with those voices, always by increasing the medication then after 12 months maybe reducing it, I’d got rid of the voices. So why were they now telling me that I had to listen to a loud hailer for the rest of my life? And use destruction devices? They should have been going back and saying: “This has never happened before.” Why should I have to live with that? Where was the policy of optimum recovery (MFG p.16). Participants supported a more responsive service where mental health professionals sought to understand consumers and work with them in a more collaborative fashion. This principle is reflected in the following quote: I think you need to know that you’re talking to professionally-focussed people that are going to try and figure out what you’re talking about (RFG p.3). In some instances this might involve considering other forms of treatment as alternatives to conventional medications: I believe in natural things like herbs and organic things and when I was first delivered with medication twenty-eight years ago … I was shattered by the experience that I had to be pumped full of medication (MFG p.6). Counselling was also seen as an important part of service delivery that was often neglected in preference to medication: That’s pretty much the extent of the service, you know, these days and I know that medicine has come a long ways in regards to treating, you know Professor Brenda Happell, Central Queensland University, 2007.

47

less side-effects and not using heavy tranquillisers but just moving back to that idea of therapy. You are curing souls, not just … you know what I mean (MFG p.19). More notice about upcoming discharge was also considered important to enable consumers to adjust and plan. The following words demonstrate the difficulties when the decision is made suddenly On the discharge thing, they sort of - just a snap decision, seemed to upset me. You don't seem to get any lead way into it ... whether it is too early or not, I don't know. Just the snap decision, you are just told out of the blue, a couple of hours before you are discharged. It is sort of upsetting. I didn't like that (RFG p.11). 4.4

Summary

Consumers identified a broad range of factors that they perceived helped their recovery; however, themes tended to centre around support, whether it came from staff or other consumers, this included staff facilitating self-help strategies. In this context, support was viewed as central to consumers being able to develop the skills necessary for recovery, to enhance self-determination. The facilitators of the focus groups were not able to elicit much information about how and why these aspects helped consumer recovery. Most of the focus group discussions centred on what aspects helped participants’ recovery. Consumers identified a broad range of factors that they perceived hindered their recovery; nevertheless, themes tended to centre on poor access to staff and facilities. In some respects, this appears to be the flip-side of being supported, which is viewed as integral to recovery (i.e., the absence of support hinders recovery). The two groups identified completely different factors. Again, the facilitators were not able to elicit much information about how and why these aspects hindered consumer recovery. The importance of support and the promotion of recovery also emerged as important themes in discussing the principles and processes that should underpin the evaluation of mental health services. In addition the need for consumer involvement and independence was raised. Furthermore, participants articulated the need for care and treatment tailored to the individual needs of consumers.


48

5

Discussion and Conclusions

The findings from this project support the limited relevant literature in suggesting ROM (in particular the Basis-32) represent a tool to meet the needs of services providers, rather than producing information which is useful or beneficial for consumers (Campbell, 1998; Fossey & Harvey, 2001; Graham et al., 2001; Miller et al, 2003). The discussions with PAT group members revealed three main themes (1) assumptions behind ROM, (2) consumer concerns with ROM, and (3) consumer perspective: purpose, process and principles. ROM are designed to measure outcomes, but it has been argued that there is no evidence to suggest that this intervention produces any difference in outcomes either as part of the process itself or through an accompanying improvement in the quality of care (Lakeman, 2004). The findings from the PAT discussions support this view and indeed extend upon it. The use of ROM, the participants argued, was to benefit services, supporting the criticisms previously made by Graham et al (2001) and Miller et al (2003). However, the participants raised an important point not previously referred to in the literature, that improved outcomes might be inaccurately attributed to service delivery when it may have resulted from something external. Falling in love was an example provided by one PAT member. The “no lose” approach of services to ROM was also raised in PAT discussions. While services would willingly accept credit for improvements in outcomes, they would attribute lack of success to the illness or the individual experiencing it. The term “treatment resistant” has been used to describe people who do not respond favourable to conventional approaches to treatment. This theme also emerged in a qualitative study by Usher (2001) in which patients who did not improve through taking medication felt a high level of responsibility for not meeting medical expectations. Concerns were raised in the PAT discussions that ROM might ultimately be used as an arbitrary measure to determine entitlement or otherwise for service use. It is already known through research data that only one third of people experiencing mental health problems seek treatment at all and of those who do approximately 6% access specialist mental health services (Henderson, Andrews & Hall, 2000). Effectively this means that given the current available resources, any increase in the proportion of people seeking specialist mental health services will necessitate rationalization of existing resources. The use of ROM to determine eligibility would potentially be one approach to determining allocation. Furthermore it was felt that scores might influence funding, with monetary incentives provided to those with more favourable results, thus furthering the disadvantage for those services performing less well. Interestingly, this view directly opposes that of Coombs and Meehan (2003) who suggest that less favourable results might be used to argue for increase funding to improve quality of care and treatment. Professor Brenda Happell, Central Queensland University, 2007. 49

A major limitation of ROM was identified as a primary focus on symptom reduction and other provider determined factors, and the subsequent failure to view consumers as individual with distinct needs. This supports the available literature in suggesting that ROM focus on issues that of interest to clinicians rather than service users (Campbell, 1998; Fossey & Harvey, 2001; Graham et al., 2001; Lakeman, 2004; Miller, et al, 2003) and does not support the view of Coombs and Meehan (2003) that outcome measures can assist nurses in providing individually tailored care for consumers. The participants reinforced their concerns that the results obtained from ROM were kept private and not used to bring about change. Effectively this means the completion of these forms are at best a waste of time, and at worst a bureaucratic exercise which could be used to determine eligibility or ineligibility for services or to set levels of funding for services based on their ROM scores. If ROM are intended to improve the quality of care then surely it is beholden upon the services not only to make the results publicly available, but to clearly articulate the changes made to service delivery as a direct result of these changes. These findings were used to determine the questions for the focus groups with consumers, stage 2 of this project. Broadly speaking three questions were developed as an interview guide. The questions addressed aspects of mental health treatment that enhanced recovery, hindered recovery and the values and principles that should underpin the evaluation of mental health services. The findings from the focus groups clearly indicated a broad range of strategies were considered helpful in promoting recovery. In terms of treatment, medication was viewed as important, particularly when doctors took the wishes and concerns of consumers into consideration, which supports the findings of a study by Usher (2001). A number of nonpharmaceutical interventions were also considered useful were meditation, counselling and a crisis management plan. Several consumers also referred to the importance of cigarettes, or more commonly the anxiety and panic they felt when unable to obtain access to cigarettes. Although the perceived important place of cigarettes has not been identified in the recovery model, the literature suggests that the desire to smoke in mental health consumers may be increased as a means of self-medication or as a coping response (Lawn, Pols & Barber 2002, McChargue, Gulliver & Hitsman 2002). This is not to suggest that mental health services should promote smoking as a coping or stress relief strategy. However as the opportunity for inpatients to smoke becomes increasingly unavailable, the importance of smoking must be acknowledged and alternative strategies to relieve anxiety and promote coping must be identified and implemented. Overall, support from staff and peers were portrayed as more valuable than medication and other strategies. The importance of support has been acknowledged in the literature (Corring, 2002; Corrigan, Calabrese, Diwan, Keogh & Mussey, 2002; Mancini et al, 2005; Sjoblom, Pejlert & Asplund; Deegan, 2005). Deegan (2005) refers to the use of ‘personal medicines’ to enhance resilience and promote recovery. Personal medicines referred to the things that gave meaning to individual consumers and assisted them to overcome the negative outcomes of the illness and subsequent treatment. For the Professor Brenda Happell, Central Queensland University, 2007. 50

consumer participants in Deegan’s study, it was the personal medicines, rather than prescribed pharmaceutical treatments, that held the greater importance and any perceived conflict between the personal and pharmaceutical would generally lead to non-adherence to medical treatment. Pharmaceutical medicines were more likely to be taken as prescribed where they complemented personal medicines. Unfortunately ROM do not have the capacity to measure personal medicines or those strategies individual consumers adopt to assist with recovery. (Campbell, 1998; Fossey & Harvey, 2001; Graham et al., 2001; Miller et al, 2003). A broad range of factors perceived to hinder recovery from mental illness were identified. Inadequate staffing levels and lack of safety or security were prevalent. The recently released Not for Service Report (Mental Health Council of Australia, 2005) drew attention to workforce inadequacy and safety problems as endemic and an indicator of the failure of government to adequately resource mental health services. Shortages of mental health professionals has also been clearly acknowledged in reports of government and independent inquiries (Australian Government Productivity Commission, 2005; Commonwealth of Australia, 2001; Department of Human Services, 2001; Senate Community Affairs Committee, 2002; Senate Select Committee, 2006). A relationship between staff shortages and the standard of mental health care has been noted in these reports. The participants also commented on the tendency of staff to treat them according to symptoms rather than as individuals. This point concurs with one of the major criticisms of ROM, that the specific aspects measured do not necessarily equal improved outcomes from the perspective of service users (Campbell, 1998; Fossey & Harvey, 2001; Graham et al., 2001; Miller et al, 2003). The existence and impact of negative attitudes from mental health professionals has been discussed in the literature (Caldwell & Jorm 2001, Epstein & Wadsworth 1996, Happell & Roper 2002). In particular these findings support the work of Mancini et al (2005) by suggesting that negative staff attitudes are perceived as a major barrier to recovery by consumers of mental health services. The importance of recovery as a concept to consumers of mental health services has been acknowledged in the literature. Indeed it has been suggested that mental health services are becoming more focused on recovery. However, despite this there is limited literature describing the factors that enhance or hinder individual recovery (Jacobsen & Greenley, 2001; Mancini, 2005). The findings from the consumer focus groups therefore add to the existing body of knowledge about the important role mental health services can play in facilitating recovery for consumers. The principles or values the focus group participants considered should underpin the evaluation of mental health services, tended to be influenced by the factors that enhanced or inhibited recovery. This further reinforces the importance that recovery holds for people accessing mental health services. Peer support and more responsive, individually tailored care and treatment have been discussed above. The importance of consumer participation in rigorous, and preferably externally conducted, evaluation of mental health services was also raised as an important guiding principle. This is consistent with the Professor Brenda Happell, Central Queensland University, 2007. 51

literature (Goodwin & Happell, 2006; Weinstein, 2006) and Australian Government policy which promotes consumer participation in all aspects of service delivery, including evaluation (Commonwealth of Australia, 1998). The limited available literature suggests that the opportunity for genuine and effective consumer participation remains limited despite the promulgation of policy rhetoric (Goodwin & Happell, 2006; Lammers & Happell, 2004a; 2004b; Tobin et al., 2002). Given the importance the consumer participations have ascribed to independent and consumer influenced evaluation, over and above routine outcome measurement, the Australian and State Governments should consider allocating at least some of the considerable funds currently invested in the implementation of ROM into the development and examination of effective strategies to promote consumer participation in the evaluation of mental health services. The work of Miller et al (2003) identified the need for the ROM that measure outcomes considered purposeful and beneficial to the consumers ultimately affected by the services received. The findings of this study represent an early stage in providing a voice to consumers about their experiences of service use, their views on recovery and their thoughts about how services should be evaluated. The findings from this study are intended to contribute to the literature in three main ways: 1. To continue the debate around ROM their purpose, possibilities and limitations 2. To begin to build what will hopefully become a sizeable and recognised body of knowledge of service use and recovery from a first hand perspective 3. The use of the knowledge generated to improve the degree to which mental health services work towards the promotion of recovery. It is important to acknowledge the limitations of this study. The small number of participants from two specific locations means it is difficult to generalise the findings beyond those who actually participated. Indeed the significant differences between the responses of the rural and the metropolitan participants may be indicative of a diversity of consumer views about the recovery process and its evaluation. While the differences might be construed as representing the impact of geographical location, the small sample size precludes this being any more than an interesting observation. The extent to which these views are representative of the broader consumer voice may be the subject of further criticism, particularly the PAT members as they are part of a special interest group with a specific and defined interest in consumer perspective work, they may not be considered to be representative of the broader group of people who use mental health services (Crawford & Rutter 2004; Tobin, Chen & Leathley 2002). While the focus group participants were not part of a specific group, their choice to participate in the interviews might be interpreted as evidence of a particular interest in the subject matter that might not be shared by other consumers. Research undertaken by Crawford and Rutter (2004) suggests this criticism may be unwarranted. A survey seeking Professor Brenda Happell, Central Queensland University, 2007.

52

information on consumers’ views of the priorities for service developments revealed considerable similarities between members of consumer groups and other service users who had not been politically engaged. These findings may themselves be dismissed as unrepresentative because the study was conducted in one defined part of the United Kingdom. However, it is evidence, and the argument that consumer activists are out of step with the broader constituency is not based on evidence at all, and should therefore not be used as an argument to cloud the more important issues of encouraging consumer participation in the evaluation of mental health service delivery. Notwithstanding the acknowledged limitations, there is certainly some synchronicity between the findings of this study and the limited literature addressing this topic. Amidst the diversity of views is the clear message that recovery cannot easily be measured by conventional approaches. The diversity found most probably reflects the diversity that exists between people experiencing mental illness. However, the common factor that individual attention is more valuable than the objective measurement of symptoms is in itself a valuable lesson.


53

6 6.1

Outcomes

New Research Directions

In reflecting on the findings of this study, the project team and the reference group considered a shift in focus from the development of a user-derived outcome measurement tool towards a broader consideration of how mental health services might facilitate the recovery process of consumers rather than measuring outcomes that may or may not be influenced by the service system itself. More specifically the mental health service research agenda needs to include: •

The development of a consumer derived evaluation framework, which was always thought of as the natural spin off from this project. This framework would include strategies to encourage consumer participation in the evaluation of mental health services at both an individual and systemic level, reflecting the principles, process and practices of consumer participation

•

The development of accountability processes that are independent and transparent around what services should do with the information they collect so it becomes a meaningful process rather than a bureaucratic exercise

•

Further investigation of the practices of services than are perceived by consumers as harmful

•

The relationship between mental health services and recovery, including more information about the specific aspects that enhance, and those than present a barrier to, recovery. This should include investigation into the role that peer support plays, which is not currently captured through ROM

•

The development of a model articulating the recovery process, which is broad enough to reflect individual differences, yet specific enough to act as a guide for more consumer-focused service delivery

The successful implementation of ROM requires that mental health professionals (particularly nurses), are actively engaged in completing the measures and encouraging consumers to complete the Basis-32. Despite the important role that staff plays, there is limited current research literature describing the attitudes of mental health professionals to ROM. The degree to which mental health professionals consider the findings of ROM useful at a clinical level is a topic worthy of further exploration


54

6.2

Collaboration with other organisations

The User-Derived Evaluation Framework Project originated through a collaborative relationship between mental health nurses and consumers of mental health services. Psych. Action and Training (PAT) is a group funded and convened by the Centre for Psychiatric Nursing Research and Practice (CPNRP). However, the CPNRP does not provide limitations on the role or scope of PAT, providing the intellectual freedom for group members to consider issues pertinent to promoting consumer participation in education and research within the mental health sphere. The conduct of this project strengthened the relationship between the CPNRP and PAT. More specifically the qualitative, exploratory framework provided the ideal design to ensure PAT members could actively shape the focus and direction of the research process. This approach enabled the relationship between consumers and mental health researchers to reflect the principles for conducting researchers with rather than to consumers, and provided scope for the consideration of issues considered important to them. The formation of the reference group provided further collaboration with mental health professionals and consumers representing services and consumer organisations. 6.3

Dissemination of Study Findings

6.3.1

The project report

A copy of the final report will be sent to: •

The Australian College of Mental Health Nurses, Research Board

•

The Directors of Mental Health Services for each State/Territory and the Commonwealth

•

Peak consumer bodies throughout Australia

Electronic versions of the report will be offered for inclusion on the websites of relevant organisations including the ACMHN, CPN, and peak consumer bodies. 6.3.2

Refereed publications and conference presentations

6.3.2.1 Completed Hayman-White, K. & Happell, B. (2005). The UDEF Project: does routine outcome measurement satisfy the objectives of service users? The Seventh Annual Victorian Collaborative Psychiatric Nursing Conference. Melbourne, October.


55

6.3.2.2 Proposed Conference presentations: •

Australian College of Mental Health Nursing, 33rd Annual Conference, Cairns October (Abstract to be submitted)

Articles for refereed journals: •

Happell, B., Roper, C. & Gough, K. Meaningful information or a bureaucratic exercise? Exploring routine outcome measurement from a consumer perspective.

•

Happell, B., Roper, C. & Gough, K. Determining the effectiveness of mental health services from a consumer perspective: Part 1 - enhancing recovery”

•

Happell, B., Roper, C. & Gough, K. Determining the effectiveness of mental health services from a consumer perspective: Part 2 - barriers to recovery and principles for evaluation.


56

7

Acknowledgements

The authors extend their sincere thanks to the many people who assisted with the completion of this report: •

Ms Carolyn Graham, consumer researcher

•

Dr David Webb

•

Dr Stephen Elsom

•

Dr Caderyn Gaskin

•

Ms Shirley Jennings

•

Mr Mishka McIntosh

•

Ms Amanda Stella,

•

Ms Viviana Cohn

•

Dr Tom Callaly

•

All members of PAT


57

8

References

Andrews, G., Peters, L., & Teesson, M. (1994). The measurement of consumer outcome in mental health. Sydney, Australia: University of New South Wales. Anthony, W.A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16(4), 11-23. Australian Government Productivity Commission. (2005) Australia’s Health Workforce: Research Report. Canberra: Australian Government Printing Service. Beanland, C., Schneider, Z., LoBiondo-Wood, G., & Haber, J. (1999). Nursing Research: Critical Appraisal and Utilisation. (5th ed.). Sydney: Mosby. Borg, M. & Kristiansen, K. (2004). Recovery-oriented professsionals: Helping relationships in mental health services. Journal of Mental Health, 13(5), 493-505. Caldwell, T.M. & Jorm, A.F. (2001). Mental health nurses' beliefs about likely outcomes for people with schizophrenia or depression: a comparison with the public and other healthcare professionals. Australian and New Zealand Journal of Mental Health Nursing, 10(1), 42-54. Campbell, J. (1998). Consumerism, outcomes, and satisfaction: A review of the literature. In R. W. Manderscheid & M. J. Henderson (Eds.), Mental health, United States, 1998. Rockville, MD: U.S. Department of Health and Human Services (DHHS Pub. No. (SMA) 99-3285). Chapko, M. K., Borowsky, S. J., Fortney, J. C., Hedeen, A. N., Hoegle, M., Maciejewski, M. L., et al. (2002). Evaluation of the Department of Veterans Affairs community-based outpatient clinics. Medical Care, 40, 555-560. Commonwealth of Australia. (1998). Evaluation of the National Mental Health Strategy. Canberra: Australian Government Publishing Service. Commonwealth of Australia. (2003). National Mental Health Plan 2003-2008. Canberra: Australian Government. Commonwealth of Australia. (2001). National review of nursing education: Discussion paper (No. 0642772304). Canberra: Department of Education Training and Youth Affairs. Commonwealth of Australia. (1997), National Standards for Mental Health Services. Australian Government Publishing Service, Canberra. Professor Brenda Happell, Central Queensland University, 2007.

58

Coombs, T., & Meehan, T. (2003). Mental health outcomes in Australia: Issues for mental health nurses [Editorial]. International Journal of Mental Health Nursing, 12, 163-164. Coombs, T., & Meehan, T. (2005). Standardized routine outcome measurement: Response to Lakeman. International Journal of Mental Health Nursing, 14, 215217. Corrigan, P.W., Calabrese, J.D., Diwan, S.E., Keogh, C.B., Keck, L., Mussey, C. (2006). Some recovery processes in mutual-help groups for persons with mental illness; I: qualitative analysis of program materials and testimonies. Community Mental Health Journal. 38(4), 287-301. Corring, D.J. (2002). Quality of life: perspectives of people with mental illnesses and family members. Psychiatric Rehabilitation Journal, 25(4), 350-7. Davidson, L., O'Connell, M., Tondora, J., Styron, T. & Kangas, K. (2006). The top ten concerns about recovery encountered in mental health system transformation. Psychiatric Services, 57(5), 640-645. Deegan, P.E. (1996). A Keynote Address. Sydney Australia: The Sixth Mental Health Services Conference. Deegan, P.E. (2005). The importance of personal medicine: a qualitative study of resilience in people with psychiatric disabilities. Scandinavian Journal of Public Health,; (Supplement 66): 29-35. Department of Health and Aged Care. (1999). Mental Health Information Development: National Information Priorities and Strategies under the Second National Mental Health Plan 1998-2003 (First Edition June 1999). Canberra: Commonwealth of Australia. Department of Human Services. (2002). New Directions for Victoria's Mental Health Services: The Next Five Years. Melbourne, Victoria: Department of Human Services, Metropolitan Health and Aged Care Services Division. Department of Human Services. (2001). Nurse recruitment and retention committee: Final report. Melbourne: Department of Human Services Victoria: Policy and Strategic Projects Division. Eales, S., Callaghan, P., & Johnson, B. (2006). Service users and other stakeholders' evaluation of a liaison mental health service in an accident and emergency department and a general hospital setting. Journal of Psychiatric and Mental Health Nursing, 13, 70-77.


59

Eisen, S. V., Dill, D. L., & Grob, M. C. (1994). Reliability and validity of a brief patientreport instrument for psychiatric outcome evaluation. Hospital and Community Psychiatry, 45, 242-247. Elliott, D., & Hayes, L. (2003). Observational designs and methods. In Z. Schneider, D. Elliott, G. Lobiondo-Wood & J. Haber (Eds.), Nursing Research: Methods, Critical Appraisal and Utilisation (2nd ed.). Sydney: Mosby. Epstein, M. & Wadsworth, Y. (1996). Consumer Resistant Disorder (CRD) or cork in the mouth syndrome. Brisbane, Australia: The Mental Health Services Conference. Fossey, E. M., & Harvey, C. A. (2001). A conceptual review of functioning: implications for the development of consumer outcome measures. Australian and New Zealand Journal of Psychiatry, 35, 91-98. Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Outcome research in mental health: Systematic review. British Journal of Psychiatry, 181(8-16). Gill, K. J., Pratt, C. W., & Librera, L. A. (1998). The effects of consumer vs. staff administration on the measurement of consumer satisfaction with psychiatric rehabilitation. Psychiatric Rehabilitation Journal, 21(4), 365-370. Goodwin, V. & Happell, B. (2006). In our own words: consumers' views on the reality of consumer participation in mental health care. Contemporary Nurse, 21(1): 413. Graham, C., Coombs, T., Buckingham, W., Eagar, K., Trauer, T., & Callaly, T. (2001). The Victorian Mental Health Outcomes Measurement Strategy: Consumer Perspectives on Future Directions for Outcome Self-Assessment. Report of the Consumer Consultation Project. Retrieved 07 April, 2005, from http://www.health.vic.gov.au/mentalhealth/outcomes/concare.htm Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105-112. Happell, B. & Roper, C. (2003) The role of a mental health consumer in the education of postgraduate psychiatric nursing students: the students' evaluation. Journal of Psychiatric and Mental Health Nursing 10, 343-50. Henderson, S., Andrews, G., & Hall, W. (2000). Australia's mental health: An overview of the general population survey. Australian and New Zealand Journal of Psychiatry, 34(2), 197-205.


60

Howard, P. B., & El-Mallakh, P. (2001). Training consumers to collect data in mental health service system evaluation research. Journal of Psychosocial Nursing and Mental Health Services, 39(5), 30-39, 50-31. Hunter, R., McLean, J., Peck, D., Pullen, I., Greenfield, A., McArthur, W., et al. (2004). The Scottish 700 Outcomes Study: a comparative evaluation of the Health of the Nation Outcome Scale (HoNOS), the Avon Mental Health Measure (AVON), and an idiographic scale (OPUS) in adult mental health. Journal of Mental Health, 13(1), 93-105. Jacobson, N., & Greenley, D. (2001). What is recovery? A conceptual model and explication. Psychiatric Services, 52, 482-485. Lakeman, R. (2004). Standardized routine outcome measurement: Pot holes in the road to recovery. International Journal of Mental Health Nursing, 13, 210-215. Lammers, J. & Happell, B. (2004). Mental health reforms and their impact on consumer and carer participation: A perspective from Victoria, Australia. Issues in Mental Health Nursing, 25 (3), 261-273. Lammers, J. & Happell, B. (2003). Consumer participation in mental health services: looking from a consumer perspective. Journal of Psychiatric & Mental Health Nursing. 10(4), 385-92. Lawn, S. J., Pols, R. G. & Barber, J. G. (2002). Smoking and quitting: a qualitative study with community-living psychiatric clients, Social Science & Medicine, 54, 93104. Mancini, M. A., Hardiman, E. R., & Lawson, H. A. (2005). Making sense of it all: Consumer providers' theories about factors facilitating and impeding recovery from psychiatric disabilities. Psychiatric Rehabilitation Journal, 29, 48-55. Mansell, I., Bennett, G., Northway, R., Mead, D., & Moseley, L. (2004). The learning curve: the advantages and disadvantages in the use of focus groups as a method of data collection. Nurse Researcher 11(4), 79-88. McChargue, D.E., Gulliver, S.B., & Hitsman, B. (2002). Would smokers with Schizophrenia benefit from a more flexible approach to smoking treatment? Addiction, 97, 785-793. McPheeters, H. L. (1984). Statewide mental health outcome evaluation: A perspective of two southern states. Community Mental Health Journal, 20, 44-55. Mental Health Council of Australia (2005). Not for Service: Experiences of injustice and despair in mental health care in Australia. Canberra: Mental Health Council of Australia. Professor Brenda Happell, Central Queensland University, 2007.

61

Miller, M., Siggins, I., Kavanagh, D., & Donald, M. (2003). Consumer self-rated outcome measures in mental health. Melbourne: Australia: Mental Health Branch, Department of Human Services. Noble, L. M., & Douglas, B. C. (2004). What users and relatives want from mental health services. Current Opinion in Psychiatry, 17, 289-296. Oades, L. G., Viney, L. L., Malins, G. L., Strang, J., & Eman, Y. (2005). Consumer evaluation of mental health services: the process and products. New Paradigm. Peter, E. (2003). Review: involvement of former or current users of mental health services may improve outcomes in patients with severe mental illness. EvidenceBased Nursing, 6(3): 90. Pirkis, J., Burgess, P., Kirk, P., Dodson, S. & Coombs, T. (2005). Review of Standardised Measures Used in the National Outcomes and Casemix Collection. Canberra: National Mental Health Strategy. Poulton, B. C. (1999). User involvement in identifying health needs and shaping and evaluating services: is it being realised? Journal of Advanced Nursing, 30, 12891296. Ralph, R.O. (2000). A Review of the Recovery Literature. Retrieved December 3, 2006 from: http://www.nasmhpd.org/general_files/publications/ntac_pubs/reports/ralphrecov web.pdf. Rebeiro Gruhl, K.L. (2005). Reflections on …the recovery paradigm: Should Occupational Therapists be interested? The Canadian Journal of Occupation Therapy, 72(2), 96-103. Resnick, S.G., Rosenheck, R.A. & Lehman, A.F. (2004). An exploratory analysis of correlates of recovery. Psychiatric Services, 55, 40-547. Rosen, A. H.-P., D., & Parker, G. (1989). The Life Skills Profile: A measure assessing function and disability in schizophrenia. Schizophrenia Bulletin, 15, 325-337. Senate Community Affairs Committee. (2002). The patient profession: Time for action. Report on the inquiry into nursing. Canberra: Australian Government Publishing Service. Senate Select Committee on Mental Health. (2006). A national approach to mental health – from crisis to community. Canberra: Commonwealth of Australia. Slade, M., McCrone, P., Kuipers, E., Leese, M., Cahill, S., Parabiaghi, A., Priebe, S., and Thornicroft, G. (2006). Use of standardised outcome measures in adult mental health services. British Journal of Psychiatry, 189, 330-336. Professor Brenda Happell, Central Queensland University, 2007.

62

Sjoblom, L., Pejlert, A., & Asplund, K. Nurses' view of the family in psychiatric care. Journal of Clinical Nursing. 14(5), 562-9. Smith, G. R., Manderscheid, R. W., Flynn, L. M., & Steinwachs, D. M. (1997). Principles for assessment of patient outcomes in mental health care. Psychiatric Services, 48(8), 1003-1036. Stedman, T., Yellowlees, P., Mellsop, G., Clarke, R., & Drake, S. (1997). Measuring Consumer Outcomes in Mental Health - Field testing of selected measures of consumer outcomes in mental health. Retrieved 24 March, 2005, from http://www.health.gov.au/internet/wcms/publishing.nsf/Content/mentalhealthmhinfo-ccf-mcomh_field.htm Telford, R. & Faulkner, A. (2004). Learning about service user involvement in mental health research. Journal of Mental Health, 13, 549 – 559. Tobin M, Chen L & Leathley C (2002): Consumer participation in mental health: who wants it and why? Australian Health Review 25(3): 91-100. Usher, K. (2001). Taking neuroleptic medications as the treatment for schizophrenia: a phenomenological study. Australian and New Zealand Journal of Mental Health Nursing, 10(3), 145-55. Verpeet, E., de Casterle, B.D., Van der Arend ,A., & Gastmans, C.A.E (2005). Nurses' views on ethical codes: a focus group study. Journal of Advanced Nursing. 51, 188-95. Veit, C. T., & Ware, J. E. (1983). The structure of psychological distress and well-being in general populations. Journal of Consulting and Clinical Psychology, 51, 730742. Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Medical Care, 3, 473-483. Weinstein, J. (2006). Involving mental health service users in quality assurance. Health Expectations, 9, 98-109. Whiteford HA (1998): Renewing Australia's mental health strategy. Medical Journal of Australia. 169(8): 432-4. Wing, J. (1994). Health of the Nation Outcome Scales: HoNOS field trials. London: Royal College of Psychiatrists Research Unit.


63

9 9.1

Appendix 1 – Psych Action and Training (PAT)

Description of the PAT Group

The PAT group was formed due to a perceived need to increase opportunities for consumer perspective education and training in the Mental Health Service sector. The Consumer Academic and another CPNRP staff member – the facilitator of the Senior Nurses’ Forum – felt that ground-level consumer perspective activities could be progressed through the formation and regular meeting of a group of key consumer and nurse educators The first meeting of the PAT group occurred in October 2002. PAT meetings are attended by individuals with a demonstrated strong interest in consumer perspective. Membership currently includes the Consumer Academic, eight consumer educators – most are working as consultants in Victorian Area Mental Health Services and the Victorian Mental Illness Awareness Council, also includes trans-cultural and youth representation – and four senior nurse educators. 9.2

PAT Philosophy

PAT operates from a belief that all projects in the mental health field are best conducted by or in collaboration with consumers and where consumer perspective is the privileged perspective. PAT operates from the belief that all work done by consumers is to be properly paid for. PAT acknowledges that in all instances of consumer perspective work in the field of mental health, it is the people who currently use the service whose contributions are the most valuable of all. 9.3

What are the PAT group’s criteria?

PAT is interested in projects that: • • • • • 9.4

Have committed funds to consumer perspective work Demonstrate preparatory thinking about why and how consumer perspective is important to the project Seek consumer perspective from the beginning of projects rather than midway through Have contacts already with local consumer groups Have a history of good collaborative work with consumers What can PAT do?

PAT can offer consumer perspective on projects. Professor Brenda Happell, Central Queensland University, 2007.

64

PAT has a small consumer perspective resource library of materials that are hard to find. These resources have been collected over many years. PAT can be contracted to give advice on processes, guidelines and principles of working with consumers, or carry out aspects of project design, research or evaluation. We also have contacts with large networks of consumers. PAT works closely with the Victorian Mental Illness Awareness Council. We can help with using consumer perspective in areas of: • • • • • •

Research projects Practice issues Supervision using consumer perspective Supporting consumer networks User-monitoring of services Evaluation of existing programs and projects

Pat can also be contracted to carry out work in such areas as the above, and: • • • • • 9.5

Designing processes for involving consumers in projects Consumer perspective lit searches and reviews Conducting research Conducting needs analyses Conducting targeted training for other consumers and staff Education and Training:

PAT can be contracted to provide training to consumers, clinicians, and non-clinical workers in tertiary settings, services, or in the community. PAT can be contracted to: • • • • • 9.6

Produce guidelines for consumer perspective content in courses Produce syllabus content Support and resource Consumer Educators already involved in the training and education of mental health practitioners Assist organisations and staff with sensitisation to consumer perspective in education Find consumer perspective literature Research:

PAT is interested in research where consumer perspective directs the enquiry, content and methodology of research. PAT can be engaged to undertake research. We can also be contracted to: Professor Brenda Happell, Central Queensland University, 2007.

65

• • • • 9.7

Advise on what kinds of consumer perspective structures to use Find consumer perspective literature Develop suitable models for consultations with consumers Develop research methodologies suited to specific projects Evaluation:

PAT can be engaged to undertake evaluations. We can also be contracted to: • • • 9.8

Develop suitable methodologies for gaining evaluative commentary from consumers Find consumer perspective literature Advise on how evaluative commentary can be used to inform change Consumer Participation

PAT can be engaged to undertake a consumer participation project. We can also be contracted to: • • • • 9.9

Work with organizations on sensitization to consumer perspectives Advise on good methods of working with consumers Advise on structures for consumer participation initiatives Use existing consumer networks to inform comparative examples of what is working well First Project – Offering training opportunities to Consumer Educators

Together, the VMIAC, CPNRP and PAT will be offering professional development training to Consumer Educators so that they will be in a more confident position to try different teaching methods and share educational content. A training needs analysis was carried out by the VMIAC in Moama in March of this year. The Consumer Academic was invited to attend day one and undertook to write up the identified areas of interest from discussions and drafted a program. At the next Consumer Consultant forum, those present further contributed to the shape of the training. The first day of training will take place on October 20 and will include content: • • • • •

Public speaking Using PowerPoint Consumer perspective content Working with small groups Ice-breakers

PAT continues to meet on a six weekly basis and is continually seeking funds to get consumer perspective projects off the ground. Services, institutions can contract with PAT to provide a service. Professor Brenda Happell, Central Queensland University, 2007.

66

10 Appendix 2 – Project Reference Group 10.1 Project summary There is an obvious need for user-derived strategies to evaluate the utility and effectiveness of mental health services. Recent evidence indicates that domains of primary interest to consumers are not canvassed by measures currently utilised by Victorian services. In response to the need for user-derived strategies, the CPNRP sought and received funding from the ANZCMHN Research Board to undertake a research project in collaboration with the PAT Group. The PAT Group is a group of consumer and senior psychiatric nurse educators who aim to promote consumer perspectives in the professional development activities of mental health sector. The main aims of the project are to conduct a review and evaluation of outcome measures used throughout Victorian mental health services, including the HoNOS and BASIS-32, and to identify alternative strategies for service evaluation to be used alongside existing measures. A group discussion between PAT Group members and two focus groups discussions with consumers of mental health services will be used for this purpose. Qualitative content analysis will be used to analyse transcripts of these discussions. A Reference Group comprising eight collaborative partners with expertise in the field will provide feedback on the conduct and outcomes of this project. Project findings will be disseminated via a brief project report, journal articles and conference presentations. 10.2 Aims of the Reference Group •

Give advice and feedback on the implementation of the project.

•

Give advice and feedback on the development of the project.

•

Give feedback on the interpretation of study findings.

•

Assist with the dissemination of project findings and recommendations.

10.3 Operation of the Reference Group •

It is expected that the Reference Group will meet approximately four times throughout the duration of the project.

•

Minutes of meetings will be distributed to all Reference Group members.

•

Email will be used to complement face-to-face meetings.

•

Reference Group members will give feedback on the project between formal meetings, where necessary.


67

11 Appendix 3 – Discussion Stimulus for PAT Discussions Points raised: • PAT can draw upon prior conversations around philosophies • Prompt: human rights, social justice, social rights, how would you articulate these • How do people relate, consumer – clinician? • What constitutes a relationship? What do you value in a relationship (power difference)? • What is an outcome? This needs to be clearly articulated; outcome implies an endpoint. However, recovery can be staged. Are there various goals along the way? • Individual versus aggregate • What does an evaluation of a mental health service look like?

Proposed ‘discussion stimulus’: “Any new evaluation framework will be consumer-driven and will take into consideration the relationship between consumers and clinicians. What is an appropriate philosophy to guide the development of the framework?”

What does the PAT Group think about evaluation? • •

Simply measuring outcomes doesn’t change the service How can we be sure the any evaluation will impact on the way services are run, will it make a difference?


68