Using the Australian Therapy Outcome Measures ... - Semantic Scholar

Using the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs-OT) to Measure Outcomes for Clients Following Stroke Carolyn A. Unsworth

Purpose: To examine a range of measures used to document client outcomes following stroke, describe the Australian Therapy Outcome Measure for Occupational Therapy (AusTOMs-OT) as a tool suitable to measure multiple outcomes, and provide an overview of three outcomes research programs using this measure. The AusTOMs-OT was developed to measure global therapy outcomes and offers therapists a choice of 12 function-focused scales (including self-care, domestic life, community life, upper limb function). Therapists evaluate the client’s status globally in relation to four domains: the underlying impairment, activity limitation, participation restriction, and distress/well-being. Method: The first study presents a comparison of outcomes for clients at two Australian acute care facilities on the self-care scale (n = 82). Similarly, the second study presented is a comparison of stroke rehabilitation outcomes using the self-care scale for clients in Sweden and Australia (n = 70). The final study is an Australian benchmarking study using the upper limb scale (n = 40). Results: All three studies demonstrated that clients improved during therapy as measured on the four domains of AusTOMs-OT. Study 3 examined client outcomes at one facility against an agreed benchmark using the AusTOMs-OT upper limb scale and found that clients attained benchmark outcomes. Conclusions: A variety of outcome measures are available for clinicians to document the progress clients make during stroke rehabilitation. However, the AusTOMs-OT can measure global outcomes across multiple domains in just a few moments. Three studies reporting outcomes for clients with stroke using the AusTOMs-OT demonstrate its utility in documenting client change during therapy and for comparing or benchmarking services. Key words: assessment, cerebrovascular accident, evaluation, evidence-based practice, mobility, outcome measurement, self-care, upper limb function

“Improved measurement will ensure the basis of rehabilitation as a science and nourish its success as a clinical service.” 1

It is well established that stroke can have devastating physical, cognitive, and behavioural consequences for individuals and can produce considerable emotional and financial burden for families and the community.2 However, health care services find it very difficult to report these outcomes in a standard manner,3 because there is ongoing dispute on exactly how and what to measure. An outcome may be defined as ...the results of production processes which precede them in space-time, acting on inputs in a given environment. In healthcare, the term “outcome” usually refers to postintervention results or measurements—the observed outcomes of an intervention—whether or not one can confidently attribute those results to the preceding intervention (process).4(p110)

Reporting outcomes in a routine manner using standardized measures enables therapists to conduct research and communicate findings easily

with colleagues, both locally and internationally. Use of standard outcome measures also enables therapists to identify services that excel in the care they offer and outcomes they achieve and identify the treatments or approaches to care that underlie their success. Therapists need to report client outcomes to demonstrate • types and variations in clients referred for therapy, • that change is occurring to client/caregivers/ insurers, • that the best kinds of therapy are used, • that the therapy is offered at the best time, Carolyn A. Unsworth, PhD BAppSci (OccTher), is Associate Professor, School of Occupational Therapy, La Trobe University, Kingsbury Drive, Bundoora VIC, Australia. Top Stroke Rehabil 2008;15(4):351–364 © 2008 Thomas Land Publishers, Inc. www.thomasland.com doi: 10.1310/tsr1504-351

351

352

Topics in Stroke Rehabilitation/July-August 2008

• how the service is performing when compared with other services, and • to benchmark therapy services. Using the Structure, Process and Clinical Out come (SPO) framework proposed by Donabedian5 and adapted for rehabilitation by Hoenig et al.,3 it is apparent that rehabilitation facilities are adept at measuring and documenting structure and process outcomes such as the amount, type, timing, and setting of therapy provided. However therapists seem less able to routinely record information on the client’s status when they leave the health care facility. The International Classification of Functioning, Disability and Health (ICF)6 model provides an internationally known framework for considering the individual’s status in terms of the disease, impairment, activity limitation, participation restriction, and overall quality of life of the individual. Through recording these outcomes, therapists begin to accumulate evidence concerning the outcomes of our therapy and approaches to delivering stroke care.7 Although evidence-based practice (EBP) is still in its infancy in the allied health sciences, one way to expand opportunities for EBP is to ensure that there are psychometrically sound outcome measures available that can be quickly and easily used by therapists on a regular basis. Over the past 30 years, numerous tools have been reported in the literature to measure the impact of stroke on a wide variety of areas such as motor skills, cognition and perception, personal activities of daily living, instrumental activities of daily living, pain, quality of life, and caregiver burden. Assessments in relation to a range of categories (such as personal activities of daily living or psychological status) that have been reported in the research literature as suitable for use with clients following stroke are presented in Table 1. For example, in a systematic review of randomized studies examining whether occupational therapy focusing on personal activities of daily living (PADLs) improved recovery for clients with stroke, the most commonly used measures were the Barthel Index and the Nottingham Extended ADL Index.8 If the focus of research is to examine a particular aspect of client care such as PADLs, these measures are ideal. However, given the range of problems that result following stroke, clinicians often

need to measure therapy outcomes more broadly for clients who have had a stroke. Most clinicians would not have the time to administer more than one or two of the measures that are listed in Table 1. Hence, while an occupational therapist may administer the FIMTM* and Rivermead Perceptual Assessment Battery (RPAB) (as cited in Table 1) to measure outcomes related to self-care burden and perceptual status, this clinician would not capture other important outcomes such as the client’s level of well-being or motor functioning. Haigh et al.9 distributed a questionnaire to 866 rehabilitation units across Europe (Scandinavia/ Central Mediterranean/Northwestern Europe, UK, and Eastern Mediterranean) in 1998 to determine what outcome measures were most frequently used. They received replies from 418 units, and the results indicated that for clients with stroke the most commonly used assessments were the FIMTM (approximate number of assessments per annum: 14,441), Barthel Index (original and modified; 13,203), Mini-Mental State Examination (7,840), Modified Ashworth Spasticity Scale (4,790), Glasgow Coma Scale (3,901), National Institutes of Health Stroke Scale (2,863), Rivermead Behavioural Memory Test (2,814), Motoricity Index (2,201), Western Aphasia Battery (1,750), and the Rivermead Mobility Index (1,688). What is surprising from this review is that there was low level use of generic measures such as the Medical Outcome Study Short Form General Health Survey (SF-36) or Nottingham Health Profile (NHP) (cited in Table 1). In addition, these measures focus on outcome for impairment or aspects of activity limitation rather than including all domains of health states as identified by the ICF.6 Finally, none of the measures commonly used at that time in Europe included a measure of client participation. Therefore, it seems important to adopt measures that can be used for clients with stroke and capture information on client outcome across several domains such as impairment, activity limitation, participation, and the client’s general well-being. One such measure is the Australian Therapy Outcome Measures (AusTOMs).

*FIM TM is a trademark of Uniform Data System for Medical Rehabilitation, a division of UB Foundation Activities, Inc.

Using AusTOMs to Measure Oucomes After Stroke

Table 1. Selection of outcome measures that can be used in stroke rehabilitation Domain/area

Measure

General health status/general outcome measures

Stroke Impact Scale (SIS) Version 2.0 28,29 Short Form Health Survey (SF-36)30 Sickness Impact Profile (SIP)31 Older American Resources and Services (OARS) Questionnaire32 Nottingham Health Profile (NHP)33 Canadian Occupational Performance Measure (COPM)34 Stroke Impairment Assessment Set (SIAS)35 National Institute of Health Stroke Scale (NIHSS)36 Ashworth Spasticity Scale37,38 Tardieu Scale39 Motricity Index40 Rivermead Mobility Index41,42 Chedoke McMaster Stroke Assessment43 Disabilities of the Arm, Shoulder and Hand (DASH)44 Chedoke Arm and Hand Activity Inventory45 Action Research Arm Test46 Barthel Index47 FIMTM27 Northwick Park ADL Scale48 Nottingham Stroke Dressing Assessment49 Rankin Scale50 Assessment of Living Skills and Resources (ALSAR)51,52 Frenchay53 Nottingham Extended Activities of Daily Living Scale54 Assessment of Motor and Process Skills55 Functional Assessment of Communication Skills56 Mini-Mental State57 Burden of Stroke Scale (BOSS)58 Rivermead Percaptual Assessment Battery (RPAB)59 Short Behavior Scale60 Behavioural Inattention Test (BIT)61 Rivermead Behavioural Memory Test (RBMT)62 Social Support Questionnaire (SSQ – 6)63 Social Support Inventory for Stroke Survivors (SSISS)64 McGill Pain Questionnaire65,66 Visual Analogue Scale67 Disability Questionnaire68 Quality of Life Index69,70 Life Satisfaction Questionnaire71 Quality of Life Inventory72 Community Integration Questionnaire73,74 Reintegration to Normal Living Index75,76 Rivermead Life Goals Questionnaire77 Nottingham Leisure Questionnaire78 Patient Judgment of Hospital Quality79,80 Zarit Burden Scale81 Caregiving Burden Scale82 Caregiver Strain Index83 London Handicap Scale26 Personal Care – Participation Scales (PC-PART)84 Craig Handicap Assessment and Reporting Technique (CHART)85

Impairment Mobility/ movement

Upper limb function

Personal Activities of Daily Living (PADL)

Instrumental Activities of Daily Living (IADL)

Communication Psychological status/cognition/behavior

Social support Pain

Quality of life/ reintegration

Leisure Satisfaction with services Caregiver burden

Participation

353

354


Australian Therapy Outcome Measures (AusTOMs) The AusTOMs was designed in Australia. The AusTOMs is based on the TOMs, which was originally developed in the UK by Enderby, John, and Petheram for use by speech pathologists, physiotherapists, occupational therapists, and rehabilitation nurses.10–12 The AusTOMs differs from TOM as AusTOMs provides separate scales for occupational therapy, physiotherapy, and speech pathology and the OT scales are focussed around occupations such as self-care and transfers rather than areas of practice (e.g., wound care) or disease processes (e.g., stroke). The TOMs and AusTOMs also draw on the concepts of the ICF6 and base their scoring around the domains of impairment, activity limitation, and participation restriction and the additional domain of distress/well-being. The AusTOMs is a therapist-administered tool and is suitable for use with clients of all ages and all disabilities in any type of setting. This article focuses on the AusTOMs for Occupational Therapy.

as per usual practice, selects AusTOMs-OT scales to match the goals and then rates the client on each scale according to four domains (impairment, activity/limitation, participation/restriction, and distress/well-being). Each domain is scored on an 11-point ordinal scale, in which there are six defined scores from 0 (complete problem) to 5 (no problem); half points can be used. The selection use and scoring of the AusTOMs-OT scales is fully described in the AusTOMs-OT manual.17,18 Details concerning the reliability of the scales have been reported in Morris et al.19 In addition, a detailed reliability study of the self-care scale in which seven occupational therapists used 15 paper case studies was reported by Scott and colleagues.20 Interrater intraclass corrleation coefficients (ICCs) of over 0.79 were obtained for three domains (activity limitation, participation restriction, and distress/well-being), and over 0.70 for impairment. Test-retest reliability was also reported to be quite high, with ICCs of 0.88 for activity limitation, 0.81 for participation restriction, 0.94 for distress/wellbeing, and 0.74 for impairment. Information on face and construct (concurrent) validity have also been reported.13,14,21

The AusTOMs for Occupational Therapy The development of the AusTOMs in general13 and the AusTOMs-OT14 has been fully described. However, in summary, the AusTOM-OT scales were developed within the classical measurement model. 15,16 Twelve scales were developed for use with clients of any age and any diagnosis as follows: (1) learning and applying knowledge; (2) functional walking and mobility; (3) upper limb use; (4) carrying out daily life tasks and routines; (5) transfers; (6) using transport; (7) self-care; (8) domestic life – home; (9) domestic life – managing resources; (10) interpersonal interactions and relationships; (11) work, employment, and education; (12) community life, recreation, leisure, and play. These scales were developed around the construct of activity/limitation and assessing clients in relation to the three ICF 6 domains of impairment, activity/limitation, and participation/restriction and the fourth construct of distress/well-being from the TOM.10 In the AusTOMs, these four areas are described as domains. To use the scales, the therapist sets goals

Research measuring stroke outcomes using the AusTOMs-OT

Three studies that report AusTOMs data for clients following stroke are presented in this article. The first two illustrate simple comparisons of outcome data for clients with stroke. The first study was conducted at two acute care facilities in Melbourne, Australia.22 This study was prompted by therapists who felt that AusTOMs-OT may not be sensitive at capturing change during the acute care stage of stroke recovery. The second comparison study draws on data from rehabilitation facilities in Melbourne, Australia, and the Skövde region in Sweden. The final study describes how AusTOMs-OT can be used to formally benchmark client outcomes.23 Again, this study arose from the clinical need of occupational therapists who wanted to know how the outcomes they achieved with clients receiving rehabilitation compared to average outcomes for clients in the region. Ethical permission for each study was sought and obtained from the La Trobe University

Ethics Committee and the ethics committees of the participating sites. All data were collected by qualified occupational therapists who had been trained to administer the AusTOMs. The first study reports data using nonparametric statistics, because this was one of the original studies published on AusTOMs for occupational therapy. In subsequent studies reported in this article, the data were treated parametrically. Power analyses were conducted prior to each study to determine the minimal sample sizes for 90% power using nQuery Advisor. For each study, samples sizes of 10–11 conferred 90% power in rejecting the twosided null hypothesis at a significance level of p = .05. For all analyses, p values of .05 or less were considered statistically significant. Study 1: Comparing client outcomes poststroke in two acute care facilities Study design and method

The aim of this study was to demonstrate how AusTOMs-OT self-care data could be combined with client demographic information to compare client outcomes at two large acute care facilities in Melbourne, Australia (population 3.8 million). Site A had 52 neurological beds (with 18 occupational therapists, 3 in neurological care) and Site B had 42 neurological beds (with 13.8 occupational therapists, 3.2 in neurological care). The sample was comprised of 42 clients at Site A (age M = 64 years, SD = 17.91), number of occupational therapy contacts (M = 2.93, SD = 1.66), length of stay (M = 6.19 days, SD = 8.03), and 40 clients at Site B (age M = 56 years, SD = 22.56), number of occupational therapy contacts (M = 5.74, SD = 4.19), and length of stay (M = 9.82 days, SD = 9.27). Client demographic details were recorded, alongside AusTOMs data for Scale 7 (self-care). Full details of the method are reported in Unsworth and Duncombe.17 Results: Comparison of two acute care facilities

Initially client demographic data were compared. There was no difference between the sites in terms of client age (t = 1.705, p = .092) or length of stay (t = –1.899, p = .061), however occupational


355

therapists at Site B had a higher number of contacts with clients (t = –4.028, p < .001). Next, comparisons were made between client admission profiles for self-care at Sites A and B. Using a Mann-Whitney U test (for independent samples), it was found that clients at Site B had statistically significantly more severe impairments (U = 621, p = .034) and activity limitations (U = 533, p = .003). It was then established that more clients improved at both sites than stayed the same or deteriorated in relation to all four domains. Wilcoxon signed rank tests produced statistically significant Z scores for all domains: Site A, impairment (Z = –3.908, p < .001), activity/limitation (Z = –5.174, p < .001), participation/restriction (Z = –4.044, p < .001), and distress/well-being (Z = –3.699, p < .001); Site B, impairment (Z = –4.981, p < .001), activity/limitation (Z = –5.123, p < .001), participation/restriction (Z = –4.548, p < .001), and distress/well-being (Z = –4.300, p < .001). Using a chi-square analysis, it was found that more clients improved (rather than stayed same or deteriorated) at Site B than Site A, in relation to impairment, c2(1) = 9.259, p = .002, but there were no differences in terms of the other domains. Finally, a comparison was made of client status at discharge, again using a Mann-Whitney U test. No significant differences between client self-care scores in relation to the four domains were found. Summary and conclusion

In summary, Study 1 found that clients with neurological problems at Site B had more severe impairments and activity/limitations than clients at Site A on admission. Clients at Site B received more occupational therapy contacts. Clients at both sites improved significantly between admission and discharge in relation to all AusTOMs domains, however a greater number of clients improved in relation to impairment at Site B. By the time of discharge, there were no significant differences between the groups. It is possible that the increased number of occupational therapy contacts at Site B may be associated with improved client impairment outcomes. This study was also effective in demonstrating to clinicians that the AusTOMs was useful in providing information on client outcomes even during relatively brief acute

356


Table 2. Australian and Swedish sample characteristics Australian sample (n = 16)

Swedish sample (n = 54)

Demographic information

M

SD

M

SD

Age, years Length of intervention for self-care goal, days Number of occupational therapy contacts

72.81 31.56 21.06

12.58 23.79 14.67

77.63 16.13 9.54

10.19 18.08 6.76

care admissions. One of the limitations of this study was that the data collected for the number of occupational therapy contacts did not include data about the nature or length of the contact. This problem was overcome in the subsequent studies reported below. Study 2: Comparing client outcomes poststroke in rehabilitation facilities in Sweden and Australia Study design and method

Similar to Study 1, the aim of this study was to demonstrate how AusTOMs data could be combined with client demographic information to compare outcomes at different facilities. In this case, the comparison was made between different countries. Although exploratory in nature, this type of study helps therapists to communicate internationally, in a uniform manner, about the status of clients with stroke and what clients at the various services gain through participating in rehabilitation. Participants included 16 Australian adults (11 female, 4 male, 1 unknown) aged between 42 and 94 years (M = 72.81 years, SD = 12.58) and 54 Swedish adults (25 female, 29 male) between the ages of 47 and 93 years (M = 77.63 years, SD = 10.19). Participants were recruited from two rehabilitation facilities in Sweden and six rehabilitation hospitals located in metropolitan Melbourne, Australia. The mean characteristics of the Australian and Swedish groups are presented in Table 2. All participants had a primary diagnosis of stroke. AusTOMs data from both samples were collected over an 8-month period, on client admission to and discharge from working on the client’s self-care goal (AusTOMs Scale 7).

Results: Comparison of Australian and Swedish samples

There were significant differences between the Australian and Swedish samples for length of stay (days) for the self-care goal (t = 2.78, p = .007) and the number of occupational therapy contacts (t = 3.05, p = .007). There was no significant difference between the Australian and Swedish groups for age or sex. The AusTOMs-OT self-care scale data at the commencement (goal-start) and termination (goal-end) of treatment were compared between groups using independent t tests. There was a significant difference between the Australian and Swedish samples for goal-start AusTOMs-OT selfcare scale scores on the impairment domain (t = –2.03, p = .046), with the Swedish sample having a significantly higher mean (less impaired) than the Australian sample (see Table 3). There were no significant differences between the samples for goal-end AusTOMs-OT self-care scale scores. Two-way analyses of variance (ANOVAs) were computed to determine the degree of difference between the Australian and the Swedish sample outcomes on each of the four AusTOMs-OT domains. Table 4 presents the ANOVA results. The first column (SAMPLE) presents the p values for the differences between the Australian and Swedish samples at PRE and POST. There was a significant difference between the samples for the impairment (F = 5.01, p = .029) domain of the AusTOMs-OT self-care scale. The second column (PRE-POST) presents the p values for the difference between goal-start and goal-end. The change in scores between goal-start and goal-end were statistically significant for all four domains when both samples were considered together (p < .001, for all four domains). That is, as a whole, participants in both


357

Table 3. Differences between the Australian and Swedish samples for client demographics and AusTOMsOT self-care scale scores at goal-start Australian sample

Swedish sample

Difference between samples Two-sided p value

Demographics and self-care scores

n

Mean (SD)

n

Mean (SD)

t

Age Sex AusTOMs-OT self-care scale goal-start scores Impairment Activity limitation Participation restriction Distress /well being

16 15

72.81 (12.58) M/F = 4/11

54 54

77.63 (10.19) M/F = 29/25

–1.572

.121 .083

16 16 16 16

1.88 (1.09) 2.19 (0.66) 2.31 (1.29) 3.16 (0.72)

54 54 54 54

2.54 (1.16) 2.44 (1.23) 3.08 (1.41) 3.32 (1.26)

–2.03 –1.10 –1.96 –0.64

.046* .278 .054 .528

*Statistically significant result as p ≤ .05.

Table 4. Two-way ANOVA for the effect of the sample (Australian/Swedish) on the degree of change between goal-start and goal-end SAMPLE

PRE-POST

SAMPLE x PRE-POST

AusTOMs-OT self-care scale domain

F

p

F

p

F

p

Impairment Activity limitation Participation restriction Distress/well-being

5.01 0.64 3.16 0.11

.029* .427 .080 .916

48.68 61.30 23.35 19.32

.001* .001* .001* .001*

0.59 0.06 0.70 0.69

.808 .812 .406 .409


samples improved from goal-start to goal-end on all domains of the AusTOMs-OT self-care scale. Finally, the third column (SAMPLE x PRE-POST) refers to whether there was an interaction between SAMPLE and PRE-POST. That is, whether there was a differential effect of the sample (Australia vs. Sweden) on the difference between goal-start and goal-end. There was no interaction effect (p > .05) for any of the four AusTOMs domains. In other words, the degree of change in scores on the impairment, activity limitation, participation restriction, and distress/well-being domains on the AusTOMs-OT self-care scale were indistinguishable according to the sample. Summary and conclusion

This study demonstrated that the participants in the Swedish sample were not as impaired on admission to the goal of self-care as participants in the Australian sample and that the mean length of time (in days) spent working on self-care rehabili-

tation was half that of the Australian sample, with half the number of therapy contacts. All clients improved over their admission in relation to the four AusTOMs domains. The two-way ANOVA showed that there was no interaction effect between SAMPLE and PRE-POST. In other words, sample membership (Australian or Swedish) did not have an effect on gains made on the AusTOMs self-care scale from goal-start to goal-end. To summarise, Australian clients were more impaired overall on admission to rehabilitation, they spent almost twice as long in rehabilitation (with twice as many therapy sessions), and by the time of discharge there were no differences in the samples in terms of AusTOMs outcome on the self-care scale. However, whereas Australian participants were more impaired by their stroke, there was no difference in their activity limitation on admission or discharge, suggesting that the participants did not make additional functional gains for the additional time that they spent in rehabilitation. However, the findings must be considered in light

358


of the number and type of client co-morbidities (for example, participants in the Australian sample may have been in rehabilitation longer due to factors other than their stroke) and the types of therapy offered to clients (for example, the therapies offered to clients in Sweden may have been more efficient in achieving outcomes than those offered in Australia). Future comparisons of outcomes between facilities should carefully document these factors using similar systems such as ICD-10 to record co-morbidities and a predetermined list of therapies and what they involve, with a checklist to be completed by each therapist on client discharge. Notwithstanding these limitations, this comparison study shows the potential value of using a tool such as AusTOMs to investigate and compare, across countries, what outcomes are achieved with clients following stroke.

The results indicated several possible areas for improvement in service delivery in relation to the benchmarks. The aim of the study reported in this article was to benchmark outcomes in upper limb rehabilitation for clients receiving rehabilitation at one facility with an agreed benchmark. Hence, the study had two parts. The first involved establishing the benchmark and the second was comparing the sample data (referred to as the treatment sample) to the benchmark data. A full description of the benchmark sample and confirmation of these data as a true benchmark are provided in Unsworth, Bearup, and Rickard.23 In the current article, a summary of the two samples and the comparison between them are presented. Data were collected from 20 participants in the benchmark sample (14 female, 5 male, 1 unknown) aged between 42 and 94 years (M = 70.65, SD = 14.26) and 20 participants in the treatment sample (8 female, 12 male) between the ages of 31 and 87 years (M = 54.85, SD = 15.87). The mean characteristics of the benchmark and treatment groups are presented in Table 5. The characteristics of the sample of occupational therapists were not obtained as their participation was anonymous. AusTOMs data were collected over an 8-month period for the benchmark sample and 6-month period for the treatment sample on client admission and discharge from working on the client’s upper limb rehabilitation goal (AusTOMs Scale 3). All participants had a primary diagnosis of stroke.

Study 3: Benchmarking upper limb outcomes following stroke rehabilitation Study design and method

Benchmarking is one way to ensure quality and deliver best practice.24 Benchmarking involves comparing performance of one service against an agreed criteria or aggregate data and can be used to examine variations in clients referred for therapy, the types and variations in amounts and lengths of therapy, the amounts and variation in changes associated with therapy, and the variation in client profiles at discharge. For example in the UK, Enderby et al.25 compared TOM outcomes for clients with acquired neurological diseases at one National Trust Service against the mean performance of clients from six other Trusts.

Results: Comparison of benchmark and treatment samples

Initially, it was established that the types of upper limb interventions carried out with clients in

Table 5. Benchmark and treatment group characteristics Benchmark sample (n = 20)

Treatment sample (n = 20)

Demographic information

M

SD

M

SD

Age, years Length of intervention for upper limb goal, days Number of occupational therapy contacts

70.65 39.05 19.00

14.26 28.08 14.37

54.85 51.55 25.30

15.87 22.81 14.30


359

Table 6. Two-way ANOVA for the effect of the sample (benchmark/treatment) on the degree of change between goal-start and goal-end SAMPLE

PRE-POST

SAMPLE x PRE-POST

AusTOMs-OT upper limb use scale domain

F

p

F

p

F

p

Impairment Activity limitation Participation restriction Distress/well-being

6.64 7.38 0.50 3.49

.012* .008* .482 .066

17.32 20.23 15.74 17.77

.001* .001* .001* .001*

0.09 0.10 0.24 1.26

.767 .756 .626 .266


the two samples were similar. The most frequently conducted interventions were hand function within daily tasks, gross upper limb function within daily tasks, compensatory strategies, stretching, soft and hard splints, motor learning treatment, and oedema control. To determine sample equivalence at the start and end of the therapy goal, participant demographic and AusTOMs-OT upper limb use scale data for the treatment sample were compared with data from the benchmark sample using t tests. There was a significant difference between the benchmark and treatment samples for age (t = 3.31, p = .002). There were no significant differences between the samples for the length of intervention for upper limb dysfunction (t = –1.55, p = .131), the number of occupational therapy contacts (t = –1.39, p = .173), gender balance (calculated using the Fisher exact test, p = .054), nor any of the domains on the AusTOMs-OT upper limb use scale on goal-start: impairment (t = –1.90, p = .065), activity limitation (t = –1.63, p = .121), participation restriction (t = –0.15, p = .882), and distress/well-being (t = .056, p = .056). Similarly, there were no differences between the samples on any of the domains on the AusTOMs-OT upper limb use scale on goal-end: impairment (t = –1.74, p = .090), activity limitation (t = –2.24, p = .031), participation restriction (t = .87, p = .390), and distress/well-being (t =.57, p = .571). Two-way ANOVA was performed to determine the degree of difference between the benchmark and the treatment samples in terms of amount of change obtained on the AusTOMs-OT between goal-start and goal-end. Table 6 presents the ANOVA: The first column (SAMPLE) presents the p values for the differences between the

benchmark and treatment samples across both goal-start and goal-end. There were significant differences between the benchmark and the treatment samples in the means for the impairment (p = .012) and activity limitation (p = .008) domains of the AusTOMs-OT upper limb use scale. The benchmark sample had a significantly higher mean for the impairment domain, and the treatment sample had a significantly higher mean for the activity limitation domain. The second column (PRE-POST) presents the p values for the difference between goal-start and goal-end across both the benchmark and treatment samples. All clients’ scores on the AusTOMs-OT upper limb use scale improved significantly over the treatment period. Finally, the third column (SAMPLE x PREPOST) refers to whether there was an interaction between SAMPLE and PRE-POST. That is, whether there was a differential effect of the sample on the difference between goal-start and goal-end. There was no interaction of the sample on the difference between goal-start and goal-end (p > .05) for any of the four domains. Summary and conclusion

This study found no statistically significant differences between the treatment and the benchmark samples in terms of length of intervention for upper limb dysfunction nor the number of therapy contacts. In addition, examination of surveys completed by therapists indicated that similar types of treatments were used at the facilities. In the ANOVA, the main effects for SAMPLE revealed that the benchmark sample had a statistically significantly higher mean for impairment and the treatment sample had a

360


statistically significantly higher mean for activity limitation. This is of some interest, however on admission and then again on discharge the t tests indicated no differences between the samples for these domains, which is of more importance. The main effect for PRE-POST indicated that, overall, all clients improved between admission and discharge on the AusTOMS-OT upper limb use scale for all four domains from admission to discharge. Finally, the interaction effect for the ANOVA answers the most important question of whether the client’s sample (benchmark or treatment) affected improvement from admission to discharge as measured on the AusTOMs-OT upper limb use scale. The findings indicated that this was not the case. Participants in both samples improved in a similar manner from admission to discharge. That is, participants in the treatment sample achieved the same degree or amount of change on the AusTOMs domains for the upper limb use scale from PRE to POST as participants in the benchmark sample. To summarise, the results indicated that, overall, participants from the treatment sample were younger on average yet achieved similar outcomes as the treatment sample when measured on the AusTOMs-OT upper limb use scale. The findings of this study are reassuring for the therapists at the treatment facilities as they know they are meeting an agreed benchmark for upper limb outcomes following stroke. However, they can also now examine ways to improve outcomes with their clients beyond the benchmark. Limitations of this study include that the benchmark was set locally and may not align with international opinion or actual upper limb outcomes for clients following stroke. Discussion and Conclusion Therapists in the current health care climate face increasing pressure from insurers, employers, and clients to provide a quality, evidence-based service at the lowest possible cost. The SPO model,3,5 which was developed over 25 years ago, provides guidance for therapists to summarise clinical outcomes for clients following stroke and contextualise these with structure and process information. In terms of documenting clinical

outcomes, a huge range of tools have been developed over the past 30 years to measure client outcomes in a wide range of domains following stroke. Today we share a common international language provided by the ICF6 to describe these domains as impairment, activity limitation, and participation restriction and can extend this to include the domain of client well-being. Table 1 presented an overview of some of these tools. However, similar to the findings of Haigh et al.,9 there are relatively few outcome measures used that capture client status across a range of domains. In particular, client level of participation and distress/ well-being following stroke are increasingly seen as important to capture, and yet few tools have been developed to measure these domains. The tools that are available such as the London Handicap Scale26 would need to be administered in addition to measures that capture impairment and activity limitation such as the FIMTM,27 thus increasing the time required to record client outcomes. The AusTOMs for Occupational Therapy17,18 was presented as a new tool that could be used to capture outcomes for clients following stroke in a few moments and across several health domains. To support the utility of AusTOMs-OT as a measure of client outcomes following stroke, three studies were summarised. These studies presented comparison or benchmark research providing client outcomes across four domains (impairment, activity limitation, participation restriction, and distress/well-being) using just 2 of the 12 AusTOMs-OT scales, upper limb use and self-care. One of the advantages of using a tool such as AusTOMs to compare or benchmark services is that facilities can begin to identify their strengths and limitations. For example, in the study cited earlier by Enderby et al.,25 it was found that different facilities achieve better outcomes in different areas, which is probably due to differences in emphasis placed on different interventions by the treating occupational therapists. One facility obtained better outcomes on psychosocial outcomes than the other facilities where interventions were more focused on improving physical impairments and functioning. However, the main limitations of the three studies reported in this article relate to the relatively small sample sizes (although power was adequate) and


relatively limited information on the specific type and timing of administration of treatments for upper limb and self-care rehabilitation following stroke. Although, on the whole, differences were not found in these three studies between facilities, future research may reveal differences. If this occurs, researchers should be able to determine whether such differences are due to the specific types of treatments administered or the skill level of the therapists who administer these treatments. In conclusion, the studies reported in this article suggest that the AusTOMs shows promise as a quick and simple outcome measure for occupational therapists who wish to study the outcomes of their stroke service. If therapists can document client clinical outcomes and summarise these along with structure and process information, then patterns indicating the strengths and weaknesses in the service may begin to emerge. The subsequent opportunity to compare services and reflect on care may lead to improvements in services offered and a consistency in level of care, regardless of where clients receive rehabilitation following stroke. Acknowledgments Sincere thanks are expressed to the clients who participated in the data collection for the studies reported in this article and to my research and clinical colleagues who collected data and assisted with data analysis as follows:

361

Dianne Duncombe (research assistant) for her work on Study 1; Yvonne Widell and Anna-Karin Birath (occupational therapists, Skövde region Sweden), Dr. Elizabeth Elgmark and Professor Torbjörn Falkmer, Jönköping University, and Carmela Germano (research assistant) for their contribution to Study 2; and Andrea Bearup (occupational therapist, McKellar Centre), Kate Rickard (occupational therapist, Southern Health Network), Carmela Germano (research assistant), and Dr. John Ludgbrook (biostatistician) for their work on Study 3. This article draws some data from the foundation AusTOMs study that involved the following team members from La Trobe University Faculty of Health Sciences: Professor Alison Perry (Principal Investigator), Professor Meg Morris (Co-investigator), Associate Professor Carolyn Unsworth (Co-investigator), Professor Stephen Duckett, (Co-investigator), Ms. Jemma Skeat, Dr. Karen Dodd, Dr. Nicholas Taylor, Ms. Karen Reilly, and Ms. Dianne Duncombe (Research Associates). Professor Pam Enderby assisted the research team at La Trobe University in the application to the Commonwealth to support the AusTOMs project, and both Professor Enderby and Dr Alexandra John from Sheffield University, UK, were associate researchers to this project, providing the Research Team with advice, discussion, and support in this development of the AusTOMs.

REFERENCES

1. Johnston MV, Keith RA, Hinderer SR. Measurement

standards for interdisciplinary medical rehabilitation. Arch Phys Med Rehabil. 1992;73:S3–S23. 2. The Ottawa Panel. Ottawa Panel evidencebased clinical practice guidelines for post-stroke rehabilitation. Top Stroke Rehabil. 2006;13:1–269. 3. Hoenig H, Horner RD, Duncan PW, Clipp E, Hamilton B. New horizons in stroke rehabilitation research. J Rehabil Res Dev. 1999;36(1):19–31. 4. Boyce N, McNeil J, Graves D, Dunt D. Quality and Outcome Indicators for Acute Healthcare Services. Canberra: Australian Goverment Printing Service; 1997. 5. Donabedian A. The Definition of Quality and Approaches to Its Assessment. Michigan: Health Administration Press; 1980. 6. World Health Organization. International Classification of Functioning, Disability and Health: ICF

Short Version. Geneva: World Health Organization; 2001. 7. Yamamoto L, Magalong E. Outcome measures in stroke. Crit Care Nurs Q. 2003;26(4):283–293. 8. Legg L, Drummond A, Leonardi-Bee J, et al. Occupational therapy for patients with problems in personal activities of daily living after stroke: systematic review of randomised trials. Br Med J. 2007;335:922–930. 9. Haigh R, Tennant A, Beiring-Sørensen F, et al. The use of outcome measures in physical medicine and rehabilitation within Europe. J Rehabil Med. 2001;33:273–278. 10. Enderby P, John A. Therapy Outcome Measures for Speech and Language Pathology. San Diego, CA: Singular Publishing Group; 1997. 11. Enderby P, John A, Petheram B. Therapy Outcome Measures: Physiotherapy, Occupational Therapy,

362


Rehabilitation Nursing. San Diego: Singular; 1998. 12. Enderby P, John A, Petheram B. Therapy Outcome Measures for Rehabilitation Professionals: Speech and Language Therapy, Physiotherapy, Occupational Therapy, Rehabilitation Nursing, Hearing Therapists. 2nd ed. Chichester, UK: John Wiley & Sons; 2006. 13. Perry A, Morris M, Unsworth C, et al. Therapy Outcome Measures of Allied Health Practitioners in Australia: The AusTOMs. Int J Qual Health Care. 2004;16(4):285–291. 14. Unsworth C. Measuring outcomes using the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs-OT): data description and tool sensitivity. Br J Occup Ther. 2005;68(8):354–366. 15. DeVellis RF. Scale Development. Theory and Applications. Vol. 26. Newbury Park, London, New Delhi: Sage Publications; 1991. 16. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. Oxford: Oxford University Press; 1991. 17. Unsworth C, Duncomb D. AusTOMs for Occupational Therapy. Melbourne: La Trobe University; 2004. 18. Unsworth C, Duncomb D. AusTOMs for Occupational Therapy. 2nd ed. Melbourne: La Trobe University; 2007. 19. Morris M, Perry A, Unsworth C, et al. Reliability of the Australian Therapy Outcome Measures for quantifying disability and health. Int J Ther Rehabil. 2005;12(8):340–346. 20. Scott F, Unsworth C, Fricke J, Taylor N. Reliability of the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs-OT) self-care scale. Aust Occup Ther J . 2006;53:265–276. 21. Unsworth C, Duckett S, Duncomb D, Perry A, Skeat J, Taylor N. Validity of the AusTOM Scales: a comparison of the AusTOMs and EuroQol-5D. Health Qual Life Outcomes. 2004;2:1–12. 22. Unsworth C, Duncomb D. A comparison of client outcomes from two acute care neurological services using self care data from the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs-OT). Br J Occup Ther. 2005;68(10):477– 482. 23. Unsworth CA, Bearup A, Rickard K. Benchmarking OT rehabilitation outcomes for clients with upper limb dysfunction following stroke using the AusTOMs – OT Upper Limb scale. Submitted for publication. 24. Bullivant JRN. Benchmarking for Continuous Improvement in the Public Sector. Essex, UK: Longman Information & Reference; 1994. 25. Enderby P, Hughes A, John A, Petheram B. Using benchmarking data for assessing performance in occupational therapy. Clin Governance: Int J. 2003;8(4):290–295. 26. Harwood RH, Rogers A, Dickinson E, Ebrahim S. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care. 1994;3:11–16. 27. Uniform Data System for Medical Rehabilitation. The Guide for the Uniform Data Set for Medical

Rehabilitation (Including the FIM Instrument), Version 5.1. Buffalo: UDSMR; 1997. 28. Duncan PW, Lai S, van Culin V, Huang L, Clausen D, Wallace D. Development of a comprehensive assessment toolbox for stroke. Clin Geriatr Med. 1999;15(4):885–915. 29. Duncan PW, Wallace D, Lai SM, Johnson D, Embretson S, Laster LJ. The Stroke Impact Scale Version 2.0: evaluation of reliability, validity, and sensitivity to change. Stroke. 1999;30(10):2131– 2140. 30. Ware JE, Sherbourne CD. The MOS 36-Item Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Med Care. 1992;30(6):473–483. 31. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: development and final revision of a health status measure. Med Care. 1981;19(8):787–805. 32. Fillenbaum GG, Smyer MA. The development, validity, and reliability of the OARS Multidimensional Functional Assessment Questionnaire. J Gerontol. 1981;36(4):426–433. 33. Hunt SM, McKenna SP, McEwan J, Williams J, Papp E. The Nottingham Health Profile: subjective health status and medical consultations. Soc Sci Med. 1981;15A:221–229. 34. Law M, Polatajko H, Pollock N, McColl MA, Carswell A, Baptiste S. Pilot testing of the COPM: clinical and measurement issues. Can J Occup Ther. 1994;61(4):191–197. 35. Chino N, Sonoda S, Domen K, et al. Stroke Impairment Assessment Set (SIAS): a new evaluation instrument for stroke patients. Jpn J Rehabil Med. 1994;31:119–125. 36. Lynden P, Brott T, Tilley B, et al. Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke. 1994;25:2220–2226. 37. Ashworth B. Preliminary trial of carisoprodal in multiple sclerosis. Practitioner. 1964;192:540–542. 38. Bohannon RW, Smith M. Interrater relaibility of a modified Ashworth scale of spasticity. Phys Ther. 1987;67:206–207. 39. Tardieu G, Shentoub S, Delarue R. A la recherched’une technique de mesure de al spasticite. Rev Neurologie. 1954;91:143–144. 40. Demeurisse G, Demol O, Roboye E. Motor evaluation in vascular hemiplegia. Eur Neurol. 1980;19:382–389. 41. Collen FM, Wade DT, Robb GF, Bradshaw CM. The Rivermead Mobility Index: a further development of the Rivermead Motor Assessment. Int Disabil Stud. 1991;13:50–54. 42. Lincoln NB, Leadbitter D. Assessment of motor function in stroke patients. Phys Ther. 1979;65(2):48–51. 43. Gowland C, Stratford P, Ward M, et al. Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment. Stroke. 1993;24(1):58–63. 44. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH

(Disabilities of the Arm, Shoulder and Hand). Am J Ind Med. 1996;29:602–608. 45. Barreca S, Gowland C, Stratford P, et al. Development of the Chedoke Arm and Hand Activity Inventory: theoretical constructs, item generation, and selection. Top Stroke Rehabil. 2004;11(4):31–42. 46. Kwakkei G, Wagenaar RC, Twisk JWR, Lankhorst GJ, Koetsier JC. Intensity of leg and arm training after primary middle-cerebral-artery stroke: a randomised trial. Lancet. 1999;354:191–196. 47. Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index. Md State Med J. 1965;14:61– 65. 48. Sheikh K, Smith DS, Meade TW, Goldberg E, Brennan PJ, Kinsella G. Repeatability and validity of modified activities of daily living index in studies of chronic disability. Int J Rehabil Med. 1979;1:51– 58. 49. Walker MF, Lincoln NB. Factors influencing dressing performance after stroke. J Neurol Neurosurg Psychiatry. 1991;54:699–701. 50. Bonita R, Beaglehole R. Modification of Rankin Scale: recovery of motor function after stroke. Stroke. 1988;19:1497–1500. 51. Williams JH, Drinka TJK, Greenberg JR, FarrellHoltan J, Euhardy R, Schram M. Development and testing of the assessment of living skills and resources (ALSAR) in elderly community dwelling veterans. Gerontologist. 1991;31:84–91. 52. Drinka TJK, Williams JH, Schram M, Farrell JH, Euhardy R. ALSAR-Revised Format. Madison, WI: Veterans Affairs Medical Center; 1991. 53. Holbrook M, Skilbeck CE. An activities index for use with stroke patients. Age Ageing. 1983(12):166– 170. 54. Nouri FM, Lincoln NB. An extended activities of daily living scale for stroke patients. Clin Rehabil. 1987;1:301–305. 55. Fisher AG. Assessment of Motor and Process Skills. 2nd ed. Fort Collins, CO: Three Star Press; 1997. 56. American Speech Language Hearing Association. A functional outcome measure for adults. ASHA. 1995;April:40–47. 57. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state.” A practical guide for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. 58. Doyle PJ. Measuring health outcomes in stroke survivors. Arch Phys Med Rehabil. 2002;83(Suppl 2):S39–S43. 59. Whiting S, Lincoln N, Bhavnani G, Cockburn J. RPAB-Rivermead Perceptual Assessment Battery. Windsor, England: NFER-NELSON; 1985. 60. Tsuji T, Liu M, Sonoda S, Domen K, Tsujiuchi K, Chino N. Newly developed short behavior scale for use in stroke outcome research. Am J Phys Med Rehabil. 1998;77(5):376–381. 61. Wilson B, Cockburn J, Halligan PW. Behavioural Inattention Test. Bury St. Edmunds: Thames Valley Test Company; 1987. 62. Wilson B, Cockburn J, Baddely A. RBMT-The Rivermead Behavioural Memory Test. Bury St


363

Edmunds, London: Thames Valley Test Company; 1991. 63. Sarason IG, Sarason BR, Shearin EN, Pierce GR. A brief measure of social support: practical and theoretical implications. J Soc Pers Relat. 1987;4:497–510. 64. Friedland J, McColl M. Social support and psychosocial dysfunction after stroke: buffering effects in a community sample. Arch Phys Med Rehabil. 1987;68:475–480. 65. Melzack R. The McGill Pain Questionnaire: major properties and scoring methods. Pain. 1975;1(277299):277. 66. Melzack R. The McGill Pain Questionnaire: from description to measurement. Anesthesiology. 2005;103(1):199–202. 67. Scott J, Huskisson EC. Vertical or horizontal visual analogue scales. Ann Rheum Dis. 1979;38:378– 381. 68. Roland M, Morris R. The study of the natural history of back pain Part I: development of a reliable and sensitive measure of disability in low back pain. Spine. 1983;8(2):141–144. 69. Chibnall JT, Tait RC. Quality of Life Scale: a preliminary study with chronic pain patients. Psychol Health. 1990;4:283–292. 70. Spitzer W, Dobson AJ, Hall J, et al. Measuring the quality of life of cancer patients. J Chronic Dis. 1981;34:585–597. 71. Fugl-Meyer AR, Branholm I, Fugl-Meyer KS. Happiness and domain specific life satisfaction in adult northern Swedes. Clin Rehabil. 1991;5:25– 33. 72. Frisch MB, Cornell J, Villanueva M, Retzlaff PJ. Clinical validation of the Quality of Life Inventory: a measure of life satisfaction for use in treatment planning and outcome assessment. Psychol Assess. 1992;4(1):92–101. 73. Willer B, Ottenbacher KJ, Coad M-L. The Community Integration Questionnaire: a comparative examination. Am J Phys Med Rehabil. 1994;73(2):103–111. 74. Corrigan JD, Deming R. Psychometric characteristics of the Community Integration Questionnaire: replication and extension. J Head Trauma Rehabil. 1995;10(4):43–53. 75. Nissan SJ, William P, Newman MD. Factors influencing reintegration into normal living after amputation. Arch Phys Med Rehabil. 1992;73:548– 551. 76. Wood-Dauphinee SL, Opzoomer MA, Williams JL, Marchand B, Spitzer WO. Assessment of global function: the Reintegration to Normal Living Index. Arch Phys Med Rehabil. 1988;69:583–590. 77. Wade DT. Goal planning in stroke rehabilitation: How? Top Stroke Rehabil. 1999;6:16–36. 78. Drummond AER, Walker MF. A randomized controlled trial of leisure rehabilitation after stroke. Clin Rehabil. 1995;9:283–290. 79. van Campen C, Sixma H, Friele RD, Kerssens JJ, Peters L. Quality of care and patient satisfaction: a review of measuring instruments. Med Care Res Rev. 1995;52:109–133.

364


80. Meterko M, Nelson EC, Rubin HR, Batalden DM, Hays RD, Ware JE. Patient judgement of hospital quality: a report of a pilot study. Med Care. 1990;26:S1–S56. 81. Zarit SH, Todd PA, Zarit JM. Subjective burden of husbands and wives as caregivers: a longitudinal study. Gerontologist. 1986;20(6):260–266. 82. Gerritsen JC, Van Der Ende PC. The development of a caregiving burden scale. Age Ageing. 1994;23:483–491.

83. Robinson BC. Validation of a Caregiver Strain Index. J Gerontol. 1983;38:344–348. 84. Vertesi A, Darzins P, Edwards M, McEvoy E. Development of the Handicap Assessment and Resource Tool (HART). Can J Occup Ther. 2000;67:120– 127. 85. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73:519–526.