PSSRU
Measuring Outcomes in Social Care: Conceptual Development and Empirical Design Quality Measurement Framework Project PSSRU Interim report Julien Forder, Ann Netten, James Caiels, Jan Smith and Juliette Malley
PSSRU Discussion Paper 2422 October 2007 (final revision)
PERSONAL SOCIAL SERVICES RESEARCH UNIT University of Kent
London School of Economics
University of Manchester
University of Kent Cornwallis Building Canterbury Kent CT2 7NF
London School of Economics LSE Health & Social Care Cowdray House Houghton Street London WC2A 2AE Tel: 0171 955 6238
[email protected]
University of Manchester First Floor Dover Street Building Oxford Road Manchester M13 9PL Tel: 0161 275 5250
[email protected]
Tel: 01227 823963/823862
[email protected]
www.PSSRU.ac.uk
Contents Contents............................................................................................................................................ 2 1 Introduction ................................................................................................................................ 3 1.1 Why measure outcomes?.................................................................................................. 3 1.2 What to measure? ............................................................................................................. 4 1.3 Is it feasible? ..................................................................................................................... 5 1.4 Government policy ............................................................................................................ 5 1.5 Aims and objectives .......................................................................................................... 5 1.6 Methods............................................................................................................................. 7 1.7 Structure of this report....................................................................................................... 7 2 Outcomes in social care – approaches and issues ................................................................... 8 2.1 Different conceptual approaches....................................................................................... 8 2.1.1 Welfare Economics ....................................................................................................... 8 2.1.2 Extra welfarism............................................................................................................ 12 2.1.3 (Hedonic) psychology.................................................................................................. 13 2.1.4 Capabilities and functioning ........................................................................................ 14 2.1.5 Quality of life................................................................................................................ 17 2.1.6 Older people’s utility scale and capacity for benefit .................................................... 19 2.2 Key issues ....................................................................................................................... 20 2.2.1 Outcome domains ....................................................................................................... 20 2.2.2 Measurement of functioning states ............................................................................. 24 2.3 Intermediate outputs, service specific measures or process quality ............................... 25 2.3.1 Outputs of I&A services............................................................................................... 25 2.4 Evaluating the different approaches................................................................................ 31 2.4.1 Applying evaluative criteria.......................................................................................... 31 2.4.2 Which approach? ........................................................................................................ 33 2.5 Using outcomes information............................................................................................ 34 2.5.1 Well-being index.......................................................................................................... 34 2.5.2 Outcomes-based commissioning ................................................................................ 35 2.6 Some conclusions from the conceptual work .................................................................. 39 3 Empirical strategy .................................................................................................................... 41 3.1 Aims ................................................................................................................................ 41 3.2 Care Homes Project ........................................................................................................ 42 3.2.1 Background ................................................................................................................. 42 3.2.2 Empirical methodology ................................................................................................ 44 3.3 Low Level Interventions Project ...................................................................................... 50 3.3.1 Background ................................................................................................................. 50 3.3.2 Empirical methodology ................................................................................................ 55 3.4 Information and advice services...................................................................................... 59 3.5 Preference study ............................................................................................................. 59 4 Timings and outputs ................................................................................................................ 59 4.1 Timings............................................................................................................................ 59 4.2 Outputs............................................................................................................................ 60 5 Concluding points .................................................................................................................... 60 6 Annexes................................................................................................................................... 61 Annex 1. Welfare economics and resource allocation in markets............................................... 61 Annex 2. Resource allocation absence of market prices ............................................................ 62 Annex 3. Discrete choices and compensation ............................................................................ 63 Annex 4. Extra-Welfarism: health and cost-effectiveness ratios ................................................. 64 Annex 5. Services, wellbeing and capability ............................................................................... 65 Annex 6. Capabilities and functioning - Social welfare properties............................................... 68 Annex 7. Capacity for benefit ...................................................................................................... 70 Annex 8. Using capacity for benefit to improve well-being .......................................................... 73 References...................................................................................................................................... 75
2
1 Introduction The Quality Measurement Framework (QMF) programme is being funded for three years by the Treasury under Invest to Save and led by the Office of National Statistics (ONS). The aim of the QMF programme is to create entirely new mechanisms for more effective and efficient measurement and monitoring of third sector provision of public services, reducing the burden on the third sector while releasing cash through more efficient use of public funds to provide public services. The purpose is to develop methodologies for measuring and assessing the value added of the relevant public services. This work will result in a toolkit that should be useable by service commissioning authorities to assess and monitor the performance of public services delivered by third sector organisations in a way directly comparable with performance of public or private sector providers. The work builds on previous research commissioned to feed into the Atkinson review of the measurement of government outputs and productivity for the purposes of National Accounts (Netten, Forder et al. 2006). This work developed an approach which uses research findings and routinely collected information to identify the value of the outputs of services or interventions in terms of their potential to achieve (capacity for benefit) and the degree to which this is achieved (quality). Councils spend £12.75 billion of public money on adult social care services. If we also include disability related benefits that help finance user charges and NHS spending on social care, the total is greater still. We are therefore entitled to ask what value these services produce in return. In particular, there are three related reasons to measure the value of outputs. First, there is a need to reflect the contribution of social care in the National Accounts. Second, to provide accountability for this public spend in terms of ensuring good value for money for taxpayers and also in terms of achieving public policy goals such as equitable provision of social care. Third, to give commissioners and providers this information as a management tool for allocating resources to ensure good productivity and efficient use of those resources.
1.1
Why measure outcomes?
The outputs of government funded services can be measured in terms of the numbers of people in receipt, the number of hours of care provided and so on. But this is to miss the essential point that people use services because they benefit from the consequences i.e. services have value to people. A unit of output of a service will produce value for people but the amount will vary – people have different care needs, weigh the characteristics of services differently and so forth. And this value can differ markedly from the cost of providing the service. So value cannot simply be inferred from the use of a service; it needs to be measured directly. Even if we were content to assume some unknown, but constant value accrues from the use of specific services, this is still insufficient. We need to know the relative value of different types of services in order to determine how to allocate funding to them in an efficient value overall. Furthermore, weighting services according to their unit cost is not enough. Although more costly services are often valued more highly than a lower cost services, cost is not the same as value. For instance some low cost services can be more highly valued by people using them than more costly services. With public services there is no particular mechanism to ensure that price equals cost or that marginal valuation of given levels of output equals price (although even with subsidised public sector prices, people will choose not to use the service if this subsidised price exceeds their marginal valuation). Social services are valued because of the effects people experience in using them, e.g. for personal care services: being clean, being fed, feeling independent and in control, being able to have meaningful social contact and so forth. These are the functioning states of services and much more precisely reflect impact of service use. The state of cleanliness, nutrition, control etc. that
3
services affect are functioning states. 1 So the outcome of services is in improving people’s functioning state. Ultimately, it can be argued that functioning states (i.e. improved functionings) are, in turn, valued because they have fundamental psychological and other consequences e.g. they make us happy. In practice, these fundamental consequences are much harder to measure. Knowing the value or functioning states of individual services allows the range of services commissioned and provided by councils to be configured in such a way as to produce the greater improvement in functioning states for the total budget. Put another way, knowing the functioning states of different services could allow commissioners to ensure the same overall outcome improvement for people using services at lower cost, simply because services that have a better value to unit cost return can be prioritised. At present, for the care services that the public sector commissions from the private and voluntary sectors, information about quality and functioning states is patchy – there are no routine collections of evidence about functioning states for example (Commission for Social Care Inspection 2006). With regard to supply-side, the Commission for Social Care Inspection (CSCI) regulates care home providers according to a set of National Minimum Standards (NMS), which are interpreted through a template called the Key Lines of Regulatory Activity (KLORA). The KLORA interpretation, especially, does have elements that are consistent with a functioning states approach, but it was not designed from first principles on this basis. It is acknowledged by the regulator that the current NMS are framed in input and process rather than outcome terms (Commission for Social Care Inspection 2006, p38). Furthermore, although it can be adapted to work for community-based services, there is not as yet any consistent way to compare between services. Without such information, decisions about commissioning must fall back on using cost information or made according to historical patterns. As such there is no reason to expect that the current configuration of services is that which achieves best value given the funding available. This being the case, it will be possible to re-configure services in a way that leads to greater value, given funds available. This improvement in value is a direct benefit flowing from efforts to measure functioning states. But it must be set against, (a) the costs of measuring functioning states, and (b) the costs of changing commissioning to a functioning states basis. Furthermore, the burden of these ‘transaction costs’ must not fall disproportionately on voluntary and private organisations. If cashable savings can be made in a move to functioning states-based commissioning, it would not be unreasonable for providers to be compensated for some of the extra costs they incur.
1.2
What to measure?
Outcome measures need to be accurate and allow comparability between services. How then are functioning states defined? In what way do people derive a sense of value from services? There are a number of approaches to addressing these questions. The conceptual work of the project has explored the most promising with the aim of developing that approach which will be most relevant in the case of social care services. Most of the approaches that have been taken assume that people, including service users, have fundamental motives such as being happy, fulfilled, loved, wish to avoid pain and sorrow etc. Taken as a whole, these fundamental motives are summarised by the terms utility or wellbeing. Some observers argue that utility or wellbeing are directly measurable i.e. can be captured in some quantifiable way. Others disagree, arguing that whilst we act in order to improve wellbeing, it is only these actions that are measurable. For example, being fed improves wellbeing, but that latter cannot be directly measured and instead we measure the outcome, i.e. being fed. These are inferred wellbeing approaches rather than direct wellbeing approaches. Others still believe that it is not just achieving the outcome, but the capability of achieving it that matters. People are not just happier being fed, but happier because they have the power to choose to be fed.
1
We use ‘functioning’ here in a broad sense relating to many aspects of people’s lives. It is not just restricted to physical or mental ability that a person might have in relation to walking or reading.
4
1.3
Is it feasible?
Measuring functioning states is a difficult business. If the ‘wrong’ metrics are developed in the sense of incorrectly measuring true value, then we risk that functioning states-based commissioning is no better (or even worse) than the way it is currently done (and still with the transaction costs). So our task involves not only developing functioning states metrics but also testing them to see whether they are measuring value. This project will develop low burden, possibly self-completion, tools that are abridged versions of the full metrics set. These, in particular, need testing. Developing the ‘right’ metrics has two key parts. First, being able to define what we mean by ‘value’. Whose value are we talking about – public authorities, elected politicians, the general public, service users, etc? Do we include the well-being and preferences of individuals alone or do we also build in society level criteria like moral principles, equity judgements and so on? Are we concerned with value as it relates to the direct aims of the services or a more comprehensive quality of life approach? These are challenging questions, but ones for which we must have answers, one way or the other. Second, having defined what is important, we need to find an accurate and practical way of measuring its achievement.
1.4
Government policy
The emphasis on the improvement of functioning states for people using government services is clear in the policy literature. The recent White Paper, Our Health, Our Care, Our Say (2006), set out a list of top-level objectives for services and these (mainly) concern the improvement of functioning states for service users. The focus on maximising health gain or wellbeing gain rather than maximising the provision of health and social care services goes back many years. The 1996 DH priorities and planning framework, for example, outlined the purpose of the NHS as being “to secure through available resources the greatest possible improvement in the physical and mental health of the people of England”. Concerns with value-for-money and equity figure in almost all the policy guidance that has come out of government.
1.5
Aims and objectives
This project focuses on three sets of social care services: care homes (for older people and for people with learning difficulties); low-level services – services aimed at people with moderate or lower levels of need, such as day care, befriending services, domestic support, various home aids and adaptations and so forth – and information and advice services. The full set of aims of the work is as follows: 1. To define value in the context of social care services 2. To develop metrics and tools to measure by how much services deliver value 3. To test these metrics, including developing a set of evaluation criteria with which to rate the performance of the metrics 4. To determine the consistency of the functioning states (improvements in functioning state) and quality metrics with those used in the regulatory process. 5. To determine the relative weighting of dimensions of the standard metric in order to derive a single composite measure of value change 6. To develop some (illustrative) commissioning criteria using a value or functioning states approach 7. To undertake preliminary empirical testing as to the scope of any net benefit that accrues from a functioning states-based commissioning configuration of services. In the care homes project, the first four aims are emphasised. A particular challenge is developing metrics that can be used for highly dependent people, including those who are cognitively impaired and who are unable to directly express the functioning states they experience or even their functioning state, or the degree to which these functioning states are valued. In this case, observation techniques are needed, which in turn means finding a way to map observer ratings onto outcome domains. These observation techniques, in particular, will need testing to determine 5
their validity. The care homes project will draw on the capacity for benefit methodology that, in previous forms, has been used for assessment of some social care services. The care homes project is also centrally concerned with establishing the consistency and correlation between the metrics developed in this project and the quality ratings that are awarded to providers by the regulators, CSCI. The intention is to be able to translate the routine collection of quality information from CSCI into functioning states terms that can be used to generate a standard and comparable outcome metric. This standard metric can be used for strategic accounting purposes and on-going commissioning tasks. The low-level services project will emphasise the adaptation of outcome measures to accommodate a broader range of quality-of-life considerations. Older people with lower needs may appreciate how these services (potentially) impact on their well-being, especially around a sense of self-worth, social relationships and occupation, rather than in terms of how their services address their personal care needs. In the case of the low-level services project, people will generally be less dependent and therefore more able to respond directly to questionnaires and so on. An important task will be defining and identifying services falling within the low-level services classification. In this respect there is a wide range of possibilities that vary from befriending services, handyman services, meal services, and day care services to low-intensity home care. One goal will be seeing whether a standard metric can tackle the differences between these services (aims 1 and 2). Another goal will be to assess whether a summary version of the standard metric will be feasible. The project will use interviews and more detailed questionnaires to work up the full measure of functioning states. Alongside these, we will develop, where possible, short self-completion questionnaires. The intention is that these can be administered by post or online and therefore operate at a low cost (as compared to administering face-to-face interviews). But this strategy will only be successful if the summary instrumentation can capture sufficiently the difference in functioning states as reflected in the full measure. The low-level services project is also the test-bed for feeding functioning states information into the commissioning process. Following the piloting phase, the main fieldwork will aim to use a sufficiently large survey of key low-level services, particularly day care. The intention is to make a preliminary assessment of the functioning states of services in terms of their improvement in people’s functionings for a range of different people. This outcome will be measured on our standard metric. Attaching costs to the services will then give an indication of the costs of a standard unit of improvement in functioning states delivered by the service. Using this information and some sense of the overall budget available for services, we can calculate what the most costeffective level of provision of the service is for a range of different service users and also against other services. This ideal level can then be compared with the actual level of service being provided. As noted above, any difference in these configurations of services implies a performance shortfall in functioning states terms. The information and advice project also aims to see how far standard measurement tools can be adapted. There are real challenges in trying to define what the outputs of these services are in tangible ways. We might assume that better information reassures people, reducing uncertainty and so improving quality of life. But certainty about bad news might be worse than uncertainty about it. Information and advice should also improve how people make decisions, and indeed the right decisions, which lead to improved functioning states. But isolating the effect of information on these improved functioning states is difficult. The associated task is to see how these functioning states can be measured against the standard metrics. Further to the project aims that will be addressed as outlined (in particular service areas), there is an overarching aim to ascertain the relative importance or weight people give to domains of outcome. If we are to develop some baseline standard outcome metric, this will cover a number of dimensions of how services impact on people’s lives. These are likely to be fairly diverse, for example, ranging from service users being clean and nourished, to having a reasonable degree of 6
social participation (i.e. not lonely or socially isolated), service users feeling that they have control over their lives, and so on. This diversity is important – the focus of nursing home services is quite different from befriending or sitting services. But these are all social care. Where a service does well in one domain, but not so well in another when compared to alternative services, we need a way to say overall which is the better option. Ultimately, having a single, composite outcome gain metric is most useful for this purpose. The work will include a preference study in an attempt to address this aim.
1.6
Methods
In overview, the work programme is divided in to four phases. Phase 1 covers the conceptual work and empirical methodology development. Using a range of theories that have been developed from health economics through to psychology, this part of the work involves determining what functioning states are important and how they can be specified. It also synthesises key design considerations from the literature that can be used to evaluate the different approaches for the current purpose. This evaluation method aims to select the most promising approach. Once determined, this conceptual work involves choosing the appropriate techniques for practical measurement of functioning states. Phase 1 of the work is also about designing the empirical strategy and protocols so as to make robust outcome measurement. The final component is in how to empirically judge the hypotheses around the net benefits of functioning states-based commissioning. Phase 2 of the programme is about translating the concepts into workable empirical instrumentation (questionnaires etc.). Working with stakeholders and service users, feasibility and piloting exercises will be undertaken and the instrumentation developed and finalised in light of these. Phase 3 is the main fieldwork and analysis phase. Following the piloting phase, a large sample of service users and providers will be surveyed using the instrumentation and functioning states metrics. This phase is about collecting statistical data that can be used to measure average outcome gain and other relevant information such as service characteristics and details about the people using the services. Phase 4 is the preference study. The work will build on a pilot preference study (Burge, Gallo et al. 2006) and will use a range of ‘preference elicitation’ techniques (such as discrete choice and bestworst). Particular attention will be paid to phrasing and description of outcome states, and possible context/framing effects in people’s responses. The preference study will need to accommodate the wide range of possible functioning states as exampled by the diversity of services considered above.
1.7
Structure of this report
After this introduction, section 2 describes the phase 1 work so far, i.e. conceptual framework. It reviews various approaches and draws out some conclusions in the form of key hypothesis, messages and design considerations to feed into the empirical work. Section 3 describes plans regarding the empirical work – the work covering phases 2 and 3 of the project. The intention is to describe our current empirical methodology as far as it is developed for the main project components. At this stage we are not outlining the phase 4 preference study methodology, given the staging of that phase of the work. Section 4 gives some practical details about project time-frames and dependencies. Section 5 makes concluding points and goes through the next steps.
7
2 Outcomes in social care – approaches and issues 2.1
Different conceptual approaches
2.1.1 Welfare Economics Economists have built up a scientific paradigm around the concept of a person’s utility, that is, some implicit and subjective representation of a person’s state of well-being, quality of life, or state of happiness. This has tended to be related to goods and services that people use, or more generally, about the economic resources they are able to consume. Welfare economics does not regard utility as directly measurable. Instead this welfarist body of work is about making people better off without having to precisely define, measure or compare the utility of individuals (Boadway and Bruce 1984; Drummond and McGuire 2002). From this perspective utility is inferred from choices people make between (the outcome characteristics) of services or is inferred from peoples’ willingness to pay for services. The latter is the basis of national accounts valuation for marketable goods and services. In markets, prices reflect people’s willingness to pay: services that are more highly valued will command higher prices than less valued services. Where services are not marketed, willingness to pay can be asked directly, or utility can be derived by assessing people’s reaction to (discrete) choices over the potential functioning states of services. Welfare economics is about trying to determine whether people are better off with one allocation of resources compared to another. The aim has been to make this ‘better off’ determination without having to precisely define, measure or compare the utility of individuals, but rather only to require that individuals can rank different states i.e. that one situation is preferred to another (by virtue of providing higher utility). In other words, the attention of this work has been on the resource allocation issue at a society level, not measurement of functioning states per se. Two value judgements are at the heart of welfare economics. First, that society level well-being should reflect (some adding up of) individual people’s preferences/individual utility. In other words, only the utility consequences of decisions about services (in our case) are relevant. People value services in terms of the utility they create. A second – very strong – assumption is that utility at the individual level is not directly quantifiable and that we cannot directly compare the utility of one individual against another. Rather than making inter-personal comparisons, welfare economics allows that people can rank choices i.e. we can say that one choice is better or worse than another. And this is the basis of another key value judgement, the Pareto Principle, which says that if circumstance A is ranked higher than circumstance B for one person and all other people rank A at least as high as B, then A is a better outcome than B. Where this situation holds, we need only to find out if people prefer one situation to another to say which resource choice should be made. There has been no need to quantify the strength of preference. But there is a very significant limitation: changes that make some people better off whilst others worse off cannot be evaluated in this way. In the much more likely case that some people are better off and some people worse off, some strength of preference measure is needed to make progress. Again, welfare economics has not focused on how this measure is made in practice. But in theory, strength of preference – i.e. the utility consequences of services choices – can be reflected in how much money people are willing to pay or accept in compensation for some choice compared to another. Money implies the ability to ‘consume’ goods and services, which in turn produce utility. The more we value something, the more we are willing to pay for it. So theoretically (willingness to spend or receive) money or indeed some other measure of utility can give the individual’s strength of preference. But this is not enough on its own to allow interpersonal comparisons. How does one person’s willingness to spend/receive compare to another person’s? Or more generally, is one person’s utility more or less important than another person’s? 8
There has been an important attempt to avoid having to explicitly place importance weights on people’s utility (or spending power): the (Hicks and Kaldor) hypothetical compensation principle (Boadway and Bruce 1984). This principle or test proceeds on the basis that even if two circumstances are not Pareto comparable, it still might be that case that the winners could hypothetically compensate the losers in such a way that the winners are better off and the losers are compensated to the original level. The levels of compensation of the losers will be according to their own preferences. If it turned out that after paying this compensation, the winners – relative to their individual preferences – would still feel better off with the choice in question, then it ought to happen. Indeed, no compensation would actually be paid, and so this principle cannot tell use how actual well-being changes. But it does guide us as to whether potential improvements are worthwhile. In practice, this judgement is made by adding up these compensation/willingness-to-pay amounts, both positive and negative. If the total is greater than zero, the choice should happen. However, there are problems with this approach. Some situations still cannot be compared using this principle. But most importantly, we need to know how the rate at which people convert money to utility compares between individuals (or we need to assume that this conversion rate is constant between people). Otherwise the value of a potential monetary compensation could differ between people. Another approach is to use a social welfare function (SWF). Here individual people’s utility or utility proxies are added up with ‘distributional weights’ on each person’s utility. In theory, functions can be used to combine rankings of utility as well as fully quantified utility levels, offering the possibility that a SWF approach together with a Pareto Principle might give a workable social ordering (i.e. an allocation of goods and services between individuals) in a less restrictive way than noted above. But Arrow (1951) showed this to be impossible for a general class of SWFs. What this result implies is that the measurement of utilities/functioning states is unavoidable in being able to comment on value of different allocations of resources. Even where inter-personal comparisons are conceded, the welfarist approach still makes these comparisons in terms of (personal) utility information. Functioning states are important only in as far as they affect individuals, and that this individual effect is summed up. Others argue that societal level well-being is more than just the sum of individual utility (Sen 1999). Instead, societal level principles should also count and apply at the individual level. Non-discrimination and nonexploitation are good examples. 2.1.1.1 A need to measure utility or well-being directly? If we are prepared to accept the core welfarist assumptions above and also that markets work sufficiently well, then changes in prices and quantities from one circumstance to another is enough to tell us whether the change in circumstance improves overall well-being. This is the basis on which National Accounts are calculated for marketable goods and services. We need to introduce some concepts to make progress. First, we are focused on improving the total amount of well-being in society – which is denoted by the term W. This is made up of some sum of individual people’s utility – the latter is denoted by u. In turn, an individual’s utility is dependent on what services x they use. The theory shows that if we accept that income is appropriately allocated in society (but this is clearly questionable), then, for a (small) change in the level of services, x, people receive, the change in welfare in markets approximates to:
⎛ n ΔW M = K ⎜⎜ pi xi1 − ⎝ i =1
∑
(1)
where ΔW M K
n
∑p x i
i =1
0⎞ ⎟ i ⎟
⎠
is the change in total well-being in the case of markets (as indicated by the M superscript) is a scaling term, which we can set to 1 (i.e. it drops out). 9
n
∑p x
1 i
is the multiplication of prices and outputs in time 1 (after the change) for each service i summed up for all services in question
∑p x
0 i
is the multiplication of prices and outputs in time 0 (before the change) for each service i summed up for all services
i
i =1 n
i
i =1
Annex 1 shows how (1) is determined. The term in the brackets is the Laspeyres index in level form ( l ). In other words, increases in weighted outputs from time 0 to time 1 imply increases in welfare. Moreover, if K is fixed through time (or held fixed through time) then an index of welfare ΔWt +1→t + 2 Kl t +1 l t +1 = = . changes can be created i.e. ΔWt →t +1 Kl t lt 2.1.1.2 Absence of market prices In well functioning markets, prices adjust (through demand and supply) so that they equal people’s valuation of the last unit of service purchased (their marginal valuation) or essentially how much the utility they get from that service when expressed in equivalent money terms. So in the above equation, prices times outputs gives total valuation. Summing this up gives total well-being. In the absence of market prices, or in a situation where prices are assumed to be distorted (as in social care markets), an estimate of the impact of services on utility is required. As noted above, because utility cannot be directly measured, this estimate must be inferred indirectly. We discuss below how this can be done, but it is sufficient to say at this point that estimates can be made. For a given service, suppose such an estimate is υi, where the subscript refers to service i, allowing for valuations to be different for different services. By making some assumptions about how money can be ‘converted’ into utility, then it is sufficient for us to use this estimate υi of the (marginal) valuation, to work out how total well-being will change between two time points: (2)
(⎛ n ΔW N = K ⎜⎜ υi xi1 − ⎝ i =1
∑
n
∑υ x i
i =1
0⎞ ⎟ i ⎟
⎠
This equation is equivalent to the one above, except that prices are now replaced with the estimate of the average valuation of people using service i – see Annex 2. Here ΔW N is the change in total well-being for non-marketed services (denoted by the superscript N). For non-marketed services therefore, measurement of the (marginal) valuation of those services is required. In social care, unlike health, there is both a ‘quasi-market’ in government funded care and a private market. In quasi-markets, service users are either subsidised in buying care from providers (and are often limited in what they are able to purchase) or care is purchased on their behalf by commissioners (Le Grand and Bartlett 1993). In neither case is the full price observed, meaning that we cannot easily infer the value of public output from the market price. The private market suffers significant information problems, distorting prices, and anyway constitutes less that 30% of the total spend for older people (Wanless 2006). For this reason it is most appropriate to treat social care interventions in the ‘non-marketed’ category, especially publicly funded care. 2.1.1.3 Cost-weighted index? In perfectly competitive markets that appear in economics textbooks, prices are equal to (marginal) costs, ci. This occurs because competition between providers drives the price down to cost levels (plus a reasonable slice of profits). Furthermore, through entry and exit into the market the total amount of output provided just equals the total amount people wish to buy at prevailing prices. Firms with unsold inventories will cut back or go bust. If supply is too low, prices generally rise attracting new supply which subsequently returns prices to cost levels.
10
These market forces act to ensure that prices are equal to costs. Consequently, the change in n ⎛ n ⎞ welfare can be written ΔW M = K ⎜⎜ ci xi1 − ci xi0 ⎟⎟ i.e. using a cost-weighted index of activity. But i =1 ⎝ i =1 ⎠ we need to be clear that this equivalence of prices and costs only occurs in markets, and indeed, well-functioning markets (by the action of market forces). In the absence of (perfectly competitive) markets, there is no reason to expect that the quantities and mix of services provided are such that the equivalent market price would equal marginal cost. Figure 1 illustrates the point. We aim to establish the (marginal) valuation of services. Suppose non-market output was at x3 in the figure, then an approximation using non-market marginal costs – which differ from market marginal costs because the production and organisation are different – would give a valuation of υ3 when the actual valuation at this output is υ2. Even if market marginal costs were used at this output, the valuation they would imply still differs from υ2 which is the right value. Now suppose that the market output of x1 was being provided. Then the right valuation is υ1. Again, though, using non-market marginal costs is misleading. Only when market marginal costs are measured at market output x1 will they equal the true valuation of the service (and not when they are measured at other levels of output). In other words, most of the time a non-market cost weighted activity is not a good index for measuring welfare change. The argument is the basis of the rejection of cost-weighted indices in the Atkinson Review (Atkinson 2005, see especially para 4.19).
∑
∑
Figure 1. Valuation of services
Marginal costs (non-market) υ3
Marginal costs (market)
υ1 υ2 Marginal valuation x1
x3
As outlined in the introduction, this analysis shows why we need some direct estimate of the (incremental) value that people place on public services if we are to put together an index of wellbeing change for publicly funded services (for National Accounts purposes for instance). This requirement also holds if we want to compare different configurations of services to see which gives the greatest well-being improvement – see Annex 3. 2.1.1.4 Key points This brief overview of welfare economics highlights the following: • For non-marketed services, measurement of the (marginal) valuation of those services is required. In social care, there is a both a ‘quasi-market’ in government funded care and a private market. In quasi-markets, service users are either subsidised in buying care from providers (and are often limited in what they are able to purchase) or care is purchased on their behalf by commissioners (Le Grand and Bartlett 1993). In neither case is the full price observed. In private markets where individual’s purchase their own care from their own resources, prices might be a better reflection of true value. However, the private market suffers significant information problems, distorting prices, and anyway constitutes less than 30% of the total spend for older people (Wanless 2006). For this reason it is most appropriate to treat social care interventions in the ‘non-marketed’ category, especially publicly funded care. • We should assess the aggregate utility impact of services across all people affected. 11
• •
We need to decide whether or not to concentrate just on utility or also consider non-utility information In a welfarist approach a compensating variations (i.e. willingness to pay) approach is consistent with the theory, but is this the right approach in practice? These measurement problems are considered below.
2.1.2 Extra welfarism An alternative approach sidesteps some of the problems around the definition and measurement of individual utility by instead having some mandated ‘decision-maker’ use their authority to decide what outcomes are important. This is described as an extra-welfarist (EW) approach as it differs from the welfarist approach which does not included this third-party role (Dolan and Edlin 2002; Birch and Donaldson 2003; Hansen, Hougaard et al. 2004 ; Culyer 2006). For example, a government could decide that the aim of the health service is to improve population health. So although health services have effects that go beyond people’s health, the outcome of health care is only assessed in these terms. In health care, a commonly used outcome measure is the qualityadjusted life year (QALY), which measures not only the extra life expectancy associated with a health care intervention but also the quality of those extra life years. Quality in this case can include freedom from pain and depression, mobility, social functioning and so on. In social care, a similar approach has been used to develop a standard outcome measure (Netten, Forder et al. 2006).
As with the welfarist approach, utility or well-being is not directly measured. Instead the consequences of service use are inferred from those functioning states experienced by people that are deemed relevant by the decision-maker. For example, in health care, the standard approach is to assess changes in health status as measured by the QALY. This is the methodology used by the National Institute of Health and Clinical Excellence (NICE). The Wanless Review of social care used activity of daily living adjusted year (ADLAY) as the target indicator. This measure is analogous with the QALY with an emphasis on the quality of a person’s life in using social care services. So outcomes are measured as the degree to which social care services improve people’s functioning during the course of a year, such being fed, clean, appropriately dressed, not socially excluded, feeling in control of one’s life and so forth. With top-level well-being defined by the decision-maker, the EW approach, like welfarism, is about trying to make service decisions that maximise well-being for the resources available: balancing well-being gain against cost. In the EW case, however, the well-being gain is measured in service specific functioning states terms (such as the QALY). When making comparisons of a discrete set of services, whose functioning states are measured with the same metric, this is no limitation. But this is a relatively limited use of outcome information. Potential gains from functioning states-based commissioning will also come from assessing whether a new service should go ahead in the context of current spending, or whether current services should be provided more or less intensely. A common way to circumvent these apparent limitations is to use these metrics to generate costeffectiveness ratios: i.e. the change in well-being over the change in service cost. 2 In fact, the costeffectiveness ratio can be expressed as the cost of achieving a standard unit of well-being gain e.g. £20,000 per QALY. The next step is to compare this new option to some threshold level of cost-effectiveness, where the threshold is an indicative benchmark of the value of spending the money elsewhere. For example, we might assume that the average cost-effectiveness of public spending was £30,000 per QALY when expressed in these terms. If a new service has a cost per QALY of less than this threshold amount, it ought to be funded (with money coming from a reduction in the provision of less cost-effective services). There are three main issues to tackle, with Annex 4 giving the details. First, the impact of services on individual people’s well-being (e.g. health states) using the chosen metrics needs to be measured. Second, weights need to be determined so that an individual’s well-being gain can be
2
Here the change in well-being is measured relative to the best-valued alternative use of the budget
12
aggregated to some total. Often, (implicitly) equal weight is assumed for each individual. Third, a value for the benchmark cost-effectiveness threshold needs to be decided. Overall, this analysis shows that with an EW approach, the key measurement task is wellbeing or health status change. However, as described, a number of conditions must apply for this approach to be useful. Some are quite stringent. In particular, inter-personal comparisons of wellbeing are required, with wellbeing effects aggregated over people. Are wellbeing improvements for low need people equally valued as the same change for people in high need? Resolving these issues is arguably little different from inter-personal comparisons of utility (in a SWF). With a decision-maker who makes an (arbitrary) determination of what functioning states should matter, problems of comprehensiveness are avoided (but not solved). There are numerous possible definitions of well-being or health; some are more inclusive than others. For example, the impact of health care services can potentially affect a wide range of health and health-related aspects of people’s lives. The more comprehensive the definition used, the more difficult are the measurement and aggregation tasks. Extra-welfarist approaches concentrate measurement on particular characteristics of a person e.g. their health, rather than aiming to comprehensively measure utility or well-being. Preferences of individuals are important, but are not the only consideration. Non-utility information is also relevant (e.g. concerning liberty principles or about opportunities and advantages – see Sen (1979; 1999)), as might be ‘process’ considerations. 2.1.3 (Hedonic) psychology The hedonic psychology literature looks at the questions of human welfare and happiness and the role that welfare or wellbeing can and should play in public policy (Kahneman, Wakker et al. 1997; Kahneman 2000; Kelman 2004; Kelman 2005). The hedonic psychology literature sees well-being or utility as directly measurable. People can be asked directly how happy or good they feel and this information is both meaningful and useful for public policy. The hedonics view is that inference from choices (the welfarist approach) side-steps, but not solves a key problem. That is, people’s choices (based on their preferences) systematically mis-predict the utility they would experience following their choice. People’s choices are biased by framing (the way choices are presented to them), by the difficulties people have in envisioning their situation after significant choices, by the problem that people under-anticipate how much they acclimatise to their new situation, and by the emotional state (hot or cold) they are in when making decisions. The only way to ‘correct’ these problems is by adjusting the utility people would receive from choices they make, and this requires a direct measure of utility.
Early attempts to measure wellbeing/utility directly involved people being asked to rate their ‘global’ happiness on a scale from very happy to very unhappy. But these approaches also suffer from many of the above difficulties. To assess global happiness people need to recall past events and to evaluate them relative to other events. Using this information they need to form an overall impression – this is subjective happiness because it derives from personal evaluation. These connected processes give ample opportunity for error. Instead, the ‘new’ hedonics approach concentrates on people’s momentary or instant utility. It looks at people’s spontaneous approach/avoid, continue/desist and good/bad reactions at various moments in time as they use services. These momentary experiences are summed up using an objective rule to give a total utility following the use of services. This is objective happiness because although people are evaluating momentary experiences personally, total utility is derived from an objective rule as to how these instant utilities are summed up. The argued merits of this experienced utility approach are: that context effects are more ‘averaged out’ and, more importantly, people’s own (selective) recall and interpretation of events required for global measures is avoided. Nonetheless, some suggest that summing over a (momentary) single dimension measure is too reductionist (Kelman 2004).
13
2.1.3.1 Adaptation The psychology literature has developed the important and highly relevant concept of adaptation. The argument goes that people adapt in the way they experience functionings in their life according to the circumstances in which they find themselves. When a situation worsens people are less happy initially because their experience of functionings has not yet changed, but these functionings have worsened. After time however, their experiences of functionings adapt and they find more pleasure than before from the given achievement of functionings. So whilst functionings are poorer, people gain more pleasure from this leaner set of functionings. We should note that, other things constant, subjective and objective formulations of happiness would not be different in this case. In its extreme form this argument undermines attempts by policy to try to improve people’s functionings or functioning states because even if this were successful people’s happiness (with those functioning states) will soon return towards the level it was before the improvement. Whilst there is some evidence of adaptation, and the importance of happiness as a relative but not absolute concept, the extreme form of adaptation is unlikely to be the case in practice. So improvements in services, for example, may lead to an improvement in utility at a lesser rate than otherwise expected, but some improvement will persist. Furthermore, if people’s expectations can be tempered alongside the improvement in functioning states, then utility should be improved. An alternative view to the adaptation or ‘hedonic treadmill’ hypothesis as it is sometimes called, is the idea of a ‘satisfaction’ treadmill (Kahneman, Diener et al. 1999). Instead of people’s adaptation level changing, in this case the emphasis is on changes to their aspiration level, that is, their satisfaction with the happiness they achieve. So, as a person’s situation improves and their experienced functionings improve, their actual utility increases (discounting any hedonic treadmill effect), but after some time, their aspirations regarding this experience also increases. Because their aspirations become re-aligned with their experiences, satisfaction falls back to original levels. People expect to be better off as their circumstances improve, and would be unhappy if their experiences were not as least as good as their expectations. So in this case, a person’s subjective happiness is not persistently improved, although their objective happiness is improved. The implication for policy is that although people will never be fully satisfied with improvements in services, they can be made (objectively) happier. As Kahneman (1999, p15) contends: “… the goal of policy should be to increase measures of objective well-being, not measures of satisfaction or subjective happiness”. Although this approach measures well-being directly, albeit in an objective utility sense, the broader message is a reinforcement of inferred well-being approaches because they too are objective. And where attempts are made to measure satisfaction directly, allowance is made for treadmill effects. More generally, this analysis illustrates the complicated relationship between a person’s experienced functionings (such as being clean and fed) and the underlying happiness, utility or well-being they derive from this. 2.1.4 Capabilities and functioning The capabilities and functioning approach developed by Amartya Sen has people’s well-being and advantage depending on a range of ‘functionings’ that people can potentially achieve, and which in turn are affected by the services and material situation that people experience (Sen 1985; Nussbaum and Sen 1993; Sen 1999; Ruta, Camfield et al. 2007). Examples of social care relevant functionings include being fed, being clean and dressed, having meaningful social relationships and so forth. Sen’s work argues that a (welfarist) focus on individual utility can be too limiting in social choice problems (even allowing interpersonal utility comparison and cardinal utility – see in particular Sen 1979). Non-utility information e.g. principles transcending individual utility are relevant. Sen proposes judging individual advantage in terms of the respective capabilities, which the person has to live the way he or she has reason to value (Sen 1999, p. 358). Furthermore, Sen is clear about the context specificity of people’s utility. A person may be happy with their lot in the world. However, this does not stop them from realising that a different, but unobtainable, life might
14
be better. Utility in the happiness sense is therefore not the same as a person’s valuation (the latter being a reflective concept for individuals). Sen (1985) outlines the key components of the approach. The core features of the model are in the way that people choose their functionings, based in part on the services that are available to them, and the utility implications of this choice. People choose functionings within certain limits. They derive happiness or utility from these functionings and the amount of utility gained is given by a ‘happiness function’ h(.): (3)
u j = h j (b j )
Where bj is the (vector of) functionings that a person experiences. Utility is not the same as valuation. The latter is a reflective concept for individuals, one more closely akin to the idea of an objective ranking of functioning vectors. Valuation is: (4)
v j = v j (b j )
The choice of functionings will reflect the happiness a person experiences from them and a set of constraining factors that limit and influence the person’s ability to achieve them. So if the use of ‘commodities’ – or in our case services – is denoted by x then the relationship between achieved functionings and service use depends on a person’s choice of a ‘personal utilisation’ function by f(.): (5)
b j = f j (x j )
The choice of a personal utilisation function is a choice about how to use a package of services to generate functionings. The nature of the utilisation function depends on a range of user characteristics and circumstances (e.g. whether a person is physically or cognitively impaired). This choice is functionings will be determined by the utility function. For a given service package x, a person has a choice about which pattern of functionings they will choose from those patterns possible with the available service package. A service capability set is the set of all possible functionings achievable from the service package used by person j. 2.1.4.1 Capability Sen argues that it is the capability to achieve functionings that should be the basis of evaluation. This argument can be illustrated by way of an example. Suppose a person has a capability ‘set’ of Q i.e. a grouping of all the possible patterns of functionings available to that person. The highest valued pattern of functions in this set is b*. Now consider that for some reason the person’s capability set becomes far more constrained so that only the pattern b* is available. In terms of valuation, the value of achieved functionings (b* in both cases) – the person’s well-being in a narrow sense – is unchanged. But it is not easy to claim that this person’s ‘freedom’ is unchanged. In an important sense the person can do less than they could before, even though the best they can do is unchanged (Sen 1985, p. 15). In this way, Sen’s approach differs from the standard evaluation approaches above which measure the consequences of use of services given an underlying set of conditions. For short-term health care treatments putting aside the freedom aspect of a person’s well-being may not be too serious. But for support in relation to long term conditions this under-emphasis is clearly difficult to sustain in any rounded evaluation. What is less clear, however, is exactly how a capability set can be valued (rather than a functioning vector). One way around this difficulty is to include a choice/freedom functioning in the capability set. Suppose again that including this function, the best pattern is b*. Then if the capability set is restricted, as in the above example, the original b* cannot be obtained as the choice/freedom 15
functioning is limited by definition. With the restricted set only bR* can be obtained where v (b * ) > v b R * . This solution is detailed in Annex 5.
( )
In terms of designing outcome measure for long-term care, this analysis underlines the importance of accounting for some form of choice or capability. Including a choice functioning is one approach. It has limitations – such as the danger of losing interactions between each (non-choice) functioning and the person’s capability to achieve it. But it also has advantages. This approach is easier to use empirically and it also allows people to explicitly trade off choice against non-choice functionings. The latter also helps with the issue of knowing how to measure (the extent) of ‘choice’ as a functioning 3 : freedom can be preference-weighted relative to other functionings. The degree of choice people have will be mediated, in part, by the range of personal utilisation functions f(.), that are available to them. The condition that potential service users are suffering will limit their choices in this way. Therefore, people with different care needs will experience the effects of services differently, for example. This is a technical relationship. But a person’s preferences regarding (available) functioning vectors will also be influenced by their characteristics like their dependency and wealth, and by their living circumstances, some of which will be affected by the service, and some by the person’s environment, physically and socially. In particular, these characteristics and circumstances affect the happiness function h(.). We can write (3) to include a parameter θ that mediates the relationship between functionings b and utility u, that is, u j = h j (b j , θ ) . This parameter can change with given circumstances (like disability) but also chosen services. So services impact on utility by both affecting the choices of functionings that are available, and also via θ, the happiness people derive from those functionings. For example, there is an argument that some care homes have environments that lead residents to adapt – place less weight on – their preferences about dignity and autonomy. 2.1.4.2 Adaptation The distinction between utility (in the happiness sense) and a person’s valuation of their functionings resonates with the ideas of subjective and objective happiness in the (hedonic) psychology literature (Kahneman, Diener et al. 1999; Stone, Shiffman et al. 1999; Kahneman 2000; Kelman 2004; Kelman 2005). Utility can be subject to adaptation effects, that is, causes the happiness function h(.) to return utility for each level of functioning that is dependent on a person’s context. In line with the experienced utility literature, we would not expect that the parameter θ influences the function v. Or rather, the valuation function v is based on an ‘objective’ assessment of utility consequences of a given level of functioning, one where θ is held constant. However, in any given context, people will choose functionings that maximise their utility or happiness, not the reflective valuation of those functionings. Indeed, there is no reason to expect the choice of functionings based on utility to be equal to the choice based on the valuation function. For purposes of evaluating alternative service packages therefore, which should be used: value or utility? In other words, to allow for adaptation effects (θ) or not? In a social care context there are two aspects to this question. First, we can consider how a person adapts to their condition in the absence of services. Second, in using services, how those services might lead to further adaptation effects by changing the service user’s environment and aspirations about their lives. For example, services might have differential effects on a person’s expectations about what they can achieve and therefore their happiness given actual functioning with those services. Sen is clear, writing in the context of economic deprivation, that “there may be good ethical grounds for not concentrating too much on mental-state comparisons – whether of pleasures or of desires. Utilities may sometimes be very malleable in response to persistent deprivation” (Sen 1999, p. 358). People can be successful in adjusting to their circumstances but this would not make their deprivation go away. In other words, even if a person has learned to find happiness in the current (deprived) situation, there is a moral argument for doing more.
3
As noted by Robert Sugden (Economic Journal, 1986, vol 96 (383), p 821).
16
A frail elderly person who expects their life to slow down in older age or is reconciled to it, and so is relatively happy in that context, can still reflect on and lament lost ability and opportunity compared to a time before they were frail. Even if a person who became disabled had experienced utility levels that was no worse than before their condition developed – and fully understood this – would that person really be indifferent from being disabled or not? And even if they were indifferent, an external observer may see this type of adaptation – i.e. that the person’s has diminished expectations compared with before – as a bad thing, a negative non-utility consideration. A decision-maker might weigh both the person’s utility (positively) and also any deviation in a person’s circumstances – that leads to adaptation – when making decision’s about resources. The second consideration is that services also have an adaptation effect (as well as a direct effect on improving people’s functioning). Consider two care services. One (service 1A) provides rehabilitation and confidence building in order to help people function on their own. The other (service 1B) has staff members that do most things for people, but create dependency and suppresses people’s aspirations so that they take more happiness from an equivalent improvement in functioning. Suppose they both result in the same experienced utility post hoc i.e. where u j b1j A * x j1A , θ1A = u j b1j B * x j1B , θ1B with b 0j * < b1j B * < b1j A * and θ1A < θ1B . And they both have the
(
( ) )
(
( ) )
(
( )) (
( ))
same cost. In this case, however, v b1j A * x j1A > v b1j B * x j1B . In this example, one which is particularly pertinent to social care services because these services impact persistently on people’s lives, the adaptation with service 1B appears to imply a diminishing of the person using the service. Even if the person themselves cannot reflect on this effect, society more broadly can. In this case, as the effect comes from services, it might be more difficult to sustain that adaptation through diminished expectations is not worse than the in above case of people adapted to conditions (absent services). For further details, see Annex 6. An implication for the evaluation of social care services is whether we should use preferences of service users i.e. those people that have experienced any adaptation or the preferences of the general public. This will also have a bearing on whether we use a decision utility elicitation approaches or an experienced utility approach (e.g. a moment-by-moment technique - Kahneman 2000). Only using experienced utility measures is difficult to sustain if adaptation is seen to have any negative connotations beyond personal utility changes. Only using decision utility based on general population preferences is also difficult to sustain because it might bias resource allocations between conditions and services where adaptation occurred to a greater or lesser degree. A potential resolution would be to see adaptation as an externality and develop empirical techniques to try to measure its significance in the general public. 2.1.5 Quality of life The quality of life literature has a significant contribution to play in the assessment of the value of care services in a number of ways. Primarily, it provides a framework for identifying and understanding the multiple facets of people’s lives that they value. In the quality of life context, Albrecht (1993) defines quality as a measure of the extent to which a thing or experience meets a need, solves a problem, or adds value for someone (Terrill 1996, p34). Quality of life (QoL) resonates closely with the concepts of well-being developed above. Consequently, it can help to point to which functioning states or functionings are important to people. The QoL research also helps our understanding of how to measure functioning states as aligned with quality of life domains.
QoL is a complex concept due to its multiple perspectives and dimensions, and to the fact that it can be operationalised in many different ways (Schalock 1996). Furthermore, QoL is notoriously difficult to define as it encompasses many dimensions and can be viewed from a range of perspectives (Reed 2007). As Schalock (1996) points out, there are over 100 definitions and models of quality of life (Cummins 1995). This rapid increase reflects both the interest in the concept of QoL and the lack of consensus regarding its meaning. QoL has become one of the most important manifestations of quality of care (Maes, Geeraert et al. 2000). However, QoL is often confused with quality of care, where the latter refers to the way in 17
which care is delivered and the standards that it meets. Rather, improved quality of life is a consequence of good quality care. For example, care that is timely and reliable is likely to lead to improved functioning states for care recipients, such as improvements of feeling in control, being safe, being nourished, being clean (unsoiled) and so on. Timeliness and reliability are attributes of quality of care, but in most cases care is not valued for those attributes alone, but rather the consequences in improving a person’s quality of life or well-being. Nonetheless, separating QoL from quality of care is a difficult process, especially as the two are interconnected. As Reed notes, if care is of a high standard it can support and promote QoL. But, QoL can be independent of quality of care. For example, QoL may be high while quality of care is low: that is, people may feel fulfilled and happy even if the care they get is poor or non existent. Alternatively, people may have a high quality of care, in that it meets a number of standards, but be experiencing low quality of life (Reed 2007). 2.1.5.1 Quality of Life Domains The term ‘quality of life domains’ refers to the set of factors composing personal well-being. Most QoL investigators suggest that the number of domains is perhaps less important than the recognition that any proposed QoL model must recognise the need for a multi-element framework, the realisation that people know what is important to them, and that the essential characteristic of any set of domains is that they represent in aggregate the complete QoL construct. Thus, ‘quality of life domains’ should be thought of as the set of elements to which a variable is limited, or the range over which the concept of QoL extends. Analysis of the literature on individual-referenced QoL domains carried out by Schalock (2004) found considerable agreement regarding QoL domains. The 16 published studies analysed yielded a total of 125 indicators. The vast majority (74.4%) of these indicators related to eight core QoL domains: • • • • • • • •
Interpersonal relations; Social inclusion; Personal development; Physical wellbeing; Self-determination; Material wellbeing; Emotional wellbeing; Rights.
Despite the efforts in conceptualising QoL and deciding what domains or functionings are important to people, there are still gaps in our knowledge, particularly with different client groups such as people with learning disabilities and older adults. Public policy and rehabilitation organisations are struggling to reformulate themselves within a QoL paradigm that reflects quality revolution. Policy developers and program administrators need the most current thinking about the concept of quality of life and its measurement in order to improve services and promote rational public policies (Schalock 1996). 2.1.5.2 Implications The QoL research gives us a comprehensive map of the ways in which services could potentially impact on people’s lives. It allows us to assess the impact of services from a person’s whole life perspective. Furthermore, QoL work offers indicators of quality of life domains that can be more easily measured in a practical sense. There are some limitations. First, there is a sense in which some domains of quality of life appear to be primarily instrumental rather than final functioning states. For example, most people value economic well-being because it facilitates achieving other quality of life functioning states. The risk is in double counting the effect i.e. of both the instrumental and the final effect. Second, there is
18
little emphasis on considering the relative importance or different weighting of domains (with most global scores being simple equal-weight sums of domains). 2.1.6 Older people’s utility scale and capacity for benefit Whilst there is considerable work focused on functioning states for people using social care services, only a small part of this research has attempted quantitative, preference-based measurement of functioning states. The latter includes: work on the Older People’s Utility Scale (OPUS); and a subsequent development that can be labelled the capacity for benefit (CfB) approach (Netten, Ryan et al. 2002; Burge, Gallo et al. 2006; Netten, Forder et al. 2006; Ryan, Netten et al. 2006).
The CfB approach is an inferred well-being approach. The focus of work has been to develop an outcome measure for people receiving social care using a set of outcome domains (functionings relevant to service use), and to determine the relative importance of these domains. Arguably this work has welfarist features (being preference based). On the other hand, because it is focused on those functioning states deemed important to social services authorities (and policy makers) it has extra-welfarist characteristics. Strictly speaking, most of the attention to date is on definition and measurement of functioning states, rather than resource allocation, making a categorisation quite difficult (but see Netten and Forder 2007). The recent broadening of the objectives of services outlined in policy (e.g. the 2006 White Paper, Our health, Our Care, Our Say) to be more consistent with well-being and quality of life implies a much greater overlap between what individuals want (as expressed in their preferences) and the goals of the service (see 2.2.1.1 below). This is very much a part of the individualisation agenda coming from Government (Department of Health 2006). The capacity for benefit approach aims to attribute well-being changes at the service or intervention level rather than the individual level. The capacity for benefit measure represents the potential of the service or intervention to deliver well-being according to the domains of outcome that the service affects, the degree to which users are reliant or dependent on that service (i.e. compared to their functionings state without the service) and the quality of the service (Burge, Gallo et al. 2006; Netten, Forder et al. 2006; Netten and Forder 2007). In this way, the well-being conferred by a service i is: (6)
(
)
w i = φi − φ i q i x i
as derived in Annex 7.
(
)
The term φi − φ i is the capacity for benefit of (a unit of) service i and it is the potential functionings state (the preference weighted sum of each functioning domain) of people using service i less the average functionings state of those same people in the absence of that service. This is the marginal effect of services on people’s functionings. Capacity for benefit is weighted by the outcome-quality, qi, of the service in that only top quality services can deliver all potential improvements. Actual improvements may be less than potential levels. 4 In this approach the well-being improvement offered by the service is the average of individual well-being improvements of people using the service. This assumes that where a person uses more than one service they have independent effects. With these assumptions, to calculate a change in well-being we need information about the (average) potential level of outcome in each domain for the service in question and also the (average) level of outcome in each domain absent 4
(
)
As shown in the annex, φi − φ i q i =
1 xi
xi
K
∑∑ α
k b jj i i q i
j i =1 k =1
−
1 xi
xi
K
∑∑ α j i =1 k =1
k b kj i q i
, being the potential outcome,
b , of person j over k functioning domains less the outcome without service i, which is b . 19
from the service. We need information about preferences α in order to weight together the total from all domains. And we need to find a way to create a (functioning states) quality rating qi. But, the starting point is to be clear about what domains of outcome are to be included and how we define and distinguish between levels of achievement of functioning states. The work on capacity for benefit (CfB) has used nine domains of outcome and four levels of need (no needs, all needs met, low needs, high needs) within these domains. The preference studies that have been conducted aim to determine a preference weight for each domain and its levels. The domains are: • • • • • • • • •
Personal cleanliness and comfort: being clean, comfortable, presentable and in bed or up at appropriate times. Social participation and involvement: being content with emotional support, social contact and community participation. Control over daily life: able to choose what to do and when to do it. Meals and nutrition: having nutritious, varied meals at regular, timely intervals. Safety: feeling safe and secure. Accommodation cleanliness, order and accessibility: the environment is clean, comfortable and easy to get around. Employment and occupation: occupied in meaningful activities whether formal employment, unpaid work or leisure. Role support (as a carer or parent): being able to care for their dependent(s) without being overburdened. Living in own home: being able to live comfortably in own home.
The above domains cover fundamental areas of a person’s life, affecting individuals’ perception of the quality of their lives. The work thus far using capacity for benefit has measured functioning states using individual level questionnaires to service users. These surveys ask service users about their attainment of functioning states with services and also their expected levels in the absence of services. Measurement of preferences has been conducted using population surveys (mainly) and hypothetical choice-based techniques. These are discussed in more detail in section 2.2.2 below).
2.2
Key issues
The above ‘inferred well-being’ approaches argue that we should seek to measure a small set of final outcomes when assessing services. The choice of which outcome indicators should be included in the set is specific to the area of public policy we are assessing, which in this case is social care. There is a question of what measurement techniques should be used to quantify these outcome concepts empirically. 2.2.1 Outcome domains Aside from the psychological research, all the above approaches are of the ‘inferred well-being’ type i.e. well-being is ‘proxied’ by a set of functioning states indicators. With the exception of the quality of life research, the above accounts are mainly concerned with how well-being is conceived and how the concept is used in relation to the resource allocation question. In regard to social care services we then need to ask which functioning domains are likely to be important. What set of outcome indicators is sufficiently comprehensive to capture the main effects of services, but also small enough to be manageable and of practical use?
The extra-welfarist approach sees these as being questions for mandated decision-makers. To this end, we first turn to the functioning states laid out in the 2006 White Paper Our Health, Our Care, Our Say. 2.2.1.1 White Paper functioning states The White Paper outlined a changing strategy for health and social care focusing on prevention and supporting individual well-being in the community. It set out seven functioning states for adult
20
social care services based on the concept of wellbeing which were developed following the consultation conducted as part of the preceding Green Paper ‘Independence, Well-being and Choice’ (Department of Health 2005). • • • • • • •
Improved health and emotional wellbeing. Enjoying good physical and mental health (including protection from abuse and exploitation). Access to appropriate treatment and support in managing long-term conditions independently. Opportunities for physical activity. Improved quality of life. Access to leisure, social activities and life-long learning and to universal, public and commercial services. Security at home, access to transport and confidence in safety outside the home. Making a positive contribution. Active participation in the community through employment or voluntary opportunities. Maintaining involvement in local activities and being involved in policy development and decision making. Choice and control. Through maximum independence and access to information. Being able to choose and control services. Managing risk in personal life. Freedom from discrimination. Equality of access to services. Not being subject to abuse. Economic wellbeing. Access to income and resources sufficient for a good diet, accommodation and participation in family and community life. Ability to meet costs arising from specific individual needs. Personal dignity. Keeping clean and comfortable. Enjoying a clean and orderly environment. Availability of appropriate personal care.
There are a number of similarities between those QoL domains identified by Schalock et al. (2002) (see section 2.1.5 above) and the White Paper functioning states. However, the White Paper identifies ‘improved quality of life’ as a single outcome domain, which appears somewhat inconsistent – as quality of life itself (as defined by the QoL literature) arguably contains all those White Paper functioning states that have been identified. However, central to both the White Paper functioning states and Schalock et al.’s QoL domains is the actual day-to-day living experience of the individual. For example, ‘personal development’ and ‘making a positive contribution’ are only likely to be possible if individuals participate in activities which broaden their experience and allow them to develop new skills and interests and ‘interpersonal relations’ and ‘Improved emotional wellbeing’ depend on interacting with other people on a daily basis. The Department of Health White Paper functioning states are developed in a way that is relevant for the assessment of (health and social care) services (even though they appear to be rooted in a quality of life approach which starts from a person’s whole life experience). 2.2.1.2 Functioning approaches The capabilities and functioning work as developed by Sen’s work outlines the framework in which well-being and utility is conceived. However, Sen did not produce a definitive list of functioning domains, although he did refer to a number of fundamental functions around nutrition, to have shelter and so forth (in relation to work on poverty). His colleague Martha Nussbaum has developed a comprehensive formulation focused more strongly on imperatives than on freedoms and choice. This is the universalistic capabilities approach, which, according to Gasper (Gasper 1997, p299) ‘gives a rich picture of what is a full human life, and talks in terms of real people, real life, and not thin abstractions.’
There are four areas where Nussbaum develops Sen’s capabilities approach. First, she argues for objective universal norms of human capability across cultural boundaries which therefore are not constrained by cultural relativism. For her, ‘the good life is non-relative – in that it is invariant across classes, societies and cultures’ (Qizilbash 1998). Second, her account of the good human life is singular: she demands an account that ‘should preserve liberties and opportunities for each and every person, taken one by one, respecting each of them as an end’ (Nussbaum 2000). Third, Nussbaum is explicit about the functionings that make up a distinctively good life which for her are based on Aristotelian foundations of being organised by practical reason and which takes shape around other-regarding affiliations, ‘so, for Nussbaum, the good life is the life of the correct choice 21
of action’ (Qizilbash 1998). Finally, Qizilbash states that she distinguishes ‘those capabilities for functioning that are to do with the individual’s personal constitution from the external conditions which facilitate the exercise of such capabilities’ (Qizilbash 1998). Her most tangible difference from Sen is that she provides an extensive list of central human functional capabilities as a basis for determining a decent social minimum in a variety of areas (Nussbaum 2000). The version of the list given below is derived from two lists developed by Nussbaum (Nussbaum 1995; Nussbaum 2000). 1. Life: being able to live to the end of a normal human life. 2. Bodily health: being able to have good health, including reproductive health; nourishment; shelter. 3. Bodily integrity: being able to move freely; having bodily boundaries (i.e. secure against assault); having opportunities for sexual satisfaction and choice in reproduction. 4. Senses, imagination and thought: freedom of use and expression of all three, including politics and religion [all informed and cultivated by education] – and pleasure and avoiding non-necessary pain. 5. Emotions: to be able to feel them and not for them to be blighted by fear, anxiety, abuse or neglect. 6. Practical reason: being able to form a conception of the good and to engage in critical reflection – including the use of conscience. [Being able to plan one’s life, seek employment and participate in politics]. 7. Affiliation: to be able to interact, show compassion, to have friendships. Protection against discrimination and enabling of dignity, self respect. 8. Other species: being able to live with concern for and in relation to animals, plants and the world of nature. 9. Play: being able to laugh, play and enjoy recreation. 10. Control over one’s environment: political – effective participation; material – being able to hold property on an equal basis with others and to have equal access to employment. [Being able to live one’s life in one’s own surroundings and context, i.e. guaranteed freedom of association and freedom from unwarranted search and seizure. Freedom of assembly and speech]. 11. Being able to live one’s life and nobody else’s: i.e. non-interference with choices over marriage, child bearing, sexual expression, speech and employment. Nussbaum claims that there is component pluralism and irreducibility in her list, and furthermore, a lack in any one of them leads to a shortfall in ‘a good human life.’ Practical reason and affiliation (items six and seven) are particularly significant for Nussbaum. She identifies ‘spheres of human experience that figure in more or less any human life and in which more or less any human being will have to make some choices rather than others’ (Nussbaum 2000). For Nussbaum a human being is a ‘dignified free being who shapes his or her life in co-operation and reciprocity with others…a life that is truly human is one that is shaped throughout by these human powers of practical reason and sociability’ (Nussbaum 2000). 2.2.1.3 Developing attributes for a measure of QoL for older people In their 2006 paper, Grewal et al. explore the views of people aged 65 and over regarding what is important to them in terms of QoL. The work concentrates on the development of attributes for a new measure focusing on QoL for older people, rather than health related quality of life or the influences upon quality of life. The purpose was to determine the important attributes of quality of life for older people.
In total, five conceptual attributes were developed 1. Attachment: incorporates feelings of love, friendship, affection and companionship. Sources of which include partners, family, friends and pets. 2. Role: incorporates the idea of having a purpose (or put simply ‘doing something’) that is valued, either by the individual and /or by others. 22
3. Enjoyment: the notions of pleasure and joy, and a sense of satisfaction. Sources of which include personal and communal activities. 4. Security: incorporates the ideas of feeling safe and secure, not having to worry and not feeling vulnerable, influences upon which include having sufficient finances, sufficient practical and emotional support and sufficient health. 5. Control: involves being independent and able to make one’s own decisions. Influences upon control include health, both mental and physical, and having sufficient finances. Similar to the work of Sen (Sen 1993; Alkire 2002; Gasper 2002; Robeyns 2003) , according to Grewal et al. (2006) what was noticeable about the discussions with participants regarding what reduced their quality of life, was the extent to which quality of life was limited by the loss of ability to pursue these five conceptual attributes of quality of life. For example, it was not poor health in itself which was perceived to reduce quality of life, but the influence of that poor health upon each participant’s ability to achieve the attributes of quality of life that appeared to be particularly important. What is also noticeable about the work of Grewal et al., when compared to the writings of Sen, is the importance of context. At no point was the basic capability of nourishment that is the focus of many of Sen’s examples discussed by participants. This suggests one of two things: the first is that in the context of the UK, even among older people, many of whom are officially defined as living in poverty; it is assumed that such basics as having enough to eat will be available. The second is that, because the focus here was on QoL rather than survival (or quantity of life), basic survival needs were not discussed. Indeed, informants concentrated almost exclusively on health as it related to their quality of life rather than the need for health for survival. It may also be noted that, because these informants were older, the need for survival, say to look after young children, was perceived to be less important. The attributes of quality of life identified by Grewal et al. (2006) are clearly functionings. The set of functionings is one that is important for older people in the UK in the context of the early 21st century. Yet it is also a set of functionings that is (a) clearly related to QoL or wellbeing, rather than to survival and (b) clearly distinguished from the influences upon QoL such as health, which are also suggested by Sen and others (Robeyns 2003) to be functionings. Further, the interpretation that it is the capacity to achieve these functionings that is important to older people is entirely consistent with the findings reported by Grewal et al. (2006), particularly considering the negative influences on QoL where participants frequently discussed their inability to achieve certain endpoints. Thus it is the capability to achieve attainment, to feel secure and so on which are the ultimate attributes identified. Grewal et al. (2006) suggest that future work should decide how the relative values of these attributes should be determined, and what those relative values are. But in deciding how to determine the relative values of the different attributes, there are important questions about whose views should be used: is this a societal issue? Or is it to be determined by academics or policy makers on behalf of society? Or is it the views of older people themselves that are important? The work by Grewal et al. (2006) set out from the perspective of determining attributes of quality of life for older people for use in a preference-based measure of QoL. As a result of the findings it ended in somewhat different territory. Aspects of QoL were indeed defined, but another way of looking at these is as a set of functionings. However, crucially, it is the capability of older people to achieve these functionings that appeared to be of greatest importance. For this reason, Grewal et al. believe that the continued work of developing a measure for use in health and social policy decisions that aim to enhance QoL should focus on the development of an index of capability for older people using attributes that are explicitly concerned with capability rather than functioning. 2.2.1.4 Health status Although Grewal et al. do not see health as a final outcome, many of the other approaches see health or at least physical wellbeing as an important outcome in its own right. In the health
23
literature the QALY takes centre stage as an outcome measure. Further investigation shows that the main QALY measures also incorporate quality of life or non-health well-being type dimensions. For example, a standard QALY instrument, called the EuroQol EQ5D (because it has 5 dimensions), includes usual activities (e.g. work, study, housework, family or leisure activities) and self-care (e.g. washing and dressing) as well as: depression, pain and mobility 5 . The EQ5D is an instrument that asks people to rate how well they do in these 5 dimensions. The answers are scored and added together using a pre-determined set of weights (derived from a sample of people’s preferences for functioning in these 5 dimensions). The functioning state measures for social care discussed above cover some of this ground. Anecdotally at least, we would expect social care services to also have an effect on people’s physical and mental health. Allowing for the impact of social care on health outcome domains such as pain and depression would therefore seem to be important. Whether and how this should be done are questions for further work. 2.2.2 Measurement of functioning states There are a range of practical considerations in how to measure the functioning states of social care services, having identified what to measure. Clearly, these will depend on what ‘outcome’ approach is adopted. The conceptual work has highlighted a range of techniques with broadly three types. First, those that measure the functioning states of services directly as they are experienced. This type is associated with conceptual models above that feature direct measurement of utility or wellbeing. Hedonic psychological approaches are an example where people are asked at various intervals during their experience of the service to rate their happiness (Stone, Shiffman et al. 1999). The second type is also based on direct measurement of utility but where people are asked to rate different services or service options. Willingness to pay studies are an example (Shackley and Donaldson 2000; O Shea, Stewart et al. 2001; Shackley and Donaldson 2002; Olsen, Kidholm et al. 2004; Protiere, Donaldson et al. 2004). Here, a sample of people is asked to say how much they are willing to pay to receive or to lose one service (option) compared to another. The more highly valued service option is the one that attracts the greatest willingness to pay. The costs of the service can also be subtracted from this amount to give the net value of the service, B – C, with a positive value indicating that the service should be considered for funding.
The third type relates to the conceptual approaches above where direct measurement is not possible and instead final utility is inferred from a small set of outcome indicators. It has two parts. People are asked to rate how services have (or might) change their achievement of a set of functionings as compared to some (pre-service) baseline situation. The other part involves asking people to weight the relative importance of the set of functionings used. There are a range of techniques to elicit people’s preferences. The discrete choice method gives people a series of binary choices between sets of functioning states and uses statistical techniques to infer a weighting from these expressed choices (Burge, Gallo et al. 2006; Ryan, Netten et al. 2006) . The Best-worst method is also choice based where people are given a series of scenarios that combine various functioning states at different levels of achievement. In this case, however, respondents rate each outcome attribute in the scenario from best liked to worst liked. Again statistical modelling is used to infer overall weightings (Grewal, Lewis et al. 2006; Flynn, Louviere et al. 2007). Both discrete choice and best-worst techniques have been used successfully in social care (Burge, Gallo et al. 2006) building on the work around the older people’s utility scale (OPUS) (Netten, Ryan et al. 2002). One of the challenges in making these measurements is in defining the reference states, that is, putting a numerical score on the worst possible state in any particular outcome domain and the best possible state. In the QALY work, death is the zero rated reference state. Complete health is the best possible state and this is scored at 1. So states between death and full health are rated between 0 and 1. It is possible however, for people to be alive in a state that is worse than death, and this accrues a negative score. In the OPUS/CfB approach in social care all needs met has a score of 1 and no needs met a score of 0. But in this approach there is no external reference point. Achieving a zero score – no needs met (in all domains) – depends on what domains are included in the measure. This set-up strictly limits the difference between best and worst to take a value of 5
See http://www.euroqol.org/
24
1. When comparing services measured in the same terms, this is perhaps not a significant problem. But it is more of a problem when deciding service levels relative to a cost-effectiveness threshold. For example, suppose a service improved people’s functioning state from ‘no need met’ to ‘all needs fully met’. In the above terms, this is an outcome gain of 1, which might justify, say, a spend of up to £30,000 (incremental) cost on the service. But what if ‘no needs met’ was not that bad (or not that bad relative to death) and ought to be scored at 0.2 rather than 0. In this case, as a result only of changing the scoring system, the service is now only justifiable at a cost of less than £24,000.
2.3
Intermediate outputs, service specific measures or process quality
The conceptual work suggests that the ultimate effect of services is on the well-being or utility of service users (and others that are affected e.g. carers). But this effect is difficult to measure, not least because a whole range of factors in people’s lives affect well-being in addition to services. Specifying functioning states that are related to services is one way to resolve this problem, but attribution problems remain. An alternative is to use measures that are more ‘service specific’. These often concern so called ‘process quality’ measures, such as whether the carer turns up on time. In the causal chain, process quality affects functionings (such as people’s sense of being in control of their lives and also in terms of how long they remain in a state of undress etc.). In turn, functionings states affects well-being. So we can infer that better (process) quality will ultimately imply better well-being, and that’s really what we want to achieve. Process quality is (relatively) easy to measure, but it cannot be used to compare services for which those processes are irrelevant. For mainstream social care services, a range of specific measures have been developed. Those used in the (extension to the) DH user experience survey are good examples (Malley, Netten et al. 2007). 2.3.1 Outputs of I&A services Thus far we have been concerned with services that have a clear and direct effect on people’s quality of life. They are services designed to help individuals in tangible ways. Some services, like information and advice services, also ultimately affect well-being. However, their effect is perhaps more indirect than (personal) social services. Information is likely to affect choices of services and choices about lifestyles. For these reasons the problem of attribution with measures of final functioning states is more severe and so service specific measures are more important.
An initial scan of the literature suggests that very little work has been done in the field of conceptualisation and measurement of the value and outcomes of information and advice services. There is a lot of literature around the quality of library and information services but these tend to be focused on the quality of inputs and processes. There is some work on the benefits of library services (Poll and Payne 2006), the benefits of providing welfare rights advice in health settings (Abbott, Hobby et al. 2006), service user experiences of legal advice and some discussion and guidance about quality and quality assurance in other fields, but little that focuses on output and outcome measurement, particularly from the perspective of the service user. One useful approach that develops service specific measures in information and advice services is that of Saxton et al (2007) for measuring the benefits and costs of 2-1-1 services in the USA. 2-1-1 has been introduced in North America as a free, easy to remember number for finding human services answers. By February 2007, 2-1-1 was being used in five areas in Canada, all or part of 41 USA states, plus Washington DC and Puerto Rico. Each service provides information about a wide range of services and resources in the local area including: basic human needs (e.g. food banks); health services; employment support; support for older and disabled people; support for children and families; and volunteer opportunities. The model was developed as part of a study which explored the benefits that users receive from 21-1 services that were distinct from the social assistance provided by the various service agencies. The wide ranging nature of 2-1-1 services provides a very helpful starting point, although, as we describe below, we extend it to cover more intensive types of support that are not normally provided by these services. 25
The model shown in Figure 2 identifies the inputs, activities, reach, outputs and outcomes (short, medium and long-term) for individuals, organisations and society as a whole. In the context of 2-11 services Saxton and colleagues maintained that important economic benefits would result through increasing the efficiency of those agencies that received increased referrals as the result of the service. Also at the broadest level 2-1-1 services would build an information infrastructure that would potentially both increase social capital and create relationships between organisations, reducing overlap and increasing co-operation. While acknowledging the wider benefits that advice and information services can provide for society and organisations, our focus for the purposes of the QMF study is the benefits to individuals and it is on the measurement of the outputs of these activities that we are focusing.
26
Figure 2. Model of Information and advice service inputs, outputs and outcomes (Saxton et al 2007)
Inputs
Activities
Reach
Funding
Telephone: • Provide information and referral services to callers • Problem solve
Individuals • Employment • Health • Child care • Housing
Aggregate info: • Produce aggregated database of community resources
Organisations: • Non-profits • Government agencies • Businesses
Partners: • Participate in local community non-profit and public services community
Society: • Communities • Neighbourhoods • Cities/councils • States • Countries
Partners/stakeholders • United Way • Non-Profits • State agencies • Local agencies • Public
Research and best practices
Public information
27
Figure 1 Model of Information and advice service inputs, outputs and outcomes (Saxton et al 2007) continued OUTCOMES
Outputs
Individuals
Organisations
Short-Term
Mid-Term
Long-Term
• Comprehensive information and referrals • Problem solving • Support
• Immediate answers • Saved time • Correct info.
• Comprehensive solving of problems • More time to focus on deeper issues
• Fewer people in need of resources
• Cost savings • Collaborative opportunities
• Saved resources • Knowledgeable clients
Organisations: • Better able to focus on strengths • Less time overall wasted
• Stronger and more focused
• Public resource • Knowledge base
• Better informed users of services
• Knowledgeable public • Social capital
• • • •
Society
28
Better services Less needy Social capital Disaster infrastructure
Activities At the individual level information and advice services can include a wide range of activities. We draw on the framework developed by the Community Legal Services which formed the basis of a quality mark (Community Legal Services 2000). In this, organisations were classified in terms of whether they provided:
• • •
• •
Self-help information: (website, leaflets etc) service staff have little or no interaction with the client when obtaining information Assisted information: providers will assist clients in finding information. May not be sole purpose of the organisation e.g. library or benefits agency General help: providing advice relating to personal circumstance usually person to person (telephone or face to face) bringing in a new perspective, giving information and explaining options, identifying further action clients can take. Includes basic assistance e.g. filling in simple forms General help with casework: includes negotiation on a client’s behalf with a third party/ representing someone through writing on their behalf. Specialist help: in law is for organisations that provide legal help on complex matters in specific areas of law.
In addition to this we might include • Long-term advocacy - where the relationship with the client forms a fundamental objective We might decide to exclude this type of advocacy from Information and Advice and include this in low-level interventions or mainstream services where we would expect to evaluate through our functioning states framework. For the time being, however, we will include this as the most intensive type of information and advice service. Reach In Saxton et al’s model, reach at the individual level covers the nature of advice and information: whether it relates to child care, employment, health and so on. It seems helpful to be able to classify a service in terms of areas of advice covered. 6 We need to decide on what type of taxonomy should be employed. Ideally this would be a pre-existing and intuitively sensible set of domains. The sorts of headings we might expect would be: • Welfare rights • Employment • Health • Education • Child care • Social care for adults (distinguishing client groups?) • Criminal legal advice
In addition to the scope of information provided, reach might cover the information needs of the individual in the absence of the service. Initially we may simply classify individual’s information and advice needs as ‘general’ or ‘additional’ although we should consult on whether more divisions would be needed. General needs would be the equivalent of the general population. Additional needs might include people with learning disabilities, people with communication difficulties and those who are insufficiently fluent in English (where English is the main language in which information is provided). Perhaps this might also include difficult to reach/ socially excluded groups. 6
Early thoughts had been that these might relate to broad quality of life domains but discussions with an information and advice expert clearly showed that services would not be easily distinguished on such a basis as a wide range of services would be able to demonstrate some contribution to almost all domains.
29
Outcomes Using Saxton et al’s definition of short, medium and long-term outcomes as a starting point: • Short-term outcomes for individuals include immediate answers, saved time, correct information, knowledge of options and knowledge of rights. We might also anticipate for some service users a sense of empowerment and reduction of anxiety or stress • Medium-term outcomes for individuals include problems solved and access to services/benefits. We might also include health related behaviour here (such as giving up smoking, taking up exercise and so on) • Long-term outcomes for individuals include the results of actions and decisions which might include better quality of life resulting from (for example) appropriate service/housing solutions, better health and so on. Thus these are equivalent to functioning states that we have defined above.
In terms of reflecting these outcomes in any measure we are unlikely to be able to pick up long-term functioning states but might look to infer these from known relationships with medium-term outcomes (see below). However, the degree to which a service delivers short and medium-term outcomes for individuals should be possible to monitor. A common situation in social care will be a relative, friend or neighbour contacting an I&A organisation on someone else’s behalf. In such instances the short-term outcome might be for the individual contacting the service but the medium and longer-term outcomes would be attributable to the beneficiary of any resulting or subsequent activity. Quality While quality is not specifically addressed in Saxton et al’s framework, as in previous work we would expect quality to reflect the degree to which outcomes are in fact delivered as aspects of the process. The process factors are likely to relate to short-term outcomes and be influenced by the level of information and advice provided. One of the purposes of the Community Legal Services Quality Mark approach (see above) was to identify aspects of quality that would be expected of services providing the different types of activity identified above. Where the service is limited to information, quality might include: accuracy, being upto-date, relevance (what was needed), clarity, and timeliness/availability. Where assistance and support is provided in addition to these aspects of quality we might expect courtesy, impartiality, helpfulness, complaints procedures being clear, respectfulness, people feeling listened to, and advice addressing relevant issue to be important. Where there are additional needs it may be that there are further aspects of quality that would need to be considered such as providing information and advice at the right level in terms of amount and interpretation so people can understand and use it, sensitivity, and being non-judgemental.
Another aspect of quality we might want to consider is the degree of ‘fit’ between the medium and the message (in the field of health information the transmission of information in various forms – video/leaflets/and so on – has been found to have both positive and negative affect on patient satisfaction with a service). In all instances, we would want to be clear that our quality indicators relate to outcomes. 2.3.1.1 Measuring outputs of information and advice services In order to reflect the benefits of a service our output measure should reflect short, medium and long-term outcomes. Short-term outcomes are the immediate impact of information, closely linked to quality of process and might include peace of mind, saved time or similar. These could be attributable to someone other than the eventual beneficiary of subsequent action.
We could define medium-term outcomes as actions taken as a result of the information and advice. These could then be measured as activities, and weighted to reflect the results of
30
these activities (that is their long-term outcomes). In social care, the principal subject of this report, these long-term outcome weights could reflect our measures of outputs of care homes, low-level interventions, and so on. For example, if the identified activity was someone moving into a care home or remaining in their own home instead of doing so, once national data were available we would have a basis for estimating the value of the action taken. Clearly, the weighting would need to reflect just the relative importance of this activity compared with other activities undertaken as a result of I&A services, as the actual output would be attributable to the care home. In terms of ensuring our measure of outputs reflects what an I&A service is actually providing itself (as opposed to ensuring access to) we could start with the probability that the individual would have accessed the medium-term outcomes in the absence of the service. For vulnerable adults we might assume this is zero, for the general population this might be overall levels of access that might be available from other sources (e.g. non take-up of financial benefits). Once somebody has received information or advice or both their probabilities of achieving the outcome will have been increased (this could be reflected in actual measures of what people in fact experience from a survey of people using the service linked to the stage they get to). For those where no further action is appropriate (and the advice is that) then the outcomes will all be short-term. Using the probabilities approach we could reflect both the ‘information needs’ of the individuals and the amount of service provided. Services that just provide information would expect a lower increase in probability of successfully achieving medium-term outcomes than those that provide personalised advice/ casework/ advocacy and so on. In addition, we might want to consider the speed of take up in that information and advice services help the service user to more quickly achieve beneficial outcomes. This would be an further well-being gain. Clearly there are many challenges ahead. However, this provides us with a starting point in considering how we might approach valuing of I&A services.
2.4
Evaluating the different approaches
2.4.1 Applying evaluative criteria The conceptual work highlights some important considerations in developing a practical approach, drawing on those identified above. But what constitutes a good way of measuring functioning states? The following is a list of key design considerations that have been synthesised from the literature.
• •
•
Relevance ¾ Does the approach ultimately produce a good measure of value? Measurability ¾ Can these functioning states concepts be measured in an unbiased and scalable way? ¾ Is reliability and consistency of measurement high? What about framing effects (people being led by questioning)? What about adaptation behaviours (people re-aligning their expectations and aspirations to their current circumstances)? Attribution & association ¾ Can we ensure that changes in measured functioning states are due to changes in services and not other factors? ¾ Are people able to interpret and express their experience of services in terms of (more abstract) functioning states concepts?
31
•
• •
Aggregation ¾ Can we find a reasonable way to add-up the valuation of services across individual people? Are some people’s valuations more important than others’? Are there principles that go beyond the simple aggregation of individuals’ experiences (e.g. moral principles)? Comprehensiveness ¾ Are the effects of services accounted for in all relevant areas? Universality ¾ Whose preferences count: service users, general population etc? People that have actually experienced the services as opposed to those who anticipate the experience?
We can focus on the cross-cutting themes and issues that emerge from the above description of the different approaches. First, should utility be measured directly or is it sufficient and appropriate to rely on the measurement of (proxy) functionings? The latter might be a set of functionings e.g. being fed, being clean, feeling in-control etc. The former option does well on the relevance and comprehensiveness listed above but suffers, comparatively speaking, on measurability and attribution. Indeed, can utility actually be measured directly? And are these measures sensitive enough to pick up effects of services? On these grounds, indirect approaches appear more promising. Second, should we try to measure the functioning states that are important only to individuals or follow an extra-welfarist approach and measure the functioning states (and so service outcome) chosen to be relevant by a mandated decision-maker? The latter does not necessarily have to account for all the functioning states that are potentially important to individuals (assuming that could be done). It can focus instead on a subset (as in the health care example above). In this way, relevance, measurability and attribution improvements are traded against loss of comprehensiveness. In practice, a combination of approaches is possible. The decision-maker takes into account some outcomes valued by individuals but adds some more. Third, where a set of outcome domains is used, should the functioning states in each domain be added up in an arbitrary way (to form an overall score) or should some form of preference weighting be used? The latter allows the overall score to reflect the relative importance of different functioning states to people e.g. being fed as more important than being unoccupied. Relevance is highly questionable for the former option and on this basis alone, some form of importance weighting needs to be considered. Fourth, whether an external or service user (subjective) approach is taken. This option concerns how the relative importance of the achievement of each outcome (in a set) is determined. Or it can (also) refer to whether the achievement of an outcome is verified externally. In the first case, the external approach could be where the general population rather than the sample of actual service users is polled regarding the importance of functioning states. In the second case, the subjective option is where the service user themselves rates whether the service has helped them achieve an improved functioning state. Alternatively, an external observer can make this judgement. The main consideration is measurability versus relevance. Subjective approaches can potentially suffer from framing effects (in how people interpret whether functioning states are achieved) and adaptation to environment. The psychology literature is replete with examples of how people adapt their expectations and aspirations to their current situation. By contrast, objective approaches might be criticised for not actually measuring what matters – people’s wellbeing. One way to possibly square this circle is to have subjective happiness as an outcome domain in addition to domains that concern more obvious functioning states (like being clean and dressed).
32
Fifth, should we assess value in terms of the achievement of actual functionings, or the capability to achieve them? A person might be given the opportunity to engage in meaningful social activity by a service but (freely) chooses not to do so. At face value, it might appear that the service does not help people to have good social contact, but actually it does and, more importantly in this example, gives people the chance to make their own choices. Measuring potential outcome achievement is clearly more difficult than measuring actual outcome achievement. A potential solution is to measure actual functionings and also whether or not people feel in-control/have choice functioning state. Finally, there is the choice between using general functioning states or service specific proxies (which might also be called intermediate outputs) – see also next section. With the former, a service can be rated on how much it helps people to achieve a set of relatively general functioning states such as whether people feel attachment (affection, friendship, love etc.), dignity/self-respect, occupation, security, control, etc. Alternatively, functioning states can be inferred from a range of more specific measures. For example, people can be asked to rate their relationship with their care-giver. A number of studies have shown how important these relationships are for people. Arguably this stems from the impact a poor relationship has on three general functioning states: (a) loss of affection and friendship (b) loss of respect and dignity and (c) loss of control. The advantage of the latter ‘proxy’ approach is that attribution of effect to services is high. The main downside is the loss of comprehensiveness. A mix of these approaches is possible and appears the best way forward. 2.4.2 Which approach? None of the above conceptual approaches fully satisfy these criteria in isolation for our purposes (in as far as they can be isolated). Instead a synthesis is required. Each approach has relevant and useful features. As part of the capabilities and functionings approach, people value the improvement in functionings that services make possible, and not those services themselves. This approach makes clear that the functioning states as described in social care can be regarded as improvements in functionings from Sen’s perspective. Furthermore, although the capabilities and functionings sees a role for an extra-welfarist style decision-maker and the use of non-utility information, this does not in any way rule out the use of utility information and preferences in the welfarist sense. Indeed, in keeping with directions in policy regarding objectives, decision-makers are emphasising the importance of individual person’s objectives and well-being. So a decision-maker’s objective function can be used but where this embeds the outcomes at the individual level, and where the outcomes are weighted according to individuals’ preferences.
The persistent and everyday nature of (long-term) social care means that services have effects that touch many aspects of people’s lives. So whilst quality of care is not the same thing as quality of life, good quality care will impact positively on many of the domains identified as associated with a good quality of life. The quality of life literature has therefore much to offer. Welfarist (and to some extent extra-welfarist, although perhaps less dogmatically) approaches are explicit about aggregating individual preferences and also the relative importance of different outcome domains. In the latter respect, there are real issues around double counting effects i.e. where separate domains measure the same underlying concept but are summed up individually. As to the former, being clear about the assumptions required to tackle the aggregation problem is paramount. The capacity for benefit (CfB) work ties many of these features together. It incorporates the strong points of welfarism (being preference-based and concerning individual’s utilities), extra-wefarism (in thinking about functioning states and objectives from a service perspective) and quality of life (thinking comprehensively about the breadth of effect that care services can have, and how this is centred on individual service user experience). In this 33
respect it has strong analogies with the capabilities and functioning approach e.g. in defining outcomes as the effects of services in changing peoples functioning state. The capacity for benefit approach is therefore a good starting point, but there are some areas that are highlighted by the above literature survey that could guide further development. First, although it shares many desirable characteristics of Sen’s approach, not all of the latter are incorporated. In particular, we need to make more of the distinction between freedom and achievement, that is, more of an emphasis on valuing the capability or freedom to have certain functionings rather than actual achievement of functionings (Ruta, Camfield et al. 2007). One way to address the practical measurement difficulties this implies is to explicitly include a choices, control or freedom outcome domain in the set as described above. This option incorporates freedom or choice as a functioning that is or is not achieved. In doing so, nonetheless, we must seek to avoid double counting problems where we count both the achievement – e.g. being clean and dressed – and the freedom that allowed people to choose to be clean and dressed. This is not to deny that a freedom to achieve is valued beyond an actual achievement, but rather than we only count the additional value (plus the value of actual achievement). Second, the capabilities and functionings approach makes the distinction between people’s current happiness/satisfaction with services given the situation in which they receive services, and their reflective evaluation of those services. This distinction resonates with the ideas of adaptation and selective recall that are highlighted in the psychology literature and needs to be incorporated in the capacity of benefit approach. A third area in which insights from Sen’s work and (by association) extra-welfarism, and also from the welfarist literature, are relevant to capacity for benefit is with regard to the aggregation problem. CfB as outlined above uses preferences to weight outcomes at an individual level. In deriving the capacity for benefit of a service however, the usual approach is to take an average of the outcomes of a sample of people using that service. This is to make the implicit value-judgement that people’s outcomes should be added up in an unweighted or utilitarian fashion. Extra-welfarist/Sen approaches are explicit that this is a valuejudgement and that others could be used. Furthermore, in deciding which assumption to make in this regard, mandated decision-makers are arguably the default authority.
2.5
Using outcomes information
We can deploy a (modified) capacity for benefit specification of well-being for a number of aims; principally to develop a well-being based index for National Accounts; and also, to explore the benefits of using this well-being definition in the commissioning of services. The detailed workings are given in Annex 7. 2.5.1 Well-being index We can gauge a change in circumstances in the economy e.g. a change in technology through time leads to marginal changes in the commodity vector x. In section 2.1.1.2, we outlined that under the welfarist approach the implied change in total well-being was given by n (⎛ n ⎞ the formula: ΔW = K ⎜⎜ υi xi1 − υi xi0 ⎟⎟ . With a capacity for benefit, the appropriate formula i =1 ⎝ i =1 ⎠ is the same, except that now υi is the change in the service-level average of the well-being of people using services – as measured by quality-weighted capacity for benefit – rather than the change in the utility of people using services.
∑
∑
To determine this well-being difference, information is required about (national) service provision levels, xi, and about the value of these services, υi . The former is routinely available, but the valuation of services is not routine information. As established in Annex 7,
34
(
)
with the capacity of benefit approach, this mean marginal valuation is: υi = φi − φ i qi – see, in particular, equation (48) in Annex 7. This function comprises the capacity for benefit, the term in brackets and the outcome weight qi. In addition, we will need to find a proxy measure for the level of functioning state without services, φ , since this is equivalent to the level of dependency people have and this can change over time. As outlined in section 2.1.1.3, there are a number of highly relevant reasons why a costweighted index is inappropriate. We are, in other words, only left with the option of directly measuring the marginal valuation of individual services. 2.5.2 Outcomes-based commissioning The gains from outcome measurement come from being able to provide services to the range of potential users in a configuration that best achieves desired objectives at an appropriate level of spend. What are these ‘desired objectives’? Welfare economics considers the preferences of individuals as paramount. 7 Extra-welfarist approaches allow for decision makers to make (arbitrary) judgements on outcomes that are important for public services and also what value-judgements are required for aggregation over individual people’s outcomes. In recent policy statements – e.g. the 2006 White Paper (see 2.2.1.1 above) – the Government, as decision-maker, has explicitly outlined its goals for social care (including an increasing importance attached to individual preferences as reflected in the personalisation agenda). And as outlined above, these public policy goals can be defined and stated in the functioning state terms we use above, and expressed as an objective function. This being the case, the achievement of best desired objectives for given public resources via commissioning is equivalent to selecting the ‘optimal’ or efficient configuration of services. How is this latter configuration achieved?
Standard welfare economics gives us important insights in this regard. It states that in the absence of (perfect textbook) markets, there is no automatic reason to suspect that services will be provided at efficient levels (section 2.1.1.2 and 2.1.1.3 above). In public bureaucracies it is internal management processes that lead to the deployment of resources and the configuration of services to be provided (Milgrom and Roberts 1992). Where these rules do not account for functioning states e.g. where cost-weighted services indices and other inputsbased rules are used, an efficient configuration of services will be only obtained by chance, and not by design. In principle, a series of outcomes-based rules can be derived that can guide an efficient configuration of services in these terms if we have: an estimate of the impact of services on well-being, an explicit specification of desired objectives in well-being terms, and if we make some necessary assumptions. These assumptions will include decisions about the distribution of well-being gains over service users. Also, well-being gains need not be directly synonymous with actual service use by individuals, and can be in Sen’s terms, for example, about the potential to benefit. Furthermore, non-individual well-being criteria can be included in the decision-makers objective function. By making different assumptions, this specification can accommodate a broad set of principles, as long as those principles can be expressed in a quantifiable way, and allow inter-personal comparisons of well-being (Sen 1979; Boadway and Bruce 1984). 8 Where an alternative set of rules are used that systematically violate these outcomes-based rules, then there is no automatic reason to expect that with available resources, desired objectives are best achieved.
7
But has to accept some important limitations in how these individual preferences are aggregated concerning the need for inter-personal comparisons of utility and the need to take explicit positions on distributional issues. 8 Furthermore, this quantification need not be cardinal terms.
35
The standard outcome-based rule requires that if the increase in utility from services per pound of expenditure is greater for service i than for other government services (G), then more funding should be devoted to service i – see Annex 8. In practice, the gain in utility from the use of other government services is not known. But, a common way to deal with this problem is to assume a guideline (cost-effectiveness) threshold, which is expressed as a cost per unit change on the well-being scale (as outlined in Annex 4). For example, suppose well-being is measured in units called OWLYs (outcome weighted life years), with analogy to the QALY. In capacity for benefit terms, this is measured on a 0 to 1 scale with a state of ‘no needs met’ in all outcome domains receiving a zero score i.e. φi − φ i = 0 has a OWLY score of 0. The state ‘all needs fully met’ in all domains receiving a score of 1. The decision-maker needs to set a threshold of how much society is prepared to pay per extra OWLY. On the health side, a guideline figure of around £30,000 is used i.e. the global willingness to pay for an increase in one person’s functioning states for a year from being in the worst possible state to the best possible state is £30,000 per year. The standard rule is then: (7)
pi < PG φi − φ i q i
(
)
The left-hand side is the cost-effectiveness of the care services in question. The right-hand side – PG – is the standard amount of money that government is willing to spend to increase utility by the equivalent of a standard unit of well-being, W. In this case, 1 unit of W is set at £30,000 (i.e. ∂W ∂B = 1 30000 in equation (54) of Annex 8). P
We can use this criterion to assess whether the current level of provision of a service is appropriate. Take day care as an example. On average, say, one new service user can expect a package of 2 sessions of day care per week. This is found to improve functioning states of the typical mid dependency, new service user by 0.2 OWLYs i.e. for 1 year of their life, their functioning states are improved by 0.2 on the 0 to 1 well-being scale as a result of the service. For new low dependency service users the gain is between 0.15 OWLYs and 0.05 OWLYs, depending on the specific characteristics of the service user. The annual cost of 2 sessions per week is £3000.
(
)
To reiterate, for mid dependency people, the gain in functioning states is: φi − φi qi = 0.2 with a cost of pi = 3000. For this group, therefore, an economic case for further resourcing can be made since the service only costs £15,000 per OWLY. For the low dependency users, day care is justifiable for people whose benefit is more than 0.1 OWLY (which is equivalent to £30,000 per 1 OWLY). We can also consider the balance between care services. For illustration suppose that for historical (supply) reasons, total expenditure by social services authorities on day care was equal to the spend on meal services. Is this the correct balance on outcomes grounds? The decision to increase the provision of service i (day care) relative to service j (meals services) is suggested if the ratio of well-being gains is greater than the ratio of costs: 9 (8)
pj pi < φi − φ i q i φj − φ j qj
(
)
(
)
So if an extra recipient of meals services (at an annual cost of £2000) were to experience an outcome improvement of 0.1 OWLY, then its cost-effectiveness ratio is £20,000 per OWLY 9
If the extra well-being gain for each extra unit of service was diminishing for service i are a greater rate than service j, then after a certain further increase in service i the equation (8) will be balance in which case, the ratio of service levels will be correct on this criteria.
36
compared to £15,000 per OWLY for (mid dependency) day care. More day care would therefore be required. The above rules are necessary conditions for the maximisation of well-being given the available budget. Alternative rules which violate these conditions will not give the same result. In the above example, if services were provided on the basis of cost-weighted output (service recipients), then at £2000 per recipient compared to £3000 per recipient, a greater provision of meals services might be implied. This is a stark example of an alternative rule, but it does serve to illustrate how different the service outcomes might be. A more empirically relevant alternative is where decision-makers allocate services according to their potential to met need. In this way, services are resourced in order, starting from the service that caters for people with the most severe needs, and continuing with services catering for successively less dependent people until the budget is exhausted (or where the marginal utility to the decision-maker of the last pound spend on care services equals the marginal utility of a £1 spent on other uses G). Currently, councils with social services responsibilities set eligibility criteria on the basis of need, such that, in the main, people above the threshold (who are also financially eligible) receive support but those beneath the threshold do not. The position of the threshold is shifted up or down so that eligible service uptake can be contained within budget (Wanless 2006). Compared to an outcomes-based rule, this system concentrates resources at the upper end of the dependency distribution in the population, even when marginal outcome gains per pound for these groups could be less than for people below the threshold (given that those people below the threshold start with very low levels of service). The upshot is that the outcomes-based configuration gives far more support to middle dependency people than the needs-based configuration. An example can be seen in Figure 3, which shows the well-being gain in the population for services for high dependency people and services for low dependency people. Using an outcomes-based decision-rule, services are provided up to a point where the change in well-being produced is equal to a cost-effectiveness threshold. In the figure, the former is given by the slope service outcome curves, and the latter is a tangent, where the slope is ∂W ∂y (e.g., which equals 1/30000). At this threshold, both high and low dependency services are provided, respectively at total expenditure levels of yH and yL. These expenditures correspond to total well-being gain of WT = WH + WL. By contrast, a needs based rule directs expenditure to high dependency services. Suppose services for high dependency people are funded to the same level of total expenditure yN = yH + yL. To demonstrate the effect, we suppose that there are enough high dependency people in the population so that only high level services are provided before the budget is exhausted. In this case, the total well-being gain is WN generated only by the high level service. It is clear, in this case, that total well-being for the same expenditure yN, falls short by the difference WT – WN. This occurs because the marginal well-being improvement for some people in receipt of low dependency services is greater than for some people with the high dependency service. What this means in practice, is that some people with high needs, get less services under an outcomes-based rule than they would under a needs-based rule. This follows from a utilitarian assumption. An alternative assumption might be to place more weight on the outcome gain of high needs people, just because they have high needs (a so called ‘rule of rescue’). In this case, an outcome-based rule would spend a greater proportion of the budget on services for high dependency people.
37
Figure 3. Comparing funding rules
Total wellbeing gain per £ WT = WL + W H WN
Outcome gain high dependency
WH
Cost-effectiveness threshold
Outcome gain low dependency WL
0
yL
yN = yL + yH
yH
Expenditure
The outcomes-based decision rule outlined is conditional on a range of assumptions. Changes in these assumptions will change the nature of the decision rule. Moreover, changes in the assumptions are likely to produce a decision rule that is more complicated than the one above, and, as a result, more demanding on the empirical parameters. A number of the assumptions are particularly important. •
First, those about the weighting of individual outcomes in the decision maker’s objective function (the aggregation problem). For instance, in moving away from a utilitarian objective function, alternative distribution assumptions could be factored in by using certain distributional weights on individual’s outcomes or utility. The above example noted the implications of using non-utility information. But how are these weights actually quantified (Donaldson 1999)?
•
Second, the impact of services on individual people’s well-being using the chosen metrics needs to be measured, and this entails a range of assumptions. For example, as outlined in Annex 7, when using a capacity for benefit approach we need to make assumptions about the independence of services and about how average service user dependency changes as services expand or contract. We also need to make assumptions about what outcomes are important for service users, how the achievement of outcomes should be measured and so forth.
•
A third significant assumption concerns the value that is chosen for the costeffectiveness threshold. An actual estimation of this value presents huge practical difficulties. But, any assumption that is made will have a substantial bearing on the resourcing levels of care services.
The above examples illustrate the effects of different ‘rules’ for allocating resources. In practice, the goals of policy makes will be broader than just cost-effectiveness. Also, other specific considerations will be relevant. In other words, the application of these rules can be regarded as helping to guide commissioning, to act as tools which commissioners can use to aid decision-making.
38
2.6
Some conclusions from the conceptual work
The main aim of this paper is to develop a conceptual framework for understanding why we need to collect information about the well-being or outcome consequences of services, how this information should be used, and how practical measurements of these outcome consequences could be made. We surveyed the main conceptual alternatives. Welfare economics seeks to determine the best way to use resources available in the economy. Starting from the preferences of individual people, it shows how ‘social welfare’ can be maximised and makes explicit the conditions under which this can be done. It provides the basis for our thinking about deploying services in ways that best improves the utility or well-being of people using services. A welfarist approach tells us that for nonmarketed services, direct measurement of the value of services (in outcome terms) is required. It also highlights the problems of (a) measuring individuals’ utility and (b) adding this up in order to cast light on the outcome implications of services used by many (different) people. Fundamentally, welfarist theories show why we should measure functioning states/outcomes and use this information to guide resource deployment. For example, welfare theory is the basis for calculating the value of output of goods and services (including government funded services) in the economy. An extra-welfarist approach shifts the perspective from some ‘black box’ process of aggregation of individuals’ preferences to one where a mandated decision-maker is responsible for deciding what is important (what the goals are) in terms of government activity. Although decision-makers take their lead from the society, these ideas help us to explicitly integrate policy statements about the aim of public services (e.g. from the 2006 White Paper) into our analysis. It shows that a distinction can be made between the strategic objectives implied by a social welfare function approach (aggregating people’s preferences) and the goals that the service authorities set for themselves. There are also key messages from the extra-welfarist work in healthcare, such as the development of the QALY, in how it is measured, the equity concerns, and how it plays into cost-effectiveness considerations. The hedonic psychology literature argues for the direct measurement of well-being (at least in some limited sense). New hedonics points to a way to make this measurement of wellbeing more objective. This is a promising line of inquiry, as it avoids having to find a way to explicitly measure the relative importance (or preference weights) that are needed with inferred well-being approaches. However, it presents significant practical measurement problems e.g. having to ask service users at regular intervals to rate their experience of services and potentially problems in identifying the counterfactual: what well-being would be in the absence of services. The psychology literature also flags up the important issues of adaptation and contextspecific responses about well-being. The evidence suggests that people make errors in trying to recall past experiences (relevant to valuing services). Also, people’s responses will be influenced by their situation, aspirations and expectations. This behaviour will make comparing across different service settings more difficult. For example, comparing institutional-based services versus care support provided in a person’s own home. The capabilities and functionings (C&F) approach is highly relevant and offers key insights. In particular, people value the improvement in functionings that services make possible, and not those services themselves. Also, there is the emphasis on valuing the capability to have certain functionings rather than actual functionings. One way to sidestep the practical measurement difficulties this implies is to explicitly include a choices, control or freedom outcome domain in the set. The capabilities and functionings approach also resonates with the ideas of adaptation, making the distinction between people’s current happiness/ 39
satisfaction with services given the situation in which they receive services, and their reflective evaluation of those services. Finally, there are messages regarding how we specify different functioning states. The key contribution of the quality of life literature is in pointing out what types or domains of outcome are important to people. There is a level of consensus regarding the range and types of life experiences that are important and highly valued by people. Where this literature is less strong is in specifying how these outcome domains are brought together to give an overall rating that is useful for service deployment. The capacity for benefit (CfB) work links outcome gains specifically to services although, following the capabilities and functioning work, operates with ‘objective’ population based preferences regarding the importance of different functioning states. In this latter respect, CfB incorporates the strong points of welfarism. It also reflects the C&F approach in valuing services in terms of their effects on outcomes (functioning state improvements), and the quality of life literature in thinking comprehensively about the breadth of effect that care services can have, and how this is centred on individual service user experience. The paper developed a set of criteria with which to assess these alternatives in the context of our main aim. These suggested that a (further) merging of capacity for benefit and capabilities and functionings approaches would be the best way forward. The former offers a practical, service-specific focus using well defined functioning states (outcome domains) and preference-based weighting. The latter, centrally addresses the role of choice and freedom or capability in relation to well-being. It also embodies the explicit distinction between subjective and objective well-being and the effects of adaptation. Finally, the capabilities and functionings approach brings an extra-welfarist perspective, mitigating some aggregation problems and allowing non-individual utility information to be included. The respective literatures point to the outcome domains that should be included. First, there are functioning states relating directly to people’s activities of daily living and security (being clean, dressed, nourished, safe). Second, those concerning a person ‘being in control’ and having self-determination of their lives. Third are the functioning states that relate to having a good social life, social relationships and being occupied and fulfilled. Fourth are functioning states that concern the place in which people live and receive care e.g. being in one’s own home, having a clean and comfortable environment. Fifth are the health related functioning states, such as not being in pain and not being depressed. Sixth is feeling a sense of selfworth, being treated with dignity and having self-respect. Interpreting the latter as a service outcome is perhaps the most difficult because it is affected by many other non-service factors. Nonetheless, given the personal and pervasive nature of services, it is clear that they will have some effect (possibly negative) on these emotional functioning states. Turning to the specific aims of this paper, the conceptual work has outlined the benefits of knowing the outcome consequences of services in terms that are consistent with the prevailing set of strategic objectives for publicly funded services. A relatively straightforward metric for National Accounting purposes is developed using the combined CfB and C&F approaches. Furthermore, the combined approach produces a practical outcomes-based decision rule for commissioners to use to guide service deployment. We show that this rule produces different service configurations than alternatives such as cost-based or need-based rules. The emphasis in this report on conceptual development has been made to be clear about why, and also in what form we measure functioning states. Before getting into the detail of how to measure functioning states in practice, it is important to understand the motivation for the task: that is, using outcomes information to inform public policy. In this respect, it is
40
crucial to know what information to collect, how it can be used, and what assumptions need to be made. A conceptual framework is needed to answer these questions.
3 Empirical strategy 3.1
Aims
In this section we outline our plans concerning phases 2 and 3 of the project, which are the main empirical parts of the research. Phase 2 includes the development of the fieldwork instrumentation and the piloting. Phase 3 is the main fieldwork phase. Given the scale of the work in phases 2 and 3, we break this down into a number of parts. Below we outline the nature and purpose of each part and give timings and resources required. The empirical work of this project looks at three service areas in phases 2 and 3: care homes, low-level services, and information and advice services (the preference study comes in phase 4 of the project). For the three services studies the main aim of the empirical work is to use the theory to develop practical and relevant functioning states metrics to form a social care toolkit. This function includes developing questionnaires and other instrumentation, as well as testing for understanding, validity and reliability (phase 2). This will be tailored to each service area, although the intention is to maintain a high level of consistency between the metrics so that they are comparable at high level, but allow us to ‘drill down’ into certain outcome areas that are most relevant for the service in question. We will pilot and test the toolkit and then deploy it in large scale surveys of service users to determine the (average) outcome of services. We have three objectives for the empirical work. First, to establish a measure of the value of services that can feed into the National Accounts on a regular basis using routine data collections. As indicated by the conceptual work above, a value-weighted index of output is required for this task (see equation (2) above). Using the capacity for benefit approach, an estimate of the (marginal) valuation can be determined using baseline capacity for benefit and quality information, where the latter can be measured on a yearly basis, but the former at a lesser frequency. The marginal value of services in the above index is equal to the gain in well-being generated by the service, that is: υi = φi − φ i qi – see section 2.5.1. Capacity for
(
)
benefit (the term in brackets) has to be established from a bespoke survey. The outcome weight qi can be measured using routine quality assessment. For this to work, however, we need the routine quality measures to be a good proxy for functioning states – they need to correlate highly with the outcome-quality weighting factor qi. Establishing this degree of correlation empirically is therefore our main empirical objective. In addition, we will need to find a proxy measure for the level of functioning state without services, φ , since this is equivalent to the level of dependency people have and this can change over time. For example, if care homes are catering for increasingly dependent people, that is new cohorts people with lower functioning states than those before, capacity for benefit from the service will be higher, and so therefore will be the outcome of the service (the change in W) if quality is unchanged. Again we aim to look at the correlation of dependency measures we use in the toolkit and those used in the regulatory process (primarily CSCI’s Annual Quality Assurance Assessment or AQAA, which is a provider self-completion tool). The empirical work will be carried out for the care homes study where routine quality data are available. Following phase 2 piloting, the main fieldwork in phase 3 will involve assessing the functioning states of a sample of service users in care homes. This surveying will be timed to correspond with a regulatory inspection of the home to give us both capacity for benefit outcome scores (from the interview) and the home’s quality rating (from the inspection). The analysis will then involve determining the statistical correlation between the measures as outlined. One of the key tasks (in phases 2 and 3) will be to develop observational 41
techniques for determining the functioning states of people with profound cognitive or communication impairment and who are therefore unable themselves to give an account of their current functionings state or those absent services. The second objective is to use the information about the outcomes of services to test the hypothesis that outcome-based commissioning will bring significant benefits, in some combination of cashable savings on public spending and/or improvements in people’s (objective) valuation of the outcomes of services. The relevant theory is outlined in section 2.5.2. This objective mainly concerns the low-level services project. Following the developmental work in phase 2, the intention is to use the phase 3 fieldwork to determine the average outcome gain delivered by a subset of low-level services (including day care and meal services – see below). Methods will include using an interview (to ascertain outcomes) and a self-completion questionnaire that can be organised either as a postal survey or a postal survey with a telephone interview follow-up (i.e. people are posted the questionnaire and supporting material in advance and then interviewed by telephone). Our intention is to use the capacity for benefit approach where the questionnaire seeks to obtain both current outcomes and also outcomes without the service in order to find outcome gain as the difference. We may also consider the need to do case-control surveys. Outcome gain from these services will be stratified into groups that vary with the dependency level of service users (data from the questionnaire). Unit costs of these services can be ascertained from financial returns. In this way, the aim is to derive cost-effectiveness ratios for each service, and each dependency group. We will use data from the Wanless Review regarding the distribution of dependency in England. Also, we will use DH and other data to determine the current level of service provision. This will be compared with the ‘ideal’ level implied by the cost-effectiveness analysis and the numbers of older people in the population with different levels of dependency. The third objective is to test the feasibility of an outcomes approach for information and advice services. This entails at least three challenges: to be able to define the functioning states of these services; to find a way to specify different levels of achievement of these functioning states; and to be able to attribute functioning states to the use of these services. These are requirements for being able to determine the value in National Accounts of information and advice services and/or use functioning states-based commissioning of these services.
3.2
Care Homes Project
3.2.1 Background Two million people of all ages from every community used social care services arranged by local councils during 2005-06 with the aim of assisting them to live independently and to have a decent quality of life (Commission for Social Care Inspection 2006). Expenditure on social care for all clients by councils increased to over £19 billion in 2004-5; the upward trend in expenditure continued in 2005-6. Expenditure on adult’s services increased by over 4.0 per cent in real terms to just over £14 billion (Commission for Social Care Inspection 2006). These services were either provided by the council or by private and voluntary organisations commissioned by the council. In total, as of 31st March 2006, 71.8 per cent of all homes were in the independent sector with 19.0 per cent of homes in the voluntary sector. Almost 25 per cent of homes for younger adults are in the voluntary sector compared to just over 13 per cent of homes for older people (Commission for Social Care Inspection 2006). People assessed by their council contributed around £2 billion in charges for residential and community services. In addition more than 174,000 people paid directly for their place in a care home and many other people arranged their own care and brought services directly from private and voluntary organisations (Commission for Social Care Inspection 2006). In 2004-05, the number of people supported by councils to live in residential care decreased
42
from 277,950 in 2003-04 to 267,240 in 2004-5 (Commission for Social Care Inspection 2006). 3.2.1.1 Care homes In the UK, the generic term ‘care home’ encompasses the provision of two types of care: nursing care and/or personal care (Goodman and Woolley 2004; Froggatt and Payne 2006). Care homes are residential facilities in communities where groups of residents are cared for in a supervised environment under a common roof. In the UK, they include ‘nursing’ homes (where there is a 24-hour presence of qualified nurses), ‘residential’ homes (where there is 24-hour presence of staff, but not necessarily qualified nurse/s) and ‘dual’ registered homes (where both ‘residential’ and ‘nursing’ facilities exist on the same campus). Care homes differ from ‘sheltered housing’ or ‘warden controlled flats’ where residents live in their own flats in the building and look after themselves but have access to wardens for help if needed. The warden may or may not be on site, especially at night time (Purandare, Burns et al. 2004, p549).
Most care homes in England (71 per cent) offer residential and personal care only and the independent sector is the main provider of 88 per cent of these homes (Goodman and Woolley 2004; Purandare, Burns et al. 2004). In 2001, there were around 176,000 registered nursing beds in general and mental nursing homes, and 341,000 places in residential care homes (Department of Health 2002). These are places for all client groups. For residential places, 237,000 were for older people. Around 15% of staffed places in residential care homes are provided by the voluntary sector. However, over the last five years, the proportion of general nursing homes has decreased from 79% to 73% (Department of Health 2002). Publicly-funded admissions to nursing and care homes have continued to decline reflecting government policy to support more people to live in their own homes. The numbers of people supported by councils to live in residential care decreased from 277,950 in 2003-4 to 267,240 in 2004-5 (Commission for Social Care Inspection 2006). As of 31st March 2006, there were 18,718 registered residential care homes for adults, with 441,335 places (Commission for Social Care Inspection 2006). Although the number of registered places has risen by 449, there are now 315 fewer homes compared to March 2005, continuing the decline in the number of residential services registered with CSCI (Commission for Social Care Inspection 2006). Of these homes, 4,058 were nursing homes and 14,660 were care homes proving personal care only. This translates into 177,021 beds in nursing homes and 264,314 beds in care homes (Commission for Social Care Inspection 2006). 3.2.1.2 Older people Critics have argued that care homes are the place of abode until death for many older people (Froggatt and Payne 2006, p341). As of the 31st March 2006, there were 12,215 care homes with at least one bed registered for people over the age of 65. There were 169,919 places in homes that have at least one bed for older people with dementia, up from 161, 713 at 31st March 2005 (Commission for Social Care Inspection 2006). As of 31st March 2006, the average size of care homes that are registered for people over the age of 65 stood at 34 places (CSCI, 2006).
Although relatively few people (5 per cent) aged sixty-five or over live permanently in a care home setting at any given time, this is the situation for approximately one in five of those aged eighty-five and over (Wittenberg, Pickard et al. 1998). Furthermore, the lifetime risk of being a care home resident (i.e. going in to a care home at some later age) is much higher: at aged sixty-five the likelihood is one in five for a man and one in three for a woman (Bebbington, Brown et al. 1998).
43
In 2001, there were approximately 517,500 residential and nursing home places for older people in England (Department of Health 2002). However, as a result of the continued loss of capacity of care homes, in 2004, for older people, there were 486,000 places in private, voluntary and public care homes (Laing & Buisson 2004). Care homes settings are important places when considering where older people die (Froggatt and Payne 2006). 3.2.1.3 Learning disabilities National estimates are that there are 210,000 people with severe or profound learning disabilities and a prevalence rate of around twenty-five people per 1,000 populations with mild/moderate learning disabilities in England (Secretary of State for Health 2001). Since the 1970s, the social policy agenda has been to reduce the number of places in hospital and increase provision in the community both by supporting family carers and encouraging the development of small care homes with an average care home size of 6 places (Department of Health 2001).
In 2003/2004, from a sample of 2,898 people with learning difficulties who were at least 16 years old across the UK, just under one in three people (31 per cent) with learning disabilities were living in some kind of supported accommodation. Three in four people (75 per cent) were living in houses for four or more people. Fewer than one in four (22 per cent) were living in houses for more than 10 people (Department of Health 2007). Recent figures show an estimate of 36,320 council supported adults with learning disabilities in residential care in England (Department of Health 2001). At 31st March 2006, there were 57,587 places in homes registered to care for younger adults with learning disabilities. The average size of residential care homes for learning disabled adults stood at nine places (CSCI, 2006). 3.2.2
Empirical methodology
3.2.2.1 What services? The care homes project will work with care homes in England that are registered for older people and for people with learning disabilities (PWLD). Our focus on care homes for PWLD and OP reflects the academic expertise involved with this project: PWLD (Tizard centre), OP (PSSRU) and the overall objective to measure the outputs of social care services in a way that reflects quality and functioning states so that the methodology is transferable to different client groups. 3.2.2.2 Proposed instrumentation We are developing a toolkit of measures in three groups that: (a) measure need and dependency (b) measure improvement in functionings state, including capacity for benefit (c) measures of quality of service. (a) Measures of dependency A series of rating scales will be used to assess service users’ level of dependency and client characteristics such as ability, social impairment and challenging behaviour of the people in each home. These measures will be sent to the service manager in advance of visits by the researchers and completed by the key worker for each person. These measures will only be requested from the sample of the residents. For PWLD homes, measures of dependency will include the Short User Survey-SUS 10 . The SUS includes the Short Adaptive Behaviour Scale (SABS) (Hatton, Emerson et al. 2001) the Quality of Social Interaction question originally taken from the Schedule of Handicaps, Behaviours and Skills (Wing and Gould 1978) and the Aberrant Behaviour Checklist (ABC; (Aman, Burrow et al. 1995)) and a series of 10
See Mansell & Beadle-Brown, Tizard, University of Kent
44
communication questions. For older adult care homes, measures of dependency will include a number of Activity of Daily Living (ADL) scales such as the Barthel index. (b) Measures of functioning states The capacity for benefit (CfB) instrumentation will involve a questionnaire that asks people – that are able to respond – about their levels of outcome currently in the care home and also their expected functionings state if they were not receiving care. Staff will also be asked for their views as to the functionings state a resident might achieve with and without services.
Regarding people that are unable to respond we will use observational techniques to estimate current functionings state for residents. These will be adapted from two existing measures. First, the Active Support Measure (Mansell and Elliott 1996), which rates the nature and quality of staff support for each service user using a fifteen-item rating scale at the end of a two-hour observation period. Second, Engagement in meaningful activity and relationship (EMACR), which seeks to provide a momentary time sample observation of the activity of people with learning disabilities. The purpose of this measure is to provide an accurate estimate of how much time people spend in meaningful activities and relationships. This is accomplished by spending time with each individual concerned, noting at precise intervals what they are doing and what is happening to them. Interviews with staff (or other carers) will also be used to identify resident needs in the absence of services in our domains of outcome. (c) Measures of quality of service In order to target quality of services, we will potentially include a series of rating scales, administered via structured interview and two observational approaches. These measures should relate closely to the outcome measures described above since the quality and intensity of services should together generate functioning states for service users.
We will use the (un-adapted) observation measures noted above. Furthermore, there are four rating scales that are determined in structured interviews: •
•
• •
The Index of Participation in Daily Living (Raynes, Wright et al. 1994). Designed to identify the extent to which service users are given opportunities to participate in everyday domestic tasks. It contains thirteen items; each rated on a three-point scale and is completed for each service user by a member of staff during a structured interview. The Residential Services Working Practices Scale (Lowe, Felce, Perry, Baxter, & Jones, 1998). Developed to reflect the presence of organisational procedures concerned with individual planning, assessment and teaching, planning daily and weekly activities, supporting service user activity and staff training and supervision. For the purposes of our project, specific questions on physical settings will be extracted and used as part of our tool kit. Index of Community Involvement-ICI (Raynes, Wright et al. 1994). The ICI is used to assess the extent of resident involvement in social and community activities. Choice Making Scale-CMS (Conroy and Feinstein 1986). The CMS is designed to assess the extent to which service users were encouraged and helped to make choices in their everyday lives.
3.2.2.3 Fieldwork design and specific aims The empirical work (phases 2 and 3) of the care homes project is divided into four parts. Part A. Initial instrument design and sample frame development
45
The instrumentation is outlined above. The main task will be to adapt the capacity for benefit tool for people that are unable to be interviewed. The observational techniques that are currently available will be adapted so that functioning states in the CfB domains are inferred. Furthermore, since these techniques were developed for PWLD they will need to be modified also for older people (with cognitive impairment). Part B. Exploratory work with care homes providers For the pilot stage, the Commission for Social Care Inspection (CSCI) will select a sample of six care homes; three for people with learning disabilities (PWLD) and three for older adults (OA). Care homes will be selected from three regions of the UK; our initial regions of interest are: London/South East (counting as one region), South West and the North West.
Through consultation with CSCI, our intention for sampling is to include homes from a range of quality ratings (‘excellent’ to ‘poor’) in order to ensure a representative portrayal of care homes. A further focus would be for CSCI to select homes with a range of cultural backgrounds; in order to target the issue of cultural diversity in consideration of concepts such as quality of care and quality of life. An additional sampling issue will be to select homes with pending inspections. The intention is to collect data from care homes two weeks before an inspection takes place. The exploratory work is to be undertaken to test the instrumentation and particularly the feasibility and quality of the results from the observational study. Initial analysis will look at the relationship between our toolkit measures and CSCI assessments: home quality ratings and quality measures in the KLORA and AQAAs. If necessary, for the pilot, if there are significant amendments to the piloting procedure, further piloting of the finalised process may be necessary in two homes.
46
Box 1. Care home protocol
1)
2)
3)
CSCI and PSSRU/Tizard to approach providers of care homes to obtain consent for participation in our project. CSCI to contact PSSRU on care homes, which have agreed to take part. a) We will be stressing the benefits of this study to both CSCI and for care homes. For example, this project has the potential to validate the association between inspector’s reports and observed functioning states for residents in care homes. Furthermore, we will be looking at cultural aspects of quality and outcome for residents and these could develop and contribute to the regulatory process. PSSRU to send ‘information packs’ to care homes to distribute to service users or residents of the study two weeks prior to any data collection/fieldwork taking place. In order to comply with the Mental Capacity Act, PSSRU will also contact the care home manager to confirm and discuss whether this project would be in the best interests of the care home residents. Care home managers will be asked to select a sample of residents with a range of needs and characteristics. Care home staff will be instructed to collect and keep in a secure location, parts of the information packs (consent forms and questionnaires) for the PSSRU researchers to collect on fieldwork dates. Packs sent to the six care homes will include the following: a) Information sheets: to inform potential participants about the purpose of the study and to clarify what is involved (for the participant) in taking part. Participants will consist of residents and staff members and/or key workers. The information sheets will explicitly outline that all information will be treated as confidential, will be stored in a secure location at PSSRU and all participants will remain anonymous. b) Consent forms: Two weeks prior to data collection potential participants will receive a consent form to enable residents to give informed consent regarding whether or not they wish to take part in the study. If a resident is unable to give informed consent, a key worker or care home manager will complete a consent form by proxy. c) Questionnaires on client characteristics: The client characteristic questionnaires are to be completed by key workers of each resident. These questionnaires will attempt to illustrate and describe the residents’ abilities, characteristics and needs of both PWLD and OA. Basic ADL and cognitive ability data will be collected for all, or in very large care homes a substantial sample of residents. These questionnaires will provide an overall picture of the dependency profile of each resident within our sample. In dealing with both PWLD and OA, residents will have the right to withdraw their consent throughout, either through verbal or non-verbal means; such as challenging behaviours in response to the presence of observers. All information obtained from interviews and observations will remain anonymous and confidential through usage of numerical coding. Furthermore, fieldwork will only be carried out following consent from residents and staff members. Fieldwork will only take place in public settings such as living rooms.
Part C: Main fieldwork The main stage will involve CSCI selecting a sample of 200 care homes (100 for OP and 100 for PWLD). This will be a stratified sample, but where the sampling probability within each category is set to reflect the relevant national average. Any purposive over-sampling will be
47
weighted to the national average. 11 The sampling frame will be all care homes eligible for inspection by CSCI in the relevant CSCI region. . Study protocol details are in Box 1. The objective of the fieldwork is to collect information relating to quality and functioning states (CfB) using the toolkit described above i.e. via observation and structured interviews. Our objective will be to establish an insight into resident’s interactions and surroundings using modified standardised instruments as well as establishing an insight into aspects of process quality such as the level of dignity and respect for residents by staff. Furthermore, the observational and interview data will be used to identify both the degree to which needs are currently being met and expected needs in the absence of the care provided. The usage of both modified observational and structured interviews poses a relatively new challenge in estimating functioning states and quality with PWLD and OA populations. The intention is to complete fieldwork for each care home across two days. • The objective of day one will be to implement our observational measures. Day one will consist of meeting with the care homes manager to discuss the aims of our project and to confirm that the study is still within the best interests of the consenting residents. The dependency data will also be collected for a sample of five/six residents for whom much more detailed information will be collected (via observation). In the case for PWLD, we may sample all residents that consent rather than limit to a sample of five or six residents. In order to improve the prospect of residents being within the same place, observations will take place around the evening meal time (16.00-18.00). Day one will conclude with a consultation with the care homes manager regarding any feedback on the process and whether residents have commitments, which may affect day two. • The objective of day two will be to implement our self report questionnaires via structured interviews with staff members to obtain further information on the resident’s service user’s outcome, such as participation in tasks, option to make choices and community involvement. Where possible, we intend to interview residents regarding our domains. Day two will begin with a brief meeting with the care homes manager to discuss the objectives for day two and to check that the sample of residents is available. Day two will conclude with a final meeting with the care homes manager to discuss the project implementation and to discuss any initial feedback. Part D: Analysis and reporting The aims of the analysis were outlined above (section 3.1). The main analysis will be to examine the correlation of measures within the toolkit and CSCI measures. We will also undertake thematic analysis of any cultural issues derived from interviews. Furthermore, we will assess the overall fitness for purpose of the toolkit, particularly in relation to very highly dependent service users.
Outputs will include reports of this analysis (incl. strengths and weaknesses of the toolkit; refinements required etc.). We will draw out implications of the findings in relation to current CSCI regulation processes, and finally, we will use the results to feed into the National Accounts. 3.2.2.4 Research timings and protocol Table 1 gives details of the timings of the main parts of the work. The empirical work started in July 2007 and will last approximately 19 months. The timing of the project is dependent on 11
There will be some limitations on the representativeness of the sample in order to ensure that the analysis can pick up on the full range of home and resident circumstances. For example, only 5 per cent of homes are rated as ‘excellent’ by CSCI. For our analysis to be valid we need at least 10 per cent of our sample to be in such homes if our metric is to adequately reflect the higher quality of life experienced by residents in such homes. We also hope to adequately reflect the experience of BME residents and this may require over sampling of certain types of home.
48
the timing of the (significant) inputs required from CSCI. Changing inspection processes and re-organisation at CSCI mean that the main fieldwork is not anticipated to start before April 2008. This will have knock on effects for the finishing time of the project.
49
Table 1. Timings for care homes project Task
Timing
Part A. Initial instrument design and sample frame development 1) Instrument design and adaptation
2 months
Part B. Exploratory work with providers 1) Sample selection (CSCI) 2) Testing of instrumentation
5 months
Part C. Fieldwork for testing and evaluation of toolkit 1) Sample selection 2) On-site working with homes 3) Data entry and cleaning
7 months
Part D. Analysis and reporting 1) Analysis 2) Report writing and dissemination
6 months
Total time
19 months
3.3
Low Level Interventions Project
3.3.1 Background It is now widely accepted that, over the past decade, limited statutory community care resources have been increasingly targeted on high level need – often crisis interventions – at the expense of prioritising lower level, preventative services (Audit Commission 1998; Joseph Rowntree Foundation 1999). It is also generally accepted that changes in community care policy, an ageing population and wider socioeconomic forces has led to an increasing proportion of people with support needs living in the ‘community’, sometimes alone and unsupported (Quilgars 2000).
A number of recent initiatives and policy developments have begun to reassert the importance of the value of low-level support services. Most recently, the Social Exclusion Unit report ‘A Sure Start to Later Life’ (2006) centres around a primary focus of preventing a cycle of decline and promoting a cycle of wellbeing. The publication of the White Paper ‘Our Health, Our care, Our Say’ focused attention on enabling health, independence and wellbeing, encompassing a shift towards a more preventative system. It is structured around an underlying theme of prevention which is intrinsic to the Government’s commitment to modernising social care. The White Paper (2006, p142) states that ‘an increased commitment to spending on prevention should be part of the shift in resources from secondary to primary and community care. The UK spend on prevention and public health is relatively low compared to that of other advanced economies.’ The White Paper also incorporates safeguards to ensure that healthcare services adopt a preventative approach. It introduces performance measures and includes targets to be met by 2008 which reflect the new focus on prevention and the promotion of wellbeing. 3.3.1.1 Low Level Interventions – definition Low level interventions as a concept generally refers to upstream interventions which seek to help people maintain or improve health before it is compromised. The implication is that if we can maintain good health, through the means of prevention, then the need for more costly services will be reduced or delayed or, in some cases, even prevented (Office of the Deputy
50
Prime Minister 2006). In general, low-level or low-intensity interventions are those services or initiatives that require minimal resource input in terms of working hours and do not necessarily require the input of specialist professionals (Curry 2006). The underlying rationale for low-level interventions is essentially the same as with all preventive care – that investment now will yield future cost savings. It has frequently been preventive services (particularly low-level interventions) that have been squeezed as resources have been moved to focus on acute, high need cases (Godfrey 1999). It could be argued that this is a false economy, as individuals who require just a low level of assistance to live independently would, without provision of this assistance, more quickly require high intensity and high cost care. Intervening early, or in a timely manner, is intended to delay, and even reduce the intensity of this need. However, installing a hand rail in someone’s home or providing them with help to go shopping twice a week requires financial commitment, so it is necessary to prove that, ultimately, the long-term economics of shifting resources to this end of the care spectrum are robust. The challenge, clearly, is establishing the link between the implementation of a given intervention and the outcome achieved. Hitherto standard measures for functioning states have not been developed. It is argued by a number of commentators (Billis and Glennerster 1998; Clark, Dyer et al. 1998; Godfrey 1999; Quilgars 2000; Elkan, Kendrick et al. 2001; Wistow, Waddington et al. 2003) that a low-level of assistance in such areas of everyday life can enhance quality of life through enabling an older person to remain in their own home, maintain independence and reduce the risk of institutionalisation. Godfrey, for instance, argues that relatively minor alterations and help can be the difference between someone living independently in the community and being admitted to hospital or a care home and, as such, are critical to maintaining quality of life (Godfrey 1999). Although it is generally recognised that such lowlevel services, alone, cannot prevent ultimate deterioration in health, they may be able to delay this deterioration and thus delay admission to a care home (Audit Commission 2004). 3.3.1.2 Current range / mix of services A broad range of services and initiatives may be considered ‘low-level’ although no standard definition appears to exist. Examples of such services that might be classed as low-level include help with those tasks that people find difficult as they get older, such as gardening, laundry, cleaning and shopping (Clark, Dyer et al. 1998). Another tier of ‘low-level’ interventions includes home adaptations, such as the installation of handrails and ramps.
One of the most comprehensive illustrations of what constitutes a low-level intervention is provided by the Social Exclusion Unit (Social Exclusion Unit 2005) adapted from the Joseph Rowntree Foundation (2003). This model distinguishes between physical and practical lowlevel services in both the home and the external environment, and personal and social lowlevel services in the home and the external environment (see Figure 1).
51
Figure 4. Low-level services
Source: Social Exclusion Unit, 2005
3.3.1.3 Perceived benefits of low level interventions There is something of a dearth of evidence regarding the cost-effectiveness of low-level interventions, particularly studies that quantify the impact of low-level interventions in terms of the functioning states they hope to achieve. Despite this lack of quantitative evidence, there is a bank of qualitative work that endorses the value of low-level interventions, particularly for older people. In ‘That bit of help’ Clark et al. (1998) argue that low-level interventions are key to maintaining independence, avoiding institutionalisation and reducing isolation.
Research conducted on behalf of the Joseph Rowntree Foundation’s Older People’s Inquiry (Joseph Rowntree Foundation 2005) identified a ‘baker’s dozen’ of low level supports that older people valued because they enabled them to stay in their own homes. These initiatives, such as a ‘handy help’ - a scheme for small repairs around the house, ‘welcome home’ – a scheme for those returning from hospital, and ‘sole mates’ – which provides a foot bath and
52
toe nail clipping service, are all held up as examples of services that improve the quality of life of older people and help maintain their independence. The research also identified a number of themes that older people considered important in terms of maintaining independence, these were: • comfortable and secure homes; • an adequate income; • safe neighbourhoods; • the ability to get out and about; • friendships; • learning and leisure; • keeping active and healthy; • good, relevant information. In addition to practical help that can be provided by low-level interventions, The House of Lords Select Committee on Science and Technology (House of Lords Select Committee on Science and Technology 2005, p53) emphasise the importance of services and initiatives that enhance mental health and general wellbeing. They found that ‘inactivity and isolation accelerate physical and psychological decline, creating a negative spiral towards premature, preventable ill health and dependency.’ Thus, initiatives such as befriending schemes that enable older people to maintain control, dignity and independence and, in doing so, reduce or delay the need for high intensity health and social care services (Clark, Dyer et al. 1998) are important aspects of maintaining a sense of social inclusion and good mental health. Work by Layard (Layard 2005) also emphasises the importance of happiness in maintaining an effective economic, health and social care system. The cost effectiveness of services that promote happiness, independence and general wellbeing is difficult to establish. However, there has been some attempt to quantify isolated schemes. One intervention that has been widely implemented across the country is the Sloppy Slippers Campaign which aims to highlight the risks of ill-fitting slippers and encourages older people to exchange old, ill-fitting slippers for new ones that fit. The basis for this scheme is that, of the 300,000 older people who go to hospital with serious injuries from falling, around 9 per cent blame their slippers (Department of Health 2003). It is estimated that the Sloppy Slippers Campaign reduced falls by 32 per cent in the first year and 37 per cent in the second year. If this were rolled out across the country, it is estimated that some £500 million could be saved in terms of reduced falls and the resulting treatment required (Office of the Deputy Prime Minister 2006). 3.3.1.4 National picture – services use The Referrals, Assessments and Packages of Care (RAP) provide national statistics on adult community care. Data collected by The Information Centre (2007) show the following in terms of referrals, assessments and packages of care.
Referrals • An estimated 2 million contacts from new clients were made to Councils with Social Services Responsibilities in England in 2005-06, a rise of 79,000 or 4 per cent from 2004-05. • Around 1 million of these contacts (51%) resulted in further assessment of need or the commissioning of ongoing service and 996,000 contacts were attended to solely at or near the point of contact. This compares to 945,000 contacts attended to solely at or near the point of contact in 2004-05. • Of the 2 million referrals, 592,000 (29%) were self-referrals, 495,000 (24%) were referred from Secondary Health sources (e.g. hospital wards), and 281,000 (14%) were referred by family, friends or neighbours. There were 264,000 (13%) referrals from Primary/Community Health.
53
Assessments • An estimated 651,000 thousand new clients had their first assessment completed during 2005-06, an increase of 0.3 per cent from 2004-05. • In respect of waiting times for new clients aged 65 and over, about 29% of all new older clients had their assessment completed within 2 days of first contact and 59 per cent were assessed within 2 weeks. This is an increase on the 2004-05 figure of 55 per cent complete within 2 weeks, but is below the ministerial target of 70 percent. Three quarters (75%) of all assessments for new older clients were completed within 4 weeks compared to the ministerial target of 100% by December 2004. • Around 1.2 million reviews for existing clients were carried out in 2005-06, a rise of 7 per cent from a year ago and 14 per cent since 2003-04. Packages of care • During 2005-06, an estimated 1.75 million clients received services provided, purchased, or supported by Councils with Social Services Responsibilities following a community care assessment, a rise of 2 per cent since 2004-05. • Community-based services were provided to about 1.49 million clients during the year, accounting for 85% of all clients receiving services. • In 2005-06 an estimated 596,000 clients received home care, 499,000 clients received equipment and adaptations, 444,000 received professional support (e.g. occupational therapy) and 244,000 received day care as a service following assessment. Clients receiving more than one type of community-based service are included for each service received. • Some 136,000 older people received day care services and 155,000 older people received meal services. 387,000 older people were given equipment or adaptations. • 37,000 adults aged 18 and over received direct payments during the year, increasing from 24,000 in 2004-05, a rise of over 50 per cent. • An estimated 1.23 million, about 70%, of those receiving services as part of a package of care following an assessment, were aged 65 and over. Table 2 lists the total council spending on services for older people in England. The low level services (other than home care), including day care, meals, equipment and supporting people, account for over two-thirds of a billion pounds or 8% of all services expenditure. The table includes council contract funded services. Low level services funded from other sources would add to this total. Table 2. Council Personal Social Services Gross Expenditure, 2005/2006 – services for older people, England Item Assessment and care management Nursing home placements Residential care home placements Supported and other accommodation Direct payments Home care Day care Equipment and adaptations Meals Other services to older people Supporting People TOTAL
Expenditure £ 912,458,000 £ 1,490,802,000 £ 3,107,716,000 £ 40,665,000 £ 67,757,000 £ 1,856,901,000 £ 341,907,000 £ 100,186,000 £ 94,860,000 £ 225,203,000 £ 154,482,000 £ 8,392,935,000
Source: PSSEX1
54
3.3.2 Empirical methodology As with the care home project, the low-level services project has a piloting phase, a main fieldwork phase and an analysis phase.
3.3.2.1 What services We plan to focus on two groups of low-level services: day care services for older people and home-based services, including meal services and (possibly) supporting people services. These services respectively fall into the home and external environments category of the social exclusion unit ‘personal and social’ definition (see Figure 4). Also, we are exploring doing work on equipment and adaptations services in people’s own home (in the ‘practical’ definition by the Social Exclusion Unit (SEU)). Day care is chosen for a number of reasons. First, the voluntary sector plays a significant role in the provision of day care services. Second, some commentators regard this as an under-rated service (Wanless 2006). Third, it involves a variety of activities and caters for a range of people. Fourth, day care is valued by carers (for example in offering carer respite). Fifth, as outlined above, year-on-year data are available (by council) concerning the use of day care by older people. Meal services are similarly provided in large part by voluntary organisation, are highly valued and data are available. Supporting people services inhabit the interface between care and housing services. Warden services in sheltered housing or extra-care housing are good examples. The feasibility of covering supporting people services will depend on the adequacy of public data collections about these services. Equipment services are highly valued because they appear effective at helping older people to stay independent. They too are provided in large measure by the third sector. 3.3.2.2 Proposed instrumentation Although subject to revision, at this stage we plan to use a structured questionnaire of service users to ascertain functioning states. The main measure will be capacity for benefit. Therefore the questionnaire will ask service users to rate their outcome in a number of domains. To establish a baseline, respondents will also be asked about their expected achievement of functionings state in the absence of services. This questionnaire could also include other measures, such as experienced utility (see section 2.1.3 above).
In addition, the questionnaire will include some process indicators or quality measures (see section 2.3). These will be service specific by design. Potentially we could also include global happiness and satisfaction questions. The questionnaire will also cover details of service use and user characteristics (in particular, dependency information). This questionnaire is designed to be administered in an interview. We will also develop a self-completion version of the main functioning states questionnaire, which could be administered in a postal survey with/without telephone follow-up. The selfcompletion version will cover the main functioning states metrics as above but will take a streamlined or abridged form. During the pilot we anticipate also using semi-structured interviews and focus groups with provider staff in order to capture their views, particularly to feed into the design of the main service user questionnaire. 3.3.2.3 Fieldwork design and specific aims The empirical work (phases 2 and 3) of the low level services project is divided into four parts. Part A. Initial instrument design and sample frame development There are two main tasks. Following the conceptual work and drawing on previous work e.g. the analysis of home care services for development of the Relative Needs Formula (Darton
55
R, Forder J et al. 2006) a draft outcome measure and questionnaire will be developed (see above). A self-completion version will also be drafted, as will a protocol for focus group analysis. This part of the work will also involve the scoping of the nature and extent of low-level services. The aim will be to identify data sources giving a picture of the extent of different types of low-level services. We will approach providers and commissioners of these services to improve our understanding of this sector, including voluntary organisations (e.g. Age Concern). This activity will be preliminary work in developing a sample frame. We require a suitably representative database of provider organisations from which to sample for the main fieldwork phase. This database can be populated using information from local authorities, the Department of Health, private directories and national provider associations. In this part of the work we will develop the ethics application. Part B. Exploratory work with Low Level Service Providers (LLSPs). We will approach 2 day care LLSPs and 1 or 2 home-based/equipment LLSP in the third sector and these will be drawn from a purposive sample. This is our pilot sample. The intention is to undertake the following: • Consultation with service users (using focus groups) • Focus groups with stakeholders in relation to LLSP (e.g. commissioners, managers, staff and other local stakeholders). We will conduct (unstructured) focus groups from all selected LLSPs to explore/determine (a) what these services are (generally) doing for users; (b) what are the key aspects of quality facilitated by LLSPs that enable/undermine functioning states and quality of life, (c) whether providers and other stakeholders are already measuring their own quality. • Often these types of schemes have volunteers who would also have a helpful perspective. • We will then conduct interviews with service users using the draft functioning states questionnaire (see above). At this stage will we undertake cognitive testing with respondents to test understanding of questions. We will initially approach 10 service users in each provider. This will be an iterative process with refinement of questions.
We will also test the feasibility of using a self-completion method with a subset of service users for these services. We aim to use unit cost information in the analysis. Routine unit cost information from commissioners (councils) will be used where available. Otherwise, we will ask providers for this information. In both cases we will need to test the feasibility of this approach (particularly in the context of constrained project resources). The aim of this work is in determining: • Whether we have the right outcome domains • How we rate different levels of functioning states within each domain • People’s understanding of the questions in terms of whether the answers they elicit are consistent with our theoretical concepts regarding functioning states The outputs from this phase will be a piloted questionnaire (interview and self-completion). Part C. Fieldwork for testing and evaluation of toolkit Using the sample frame developed in Part A, we will construct a stratified sample of 30-50 LLSPs (day care providers, home-based providers plus others (e.g. community services/ advocacy etc) to provide a basis for the main testing and evaluation of the functioning states toolkit. There are no existing national registers of low-level providers and therefore we will construct a sampling frame by combining lists of providers used by councils (where councils
56
are happy to provide this information) and those that are covered by national umbrella provider organisations (e.g. Age Concern). Although our sampling methodology is in development, the aim is to select providers randomly from within our defined categories of provider type and within the regions of the country in which we will operate, using the above lists. We will recruit five or more service users from each LLSP (using replacement sample). They will take part in the main interview with the full toolkit. A sub-sample of these interviewees and also a group of additional service users from each provider will be asked to complete the self-completion toolkit (SCT), which will generally occur in advance of the interview. The intention is to complete at least 250 full interviews and 250 self-completed toolkit questionnaires. We will aim to have sufficient overlap in these groups so that the SCT can be validated against the responses to the full toolkit. The pilot phase of the work will give us an indication of interviewee burden, and we will alter the design at this stage accordingly. The protocol outlined in Box 2 gives details about the recruitment process for interviewees. A further 250 self-completion questionnaires will be sent out to service users of between 30-50 other providers (of a mix of types). A telephone interview follow-up will used to improve response rates. We will undertake psychometric testing of the interview plus self-completion responses. These tests establish statistically the validity and reliability of the outcomes scale being used (i.e. the capacity for benefit measure). Box 2. Recruitment protocol
1)
2)
3)
PSSRU to send ‘information packs’ to LLSPs to distribute to service users / potential participants of the study two weeks prior to any data collection/fieldwork taking place. Packs to include the following: i) Participant Information Sheets (PIS) to inform potential participants about the purpose of the study and to clarify what is involved (for the participant) in taking part. ii) Consent forms. Prior to data collection (interviews) taking place, potential participants will receive a PIS and consent form to enable service users to give informed consent regarding whether or not they wish to take part in the study. iii) Those who consent to take part in the study to take part in a structured interview (CfB and other measurement tools applied) at an agreed time/day located at LLSP premises or the person’s own home if they wish. Service users of the home-based service (e.g. meals) will receive information packs from the LLSP together with the normal service they consume. LLSP staff in contact with service users (and distributing the information packs to service users) will be briefed by PSSRU researchers regarding the purpose of the study and what is involved for service users who agree to take part in order that they can answer any initial questions potential service users may have. i) Consent forms for home-based LLS users will include a ‘consent to be contacted’ section in which service users agree to being contacted in the future by a researcher. ii) The researcher will make contact with the (home-based) participant to arrange a suitable time to conduct an interview, most likely to be conducted at the participants’ homes or at the day care centre if they also receive that service. As part of the consent procedure, people will asked to complete the self-completion questionnaire.
57
Data from these samples will be cleaned and entered for analysis. The aim of this part of the work is to determine outcomes for service user respondents and to use this information to comment on outcomes at the provider level. We will also collect information about provider and service user characteristics that will allow statistical analysis of outcome gain for sub-groups. Finally, we will aim to develop a meaningful set of process quality indicators. Part D. Analysis and reporting The aims of the analysis were outlined above (section 3.1). At this stage, we have some preliminary plans as to the main analyses required. First, in examining differences in measures of the toolkit: • Correlation of measures within the full toolkit and also between the full- and selfcompletion toolkits. Focus on similar ‘types’ of LLSPs (i.e. all LLSPs providing physical and practical home help). (To include costs of services) • Correlation of measures and results of different ‘types’ of LLSPs (i.e. comparision of LLSPs providing physical and practical home help with LLSPs that provide personal and social help). (To include costs of services) • Correlation of toolkit with existing outcome measuring tools to test performance of toolkit • Correlation of measures to occur (a) after piloting, (b) in main fieldwork phase after interviews
Second, to inform the main objectives of the empirical work, we will determine: • Output gain on standard metric scale derived from the self-completion toolkit for 60-100 providers and up to 500 service users • Unit costs applied to services • Cost-effectiveness intervals derived for service types • Distribution of actual and potential service user by dependency level. These data will feed into the optimisation analysis to determine cost-effective service configuration. This configuration will be compared with the current configuration as far as this can be determined from the sample framework. The result will be an estimation of the outcome shortfall of the current configuration. Reported outputs will include: • Fitness for purpose of the toolkit o Strengths and weaknesses of toolkit o Refinements required o Scope • Using toolkit to work up commissioning approach • Using toolkit to inform national accounts 3.3.2.4 Research timings and protocol Table 3 gives details of the timings of the main parts of the work. This work starts in July 2007 and will continue to March/April 2009.
58
Table 3. Timings for low-level services project Task
Timing
Part A. Initial instrument design and sample frame development 2) Instrument design 3) Scoping and sample frame
2 months
Part B. Exploratory work with Low Level Service Providers (LLSPs). 3) Sample selection 4) Focus groups 5) Interviews 6) Cognitive testing
6 months
Part C. Fieldwork for testing and evaluation of toolkit 4) Sample selection 5) Interviews 6) Postal questionnaires and telephone interviews 7) Data entry and cleaning
7 months
Part D. Analysis and reporting 3) Analysis 4) Report writing and dissemination
6 months
Total time
21 months
3.4
Information and advice services
Before empirical work can be undertaken we need to develop the proposed framework outlined in section 2.3.1.1 above further. This will entail a search of the literature, further conceptual work, discussions with providers, service users, experts in the Information and Advice (I&A) area and, where these are identifiable, commissioners of these services. This will need to be carried on at a low level during the coming months, when most focus will be on the care homes and LLI projects. The main task of the empirical work will be developmental. We need to find empirical proxies for service specific measures. The aim is to ensure that empirical indicators are robust and fit-for-purpose i.e. capture the implications of information and advice services. We will also need to tie these indicators back into our main social care outcome metrics.
3.5
Preference study
One of the main tasks of the work will be to establish people’s relative preferences concerning relevant functioning states/domains. As outlined in section 2, preference weights are required in order to add up outcome gains in each domain to generate an overall figure. The preference study will begin once the service projects are at or near completion, at which time, the relevant functioning states will be established and specified. At this stage we aim to conduct a survey of the general population using various preference elicitation techniques – see section 2.2.2.
4 Timings and outputs 4.1
Timings
The planned timings for the whole project are given in Figure 5. As noted above, exact timings will depend on the engagement of stakeholders, especially CSCI and provider
59
organisations for the fieldwork stage. Nonetheless, both the care homes and low-level services projects build in time at the end in 2009 to accommodate longer fieldwork periods. Figure 5. Project timings 2009
WP1 Care homes Phase 1 (dev) Phase 2 (pilot) WP2 Low level interventions Phase 1 (dev) Phase 2 (pilot) WP3 Knowledge and information Phase 1 (dev) WP4 Preference study WP5 Consultation & dissemination
4.2
Dec
2008
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov
2007
Phase 3 (field workAnalysis & Report Phase 3 (field work) Analysis & Report Phase 2 Phase 3 (fieldwoAnalysis & Report Design & fieldwork Analysis Dissem. Final rep.
Outputs
The project will use five types of output. Project reports Following this interim report on the development phases of the work, the next set of reports from the project will be the three reports from the service studies: care homes, low-level services and information and advice services (see above timeframe). We also plan a report for the preference study. The project will also have a final report.
The aim is to write reports in a way that allows material to be drawn out for other output types as follows. Articles and papers We intend to publish various articles that focus on specific themes that emerge during the project. For example, we could publish a paper on some of the conceptual work of part 2 of this report. We will aim for a spread between academic journals, trade press and also research bulletins. Website The website is an importance basis for dissemination of the work. Currently we have pages hosted in the PSSRU website. Project outlines, summaries, reports and articles will be the main types of content. We will explore linking the website between project partners. The ONS project site is at: http://www.statistics.gov.uk/about/data/methodology/specific/PublicSector/output/qmf/default. asp Seminars/presentations We anticipate giving a number of presentations to a range of audiences including social care stakeholders, academics, government, and also stakeholder organisations and user groups. These will largely involve project partners, particularly, ONS. Briefings It will be important for us to use our good contacts in social care to arrange briefings for strategic leaders within social care, including DH (e.g. the Director General of Social Care and Strategy Unit head), HMT, Social Care Institute for Excellence (SCIE), Association of Directors of Adult Social Services (ADASS) etc.
5 Concluding points This report falls into two main parts. The first part reports on the conceptual work to date, reviewing the options and drawing out the implications. Concluding points in this regard are made in section 2.6. The work points to a number of options in terms of ways forward. For the care home project we will work with the core capacity for benefit approach modified for
60
observational data. For the low-level services project and the information and advice services project, we will need to decide which of the alternative ways to proceed we use. In this regard, we expect to test a number of the options in the pilot fieldwork phase. The emphasis in this report on conceptual development has been made to be clear about why and also in what form we measure functioning states. Before getting into the detail of how to measure functioning states in practice, it is important to understand the motivation for the task: that is, using functioning states information to inform public policy. For example, welfare economics, in highlighting how difficult it is to measure people’s utility, has shown that this is nonetheless unavoidable (despite all attempts thus far) in informing the allocation of (public) resources compared to other ways of informing these decisions that do not involve this difficult step. The second part sets out plans for the empirical work. It outlines the aims of the empirical component of the project, the hypothesis to be tested, the methodology to be used and the design of the empirical component as it currently stands. These plans stem from the conceptual work. It is worth noting that these are plans and, in the nature of empirical work, they may change as the work progresses. We are also clear that the conceptual and empirical development phases of the project are crucial. Getting the metrics and instrumentation right is vital to ensure that the main fieldwork phase progresses smoothly. The next steps for the project are to: • Care homes o work up the draft instrumentation o liaise with CSCI over pilot fieldwork o select pilot samples o begin piloting instrumentation • Low-level services: o finalise conceptual work and options o work up the draft instrumentation o finalise definitions and scope of low-level services o liaise with provider organisation and commissioners to define sample frame o select pilot samples o begin piloting instrumentation • Information and advice o continue conceptual modelling work and definition of functioning states for I & A services o consult with service users and stakeholders o plan the empirical phases of the work
6 Annexes Annex 1. Welfare economics and resource allocation in markets Suppose there is a society with m individuals, each with a utility function u over a bundle of n commodities x. A social welfare function (SWF) can be specified: (9)
W = W (u1 (x1,..., x n ),.., u m (x1,..., x n ))
A change in circumstances in the economy e.g. a change in technology through time leads to marginal changes in the commodity vector x. The implications at the margin for society are found by differentiating the SWF in (9):
61
(10)
dW =
n
∂W ∂u j dx ji j ∂x ji j =1 m
∑∑ ∂u i =1
Where commodities are traded in markets, then individuals are assumed to maximise their n
utility given prices and subject to a budget constraint:
∑p x i
ji
≤ y j , where yj is the income of
i =1
individual j. Following standard textbook results, the marginal utility of income (the Lagrangian multiplier), λ, multiplied by price equals the marginal change in utility from a change in consumption (also see (21) below): (11)
∂u j ∂x ji
= λ j pi
Substituting (11) into (10) above gives the effect on wellbeing of a change in services: (12)
dW =
n
m
∂W
∑∑ ∂u i =1 j =1
λ j pi dx ji
j
Relaxing the budget constraint, i.e. giving an individual more income, converts into utility for that individual at a rate given by λj (at the optimal x*): (13)
( ) = ∂v (p, y ) = λ
∂u j x *j
j
∂y j
j
∂y j
j
With an optimal distribution of income we would have
∂W ∂W ∂W λ1 = ... = λm = = K i.e. the ∂u1 ∂um ∂y j
last £1 allocated cannot produce a larger marginal improvement in welfare by being allocated to another individual (see Hansen, Hougaard et al. 2004). The assumption of an optimal distribution of income means that if all people’s utility was equally weighted i.e. ∂W ∂W = ... = , then marginal utilities of income must also be the constant λ1 = ... = λ m . ∂u1 1 ∂um This solution is then equivalent to the compensating principle solution. Given an optimal distribution of income, (12) becomes: (14)
dW = K
n
m
∑∑ i =1 j =1
m
where
∑ dx
ji
pi dx ji = K
n
m
n
j =1
i =1
∑ ∑ (dx ji ) = K ∑ pi dxi pi
i =1
= dxi . This is the change in welfare from the optimal service allocation and
j =1
optimal income distribution. Therefore, for a (small) discrete change in x, the change in welfare (in markets) approximates to the function in (1).
Annex 2. Resource allocation absence of market prices Suppose some estimate of marginal valuation υji is available where: υ ji μ j =
∂u j ∂x ji
. Here μj is
best thought of as a conversion factor from marginal valuation to marginal utility. To simplify
62
matters, suppose the estimation is on the same scale as marginal valuation in (11), so that μj = λj. In this case, (10) in Annex 1 is: (15)
dW =
n
m
∂W
∑∑ ∂u i =1 j =1
λ j υ ji dx ji
j
The impact of a change in services will depend on the relative weight of person j’s utility in ∂W the SWF. Suppose that all utility is equally weighted so that λ j = K for all j and that we ∂u j assume that the marginal utility estimates of each person j using services can be approximated by the average value, υji, then (15) is: (16)
dW =
n
m
∑∑
Kυ ji dx ji = K
i =1 j =1
n
m
∑ ∑ υi
i =1
dx ji = K
j =1
n
∑ υ dx i
i
i =1
in which case, a (small) change in x implies a change in welfare (not in markets) of: n ⎛ n ⎞ ΔW N = K ⎜⎜ υi xi1 − υi xi0 ⎟⎟ . In this case, where μj = λj, marginal value is equal to price i.e. i =1 ⎝ i =1 ⎠ υi = pi . In practice, the conversion factor μ may be on an arbitrary scale so that μj ≠ λj, but ( ∂W μj = K . this will drop out in the differencing to give (2) in the text, where ∂u j
∑
∑
Annex 3. Discrete choices and compensation Rather than when applied to indices of welfare change, as above, the same considerations apply for looking at discrete choices between two circumstances (for example, determining the well-being implications of a new service). Suppose the old and new service configurations are denoted x 0j and x1j . For the person to have the same utility before and after the change, a compensation payment of CV can be made, where CV is a change in the person’s income and is equal to: (17)
(
)
(
u j y j − CVj , x1j = u j y j , x 0j
)
The greater is the change in utility (with constant income) from the change in circumstances, the higher the value of CVj needs to be. The amount CV is the monetary equivalent of the utility change. If CVj is greater than zero then, x1j is preferred to x 0j , and vice versa. The size of CVj gives an indication of the strength of preference for the individual, but it is a function of the person’s marginal utility of income (13). This calculation is the basis of the Hicks-Kaldor compensation principle. If a second person, k, faces a similar change in circumstances: uk y k − CVk , xk1 = uk y k , xk0 , but prefers x 0j to x1j ,
(
)
(
)
the change may still be beneficial if CVj + CVk > 0 even when CVk < 0. For this condition to work, however, the two people have the same (constant) marginal utility of income. This compensating variation (CV) is the basis for cost-benefit analysis that is the practical decision-making tool of welfare economics. The net CV = CV1 + … + CVm can be compared with the costs of services to determine whether or not the project should proceed. For example, comparing project 1 to project 0 gives CV and the difference in costs is ΔC . Project 1 is cost beneficial if CV − ΔC > 0 .
63
Annex 4. Extra-Welfarism: health and cost-effectiveness ratios Suppose that the decision maker (DM) has preferences as given by: (18)
U = U (hi (x1,.., x n ),.., hm (x1,.., xn ),G )
where h is the health status or wellbeing of person j and xi are care services. The decision maker has a budget B. A change in circumstances has implications for the change in wellbeing h. It also has implications for changes in consumption opportunities, with G a composite reflecting all non-care services. Setting up the problem in this way is highly consistent with use of a social welfare function in the welfarist approach. The budget constraint is equivalent to: (19)
B=
n
∑p x i
i
+G
i =1
The standard maximisation problem is: (20)
⎛ Z = U (h1 (x1,.., xn ),.., hm (x1,.., xn ),G ) + μ⎜⎜ B − ⎝
n
∑p x i
i =1
i
⎞ − G ⎟⎟ ⎠
∂U ∂h j = μp i , ∀i j ∂x i j =1 m
(21)
∑ ∂h
(22)
∂U =μ ∂G
These are conditions that need to hold at the margin for the (constrained) maximisation of the decision maker’s utility. Consider a (small) change in circumstances, such that care service inputs change by dx. With the DM working at the overall (economy wide) budget constraint, the utility implications of this change are: (23)
n
n ∂U ∂h j dxi − μ p i dxi j = 1 ∂h j ∂x i i =1 m
∑∑
dU =
i =1
∑
If this change is greater than zero, the change in circumstances is beneficial. This condition can be expressed in the more usual ratio form: n
∑ p dx i
(24)
i =1 m
n
∂U ∂h j dxi j ∂x i j =1
∑∑ ∂h i =1
i
