I was intrigued by a tweet from @mithransamuel this morning (sorry, can’t work out how to
screengrab it for some reason), saying “Interesting claim from @ADASSDavidP
that improved satisfaction in @hscic service user amid cuts the result of more
personalisation”. This claim was made in an ADASS ‘state of social care’ piece
available here http://www.adass.org.uk/facing-up-to-the-dilemma-of-social-care/
In this blogpost, I don’t want to talk about the
personalisation issue but about the measurement of satisfaction and the
political uses to which this measurement can be put. Specifically, there are a number
of issues with the ASCOF (as there would be with any measure of satisfaction)
which repay close attention. As my particular interest is with people with
learning disabilities, I have been intrigued, to say the least, about the much
higher rates of reported satisfaction amongst people with learning disabilities
compared to other groups of people using social care over repeated iterations
of the Adult Social Care Survey.
For example, the provisional data for 2013/14 report that
79.8% of people with learning disabilities were ‘Extremely satisfied’ or ‘Very
satisfied’ with the care and support services they receive (Question 1),
compared to 62.8% of people with physical disabilities/frailty/sensory
impairment, 63.6% of people with mental health issues, 56.6% of people with
substance misuse issues and 66.1% of vulnerable people (http://www.hscic.gov.uk/catalogue/PUB14386).
This pattern is consistent across questions and across the different years of
the Adult Social Care Survey so far.
Why are people with learning disabilities apparently so much
more satisfied with their support than everyone else? Eric Emerson and I have
been playing with some analyses of anonymised data from the previous year’s
Adult Social Care Survey to try and investigate this issue further. Unsurprisingly,
almost all people with learning disabilities completed the survey with varying
degrees of ‘support’ from other people, and that ‘support’ was most likely to
come from careworkers. People with learning disabilities were also less likely
to complete the survey on their own than other groups of people.
|
Percentage of respondents with learning disabilities receiving
support during completion
|
|
|
Type of Support
|
|
|
None
|
8%
|
|
Someone else read the questions to me
|
61%
|
|
Someone else translated the questions
for me
|
61%
|
|
Someone else wrote down the answers for
me
|
48%
|
|
I talked through the questions with
someone else
|
27%
|
|
Someone answered for me, without asking
me the questions
|
9%
|
|
Support received from
|
|
|
Careworker
|
52%
|
|
Someone in their household
|
25%
|
|
Someone from outside their household
|
15%
|
Does the degree of ‘support’ people get to complete the
survey have a bearing on their levels of reported satisfaction? The answer
quite simply is yes. Even when statistically controlling for gender, age,
support needs and type of social care support people were getting, in 15 out of
19 satisfaction/wellbeing indicators people were statistically more likely to
report the highest possible level of satisfaction if they had been supported to
complete the survey by a careworker. The effects were particularly large with
questions relating to satisfaction with care and support: for example people
were almost 9 times more likely to report the highest level of satisfaction with
care and support services helping them to feel safe if a careworker had helped
them complete the survey.
This analysis reveals some real issues with using subjective
satisfaction/wellbeing measures with groups of people who may need ‘support’ to
complete them, and disenfranchises significant groups of people with learning
disabilities from the process altogether.
But there are also some broader issues with using satisfaction
measures as a gauge for understanding people’s experience and for evaluating
the impact of social policies, services and money (or the lack of it) on people’s
lives. Put crudely, unless people’s circumstances are really appalling they are
likely to report relatively high levels of satisfaction (typically in the range
of 75% +/- 5%). What’s more, people’s levels of reported satisfaction or
well-being on these type of scales don’t change much over time, and tend to
revert to a typical level for the person pretty quickly in the face or positive
or negative changes in people’s lives.
Why are these properties of satisfaction measures useful
politically? One particularly enraging example is below – the Virgin Trains ‘performance’
posters that I find myself staring all too often on cold, dark, rainy mornings
on Oxenholme station. Look at that overall satisfaction rating baby! (and notice
they don’t ask about satisfaction with the prices of the tickets). I’ve resentfully
stared at these posters for years (my nerd eyes are irresistibly drawn to data,
even if it’s garbage) and the satisfaction numbers stay pretty constant not
matter if the punctuality and reliability of the train service is going south
(unlike the trains).
Or try this from the ONS – which constantly tells us our
well-being and happiness has remained high despite the recession?
Does the comment by David Pearson show a similar politically
useful usage of satisfaction measurement data?
As Ivan Brown, Eric Emerson and myself put it in a recent
paper (reference below): “To use indicators that may be
insensitive to the scale of change that could reasonably be expected of social
policy interventions could undermine the perceived value of progressive social
policies (additional investment or change failing to improve subjective
well-being), and inadvertently support regressive social policies (reducing the
investment in or quality of supports not reducing subjective well-being). This
may be particularly problematic when applied to people with intellectual
disabilities, given their increased tendency to provide acquiescent responses
to questions.”
Amartya Sen sums this up more pithily: ‘Our desires and
pleasure-taking abilities adjust to circumstances, especially to make life
bearable in adverse situations. …deprived people tend to come to terms with
their deprivation …. [as a result] the deprivation of the persistently
deprived may look muffled and muted’ (Sen, 2001).
And just because I’m on annual leave from round about now,
here’s a rather marvellous rendition of (I can’t get no) satisfaction from the Spooky
Men’s Chorale https://www.youtube.com/watch?v=0VLQgRWrAw4
Happy holidays (75% +/- 5%) everyone!
References
Brown, I.,
Hatton, C. & Emerson, E. (2013). Quality of life indicators for people with
intellectual disabilities: Extending current practice. Intellectual and Developmental Disabilities, 51, 333-348.
Sen, A.
(2001). Development as Freedom.
Oxford: Oxford University Press.