(I can't get low) satisfaction

I was intrigued by a tweet from @mithransamuel  this morning (sorry, can’t work out how to screengrab it for some reason), saying “Interesting claim from @ADASSDavidP that improved satisfaction in @hscic service user amid cuts the result of more personalisation”. This claim was made in an ADASS ‘state of social care’ piece available here http://www.adass.org.uk/facing-up-to-the-dilemma-of-social-care/

In this blogpost, I don’t want to talk about the personalisation issue but about the measurement of satisfaction and the political uses to which this measurement can be put. Specifically, there are a number of issues with the ASCOF (as there would be with any measure of satisfaction) which repay close attention. As my particular interest is with people with learning disabilities, I have been intrigued, to say the least, about the much higher rates of reported satisfaction amongst people with learning disabilities compared to other groups of people using social care over repeated iterations of the Adult Social Care Survey.

For example, the provisional data for 2013/14 report that 79.8% of people with learning disabilities were ‘Extremely satisfied’ or ‘Very satisfied’ with the care and support services they receive (Question 1), compared to 62.8% of people with physical disabilities/frailty/sensory impairment, 63.6% of people with mental health issues, 56.6% of people with substance misuse issues and 66.1% of vulnerable people (http://www.hscic.gov.uk/catalogue/PUB14386). This pattern is consistent across questions and across the different years of the Adult Social Care Survey so far.

Why are people with learning disabilities apparently so much more satisfied with their support than everyone else? Eric Emerson and I have been playing with some analyses of anonymised data from the previous year’s Adult Social Care Survey to try and investigate this issue further. Unsurprisingly, almost all people with learning disabilities completed the survey with varying degrees of ‘support’ from other people, and that ‘support’ was most likely to come from careworkers. People with learning disabilities were also less likely to complete the survey on their own than other groups of people.

Percentage of respondents with learning disabilities receiving support during completion
Type of Support
None	8%
Someone else read the questions to me	61%
Someone else translated the questions for me	61%
Someone else wrote down the answers for me	48%
I talked through the questions with someone else	27%
Someone answered for me, without asking me the questions	9%
Support received from
Careworker	52%
Someone in their household	25%
Someone from outside their household	15%

Does the degree of ‘support’ people get to complete the survey have a bearing on their levels of reported satisfaction? The answer quite simply is yes. Even when statistically controlling for gender, age, support needs and type of social care support people were getting, in 15 out of 19 satisfaction/wellbeing indicators people were statistically more likely to report the highest possible level of satisfaction if they had been supported to complete the survey by a careworker. The effects were particularly large with questions relating to satisfaction with care and support: for example people were almost 9 times more likely to report the highest level of satisfaction with care and support services helping them to feel safe if a careworker had helped them complete the survey.

This analysis reveals some real issues with using subjective satisfaction/wellbeing measures with groups of people who may need ‘support’ to complete them, and disenfranchises significant groups of people with learning disabilities from the process altogether.

But there are also some broader issues with using satisfaction measures as a gauge for understanding people’s experience and for evaluating the impact of social policies, services and money (or the lack of it) on people’s lives. Put crudely, unless people’s circumstances are really appalling they are likely to report relatively high levels of satisfaction (typically in the range of 75% +/- 5%). What’s more, people’s levels of reported satisfaction or well-being on these type of scales don’t change much over time, and tend to revert to a typical level for the person pretty quickly in the face or positive or negative changes in people’s lives.

Why are these properties of satisfaction measures useful politically? One particularly enraging example is below – the Virgin Trains ‘performance’ posters that I find myself staring all too often on cold, dark, rainy mornings on Oxenholme station. Look at that overall satisfaction rating baby! (and notice they don’t ask about satisfaction with the prices of the tickets). I’ve resentfully stared at these posters for years (my nerd eyes are irresistibly drawn to data, even if it’s garbage) and the satisfaction numbers stay pretty constant not matter if the punctuality and reliability of the train service is going south (unlike the trains).

Or try this from the ONS – which constantly tells us our well-being and happiness has remained high despite the recession?

Does the comment by David Pearson show a similar politically useful usage of satisfaction measurement data?

As Ivan Brown, Eric Emerson and myself put it in a recent paper (reference below): “To use indicators that may be insensitive to the scale of change that could reasonably be expected of social policy interventions could undermine the perceived value of progressive social policies (additional investment or change failing to improve subjective well-being), and inadvertently support regressive social policies (reducing the investment in or quality of supports not reducing subjective well-being). This may be particularly problematic when applied to people with intellectual disabilities, given their increased tendency to provide acquiescent responses to questions.”

Amartya Sen sums this up more pithily: ‘Our desires and pleasure-taking abilities adjust to circumstances, especially to make life bearable in adverse situations. …deprived people tend to come to terms with their deprivation …. [as a result] the deprivation of the persistently deprived may look muffled and muted’ (Sen, 2001).

And just because I’m on annual leave from round about now, here’s a rather marvellous rendition of (I can’t get no) satisfaction from the Spooky Men’s Chorale https://www.youtube.com/watch?v=0VLQgRWrAw4

Happy holidays (75% +/- 5%) everyone!

References

Brown, I., Hatton, C. & Emerson, E. (2013). Quality of life indicators for people with intellectual disabilities: Extending current practice. Intellectual and Developmental Disabilities, 51, 333-348.

Sen, A. (2001). Development as Freedom. Oxford: Oxford University Press.