Nearly two years ago, before I had joined What Works for Children’s Social Care, we published a systematic review that looked at Signs of Safety; a whole system model for children’s services that is incredibly popular in England as well as elsewhere.
The review, which was written by our academic partners at the University of Cardiff’s CASCADE centre, examined the existing literature on Signs of Safety’s effectiveness, particularly focusing on whether there was evidence to support the idea that local authorities that use Signs of Safety as their practice model would see fewer children entering care than they otherwise might.
What it found made for sobering reading. The review found no evidence that Signs of Safety had an impact on reducing entry to care – or anything else for that matter. It wasn’t that Signs of Safety had been shown to be ineffective – it was just that the kinds of research that you’d need to do to robustly say that Signs of Safety had an impact, just hadn’t been done. A few months later, a similar review by Mary Baginsky at King’s College London presented similar findings.
Today, the Department for Education is publishing a new piece of research which we are pleased to have collaborated with Dr Baginsky and her colleagues on. This is an evaluation of the Signs of Safety roll out to nine local authorities which was funded under the second wave of the English Social Work Innovation Programme. The team at King’s began the evaluation some time ago, looking principally at qualitative research, local authority level numbers and analysis of costs, and we joined with them a year ago to contribute an impact evaluation.
Because Signs of Safety is a whole system model, and because the work to roll it out had substantially begun, we made use of a matched difference in differences approach, in which the changes over time in local authorities moving to implement Signs of Safety are compared to the changes in those that are not doing so.
Unlike the systematic review, we aimed to look at a much wider set of impacts from Signs of Safety, which were developed alongside colleagues from King’s, local authority partners, and with substantial input from MTM (now Elia), the organisation which oversees Signs of Safety globally.
So, what did we find?
- Unlike a previous study conducted in evaluating the Signs of Safety as part of the first wave of the Innovation Programme, we didn’t find much evidence to suggest that assessments are completed more quickly under Signs of Safety than otherwise. It’s hard to know whether this would have been a good thing – a quick assessment needn’t be a better one.
- We found no moderate or high strength evidence of an effect of Signs of Safety on the rate of re-referral to children’s services, nor the rate of re-referrals that are followed by the case escalating.
- We found a significant reduction in the use of kinship care in local authorities using Signs of Safety. Children in Signs of Safety pilot local authorities were 13 percentage points less likely to be in kinship care compared with non-kinship care. Given that kinship care has positive impacts for children, this warrants further investigation.
When excluding local authorities with similar practice models from the comparators, the effect sizes increase in magnitude, which may suggest that comparison to similar practice models hides some of the positive impacts of Signs of Safety.
The implications of this kind of research can be unclear. We don’t find that there are very substantial harms associated with Signs of Safety, but nor, on the outcomes we look at, do we find much evidence of benefit. Families, and organisations that strongly support kinship care may take the view that our findings mean that more should be invested in kinship care support in local authorities that use Signs of Safety. Beyond this, the implications are that anyone currently using Signs of Safety is probably safe to continue, but that anyone thinking of starting should probably pause for thought.