Summary

Signs of Safety (SoS) is a strengths-based, safety-organised approach to child protection casework that is widely used across England – with two thirds of local authorities in England using SoS in some form. 

WWCSC and the Health Social Care Workforce Research Unit at King’s College London evaluated the effects of SoS in 9 pilot sites which were supported by Wave 2 of the Department for Education Innovation Programme. The study used a multi-method approach to develop the evidence base on SoS further. 

The analysis by WWCSC provides causal estimates of the effect of SoS on four outcomes for children and families and can be found here: 

Objectives

The aim of this analysis was to obtain quasi-causal estimates of the effect of SoS on outcomes for children and families involved in the programme. Quasi-causal estimates provide an accurate picture of the impact of SoS and so can help decision-makers in local authorities make decisions about whether or not to invest in SoS. This work built on WWCSC’s systematic review of the evidence base on SoS. 

Specifically, it aimed to answer the following questions*: 

  1. What, if any, is the impact of SoS on the duration of assessments?
  2. What, if any, is the impact of SoS on the likelihood of a case being re-referred if it has previously been assessed as ‘no further action’ (NFA)?
  3. What, if any, is the impact of SoS on the likelihood of a re-referral leading to a child protection plan (CPP) or to a child becoming looked after (LAC)?
  4. What, if any, is the impact of SoS on the likelihood of a child receiving kinship care instead of non-kinship care?

How we went about it

WWCSC conducted a difference-in-differences (DiD) analysis. This involves comparing outcomes for children in local authorities where SoS was introduced with outcomes for children in similar local authorities that did not introduce SoS. 

To conduct the analysis, we accessed individual-level data on children’s social care outcomes for individuals in pilot and comparator local authorities held by the Department for Education.We conducted an analysis to determine the impact of SoS on the four outcomes of interest. We then conducted secondary analyses to evaluate the extent to which the impact of SoS is more pronounced within certain subgroups. We investigated whether SoS had a differential impact on local authorities by Ofsted rating, and whether the treatment effect was stronger for pilot sites that have achieved a better delivery of the model.  

Key findings

The DiD analysis found no strong evidence that SoS significantly impacted outcomes for children and families. The analysis found moderate strength evidence that, contrary to the aims of the programme, SoS decreased the probability of a child being looked after through kinship care (compared with non-kinship care). The study found no strong evidence of SoS affecting the duration of assessments, the probability of a child being re-referred within 6 months or the probability of a child being re-referred and their case escalating.

The secondary analyses suggested the limited impact of SoS was not the result of varying degrees of embeddedness or the quality of delivery but remained similar even when accounting for these factors. When controlling for the fact that comparator local authorities used practice models similar to SoS, the treatment effects were of a higher magnitude than in the main specifications. 

Implications

For local authorities already using SoS, this research offers no reason to stop using it (although we would counsel monitoring kinship care closely). For local authorities who are considering using SoS to improve outcomes and / or reduce costs, this research does not offer evidence to support the use of SoS to further these goals. For local authorities who are considering using SoS as a values framework for their practice, this evidence is not the right type of evidence to support or oppose the use of SoS in this way.

*The original analysis plan included a fifth outcome, namely the likelihood of an initial child protection conference (ICPC). The final report does not discuss this outcome because the data available to the researchers was incomplete. The analysis of this outcome will be added in an addendum to the report.