This project seeks to evaluate the roll-out of the No Wrong Door model in five local authorities in England, as part of the Department for Education (DfE) funded Strengthening Families, Protecting Children (SFPC) programme. The local authorities analysed are Rochdale, Redcar & Cleveland, Warrington, Norfolk, and Leicester.

The evaluation of the roll-out includes an impact evaluation of the model, with a stepped wedge cluster Randomised Controlled Trial (RCT) design. This is accompanied by an implementation and process evaluation across the same five local authorities, to understand the delivery during the rollout of the model. 

The No Wrong Door Model 

Developed in North Yorkshire, No Wrong Door establishes hubs which bring together an integrated range of accommodation options, services and outreach to support young people in care or on the edge of care. The intervention involves creation of hubs which bring together an integrated range of accommodation options, services and outreach to support young people aged 12-25 who are looked after or on the edge of care, at risk of family or placement breakdown, stepping down from residential care to family based care or transitioning to independent living. 

No Wrong Door operates flexibly, bringing young people into the service quickly and supporting a slow transition out. A key non-negotiable of the programme is using residential care as a short term intervention not a long term solution and a significant indicator is that young people are always progressing to permanence within a family or community. Successful delivery of the model is considered to be contingent on a service wide practice model and approach to decision making and risk which is restorative, solution-focussed, relationships and strengths based, as well as significant support from senior leadership.

Impact Evaluation

The study design of the impact evaluation is a cross-sectional stepped-wedge cluster randomised controlled trial, where the timing of implementation is staggered across local authorities. The point at which local authorities begin implementing the intervention is selected at random, constrained by their level of readiness to implement the model. In this way, all the local authorities in the sample will eventually implement the No Wrong Door model, but randomising the start date of the implementation of No Wrong Door will allow service users in the local authorities that have not yet implemented the programme to act as a control group against service in local authorities where No Wrong Door has already been implemented. 

The primary evaluation question of the stepped wedge RCT relates to care status: What is the impact of No Wrong Door on the likelihood of children becoming looked after?

There are also three secondary questions, relating to changes in: i) the number of days children looked after spend in care, ii) the placement stability of children in care, and iii) the likelihood of children who are or have been in care recently being not in employment education or training (NEET).

Given the challenges the COVID-19 pandemic poses to evaluating a stepped-wedge RCT, a difference-in-differences analysis will be conducted in addition to provide an additional approach to analysing the programmes’ effects and to increase the robustness of the impact evaluation estimates. The analytical approach for the difference-in-differences analysis is set out in a separate protocol at the end of the page.

Implementation and Process Evaluation

The purpose of this implementation and process evaluation is to assess delivery during the rollout of the No Wrong Door Model across five local authorities. The aim of this is to help understand and explain any identified intervention effects (or lack thereof) in the concurrent stepped-wedge cluster randomised controlled trial, to identify elements of successful delivery, and to improve the understanding of the model. 

The implementation and process evaluation seeks to answer the following research questions:

1. Fidelity and adaptation
   • To what extent does delivery in participating authorities adhere to the model?
   • Are the key assumptions and facilitating factors in place?
2. Programme differentiation
   • What does the existing service structure and practice look like in participating authorities prior to the introduction of the model?
3. Reach and acceptability
   • What is the number and characteristics of families reached by the intervention?
   • What is the experience of staff and families who have been involved with the intervention?
4. Mechanism
   • Does implementing the model lead to perceived changes in the interim and ultimate outcomes identified in the logic model?
   • Is the level of effectiveness of the model perceived to differ for different groups?
   • Are there any perceived unintended or negative consequences as a result of introducing the intervention?

Timing

The first local authority is expected to begin implementing No Wrong Door in the Spring of 2020, with the other local authorities following in 4-month intervals. Final outcome data is planned to be collected in April 2024, with final reporting expected in early 2025. 

We anticipate that due to the COVID-19 pandemic that it is likely there will be delays to these timings.

Update

The COVID-19 pandemic, along with associated changes to programme delivery, are likely to undermine our original stepped wedge design. As such, we are adding a supplementary matched difference-in-differences (DiD) analysis to the current research design. The changes to programme delivery have also delayed some of the data collection for our implementation and process evaluation, but this eventuality was already accounted for in our original trial protocols so no significant changes will be required.

Relevant Documents

• Trial protocol – IPE & RCT
• Trial protocol – DiD analysis
• Privacy notice