Summary
This report presents findings from a pilot evaluation of Behaviour Outreach Support Service (BOSS). This research builds on a report What Works for Children’s Social Care (WWCSC) released in February 2020 which revisited 63 trials funded by the Education Endowment Foundation to determine what works in education for children who have had social workers. It resulted in WWCSC in collaboration with the Department for Education (DfE) to hold an open call for three school-based programmes which aim to increase educational attainment for children with a social worker, one of which being the BOSS programme.
The Behaviour Outreach Support Service was designed and delivered by Family Action. It aims to help schools improve their support to pupils who display challenging behaviour that compromises their learning and attendance. BOSS was delivered to 77 pupils aged 5 to 11, who have or have had a social worker in the past six years, in eight primary schools in York in the academic year 2020/21.
BOSS consisted of weekly school-based 1:1 sessions between BOSS workers and pupils, of around 45 minutes, for an average of 15 weeks, plus training for school staff and some parents/carers. Family Action is currently implementing a similar model in Lincolnshire schools and has a secondary school model of BOSS, outside the scope of this report. Coram carried out a mixed methods evaluation of the pilot programme.
Objectives
The evaluation was designed to answer the following key questions:
- Was the programme implemented as planned across the schools, and in relation to the previous Lincolnshire model, in the new York context? Why or why not?
- How different or similar is BOSS to behaviour-management-as-usual in the participating schools?
- What were children’s, parents’ and carers’, teachers’, and social workers’ experiences of the programme?
- What was the cost of implementing the programme?
Secondary research questions included exploring the perceived impacts and consequences of BOSS, including whether it changed the support available to target children, and the impact on outcomes such as attendance and referrals to children’s social care.
How we went about it
We collected data through:
- Semi-structured interviews with 65 pupils, parents, carers, and teachers, as well as BOSS staff and York council staff. All but one interview was carried out 1:1
- Administrative data collected from four schools on 1,094 pupils
- Case files of five Behaviour Improvement Plan (BIP) pupils (those worked with directly by Family Action) reviewed as part of a site visit
- Paper diaries completed by 193 pupils (both those with a BIP and not) in class
- An online survey completed by 41 respondents: 39 school staff and 2 other professionals.
Key Findings
- Outcomes: BOSS was a small-scale programme which ran for one academic year and the evaluation was not designed to measure impact robustly. This means any beneficial effects on child outcomes were difficult to measure, especially as the diaries and administrative data we gathered from schools were sparse, so most of our conclusions on outcomes are based on qualitative evidence.
- BOSS was an enjoyable experience: All pupils we interviewed enjoyed spending time with their BOSS worker. Parents we spoke to also appreciated having support from BOSS workers in respect of their child’s behaviour at home. Enjoyment was perceived to help improve other outcomes, in that BOSS provided another reason to go to school. None of our evidence suggested unintended negative consequences for children or schools taking part.
- Changes to practice: As with the other impacts we asked about, many felt it was too soon to say what difference BOSS had made, but around half of school staff surveyed thought BOSS had made some or a lot of difference to professional practice, in how individual teachers, teaching assistants, and schools generally worked with children.
- Behaviour: In interviews we heard examples of improved family relationships, improved self-esteem among children, improved well-being, and feeling happier about going to school. The sparse data we received did not allow us to draw conclusions on the BOSS programme’s impact on behaviour in schools as a whole, though some evidence from the small number of targeted pupils’ diaries indicated slight improvement in how much pupils enjoyed school over the course of the Spring and Summer terms. The improvements were attributed to children and adults learning self-regulation and de-escalation strategies. Children had improved their ability to cope through learning practical strategies to manage anger or remain calm.
- Attainment: 1:1 support from BOSS took pupils out of classroom teaching, but in some cases the abilities of BIP pupils were believed to have improved as a result of spending more time in the classroom due to their improved behaviour following BOSS involvement. The administrative data we received did not allow us to draw conclusions on any impact on attainment.
Implications
Interpretation of findings from this pilot evaluation should also consider that delivery of BOSS in York was affected by the COVID-19 pandemic.
Based on what worked well in York as well as ways in which delivery could be improved, when introducing BOSS model in a new school, senior leaders should consider:
- Offering BIPs to a wider pool of pupils
- A transition model which begins in Year 6 and ends, not at the end of Year 6, but in the first year of secondary school.
Future research should:
- Seek complete data on all BIP pupils, to allow analysis of which pupils appear to benefit most from the 1:1 support
- Gather primary data on behaviour, to see whether our qualitative findings on changes in behaviour by some pupils apply more widely
- Follow transition BIP pupils after their move to secondary, to follow-up on the impacts of BOSS in the new school setting.