September 2022
Summary
This report describes the findings of a pilot evaluation of Kitbag, a direct work resource that was developed by the International Futures Forum (IFF) to help children and young people discover ways to talk about their feelings and share their thoughts in a safe place, whilst helping social work staff and carers to reach out, listen and understand what children and young people are feeling.
This pilot study was conducted between March 2021 and June 2022, with 60 social worker teams involved (830 participants – including social workers, family support workers, foster carers and kinship carers) across four Local Authorities (LAs). We found that while there was low uptake across the four LAs with a number of barriers hindering usage, findings indicate that Kitbag, where it is used and training is provided, can have positive outcomes, such as improving communication and relationships between practitioners, children and young people, and their families.
Objectives
The aim of this pilot study was to explore the potential outcomes of using Kitbag during direct work with children and young people within a social care setting. The “Putting Kitbag to Work” pilot evaluation had three key objectives, namely to better understand:
- The implementation of Kitbag in a children’s social care context. This includes understanding what business as usual (BAU) looks like for direct work, and understanding the delivery of the “Putting Kitbag to Work” programme in the four participating LAs and subsequently the take-up of Kitbag by practitioners
- People’s experiences of using Kitbag and any perceived outcomes. This includes understanding whether there has been perceived change in how direct work is undertaken in the participating LAs, and any other perceived outcomes for children/young people, social workers, family support workers, and foster carers. We also want to understand what prevents or supports staff to use Kitbag
- The factors that would need to be in place to enable effective evaluations of Kitbag in the future, for example through a larger-scale pilot study or randomised controlled trial.
Methods
We used a combination of qualitative and quantitative data collection:
- Conducted two focus groups and 13 interviews in total with a range of participants who were involved in the intervention
- Attended and observed five Kitbag workshops as well as the “Kitbag Learning Event”
- Conducted a survey at an interim point, five months after the intervention launch, to get a sense of uptake among the social work teams and carers who were assigned to the treatment group, and to determine whether or not it was feasible to proceed with an impact evaluation for the study
- Requested administrative data from delivery partners to assist with our interpretation of the research questions.
Key findings
The findings paint a varied picture of BAU direct work, with practice largely dependent on individual social workers and their wider team culture. Where they were used, social workers tended to source direct work tools themselves, while also making use of resources provided by their LA. There was general agreement that LAs should provide additional resources and training, but there was also a sense that social workers should use creativity in conducting dynamic direct work.
Findings indicate that Kitbag, where it is used and training is provided, can have positive outcomes, such as improving communication and relationships between practitioners, children and young people, and their families. There was also a strong suggestion that children can take a lead on how the Kitbag is used, allowing for a shift in the power dynamic between adult/social worker and child.
Overall, there was low uptake across the four LAs with a number of barriers hindering usage, including issues LAs faced in the distribution of Kitbags (particularly due to COVID-19 when staff were frequently working from home), a lack of understanding of the purpose of Kitbag, and Kitbag not being seen as suitable for use for children and young people of different ages and with different needs.
Conclusions, recommendations and implications
Due to low uptake resulting in a small sample size, we weren’t able to quantitatively assess the impact of using Kitbag, meaning we are not in the position to strongly recommend the resource or further impact evaluation. However, interview data suggests that there are no significant harms to using Kitbag, and some practitioners and carers found using the bag aided their communication. We therefore recommend making Kitbag available to practitioners and carers, along with other direct work resources.
Our findings also have a number of implications for development and delivery of the intervention, and for future research. For example, we questioned whether there is impetus to run a rigorous impact evaluation of Kitbag, such as an RCT, as our pilot did not identify any clear harms of providing Kitbag yet did highlight negative outcomes associated with evaluation, such as the randomisation element creating a sense of unfairness, and the added burden of consent forms for families to participate. This led us to propose more development work before conducting any further impact evaluations, such as exploring how Kitbag is different from BAU.
We also made recommendations around planning a longer run-in period in LAs before distributing Kitbag, and various suggestions to increase familiarity with the resource before implementation. We concluded by highlighting the need for more research into how foster and kinship carers define and engage in direct work, and more formative work around investigating how to include children and young people in research.