This blog was originally posted on the Education Endowment Foundation website
Often in research and public policy we hear the expression ‘collect once, use many times’.
Rarely do we practice what we preach.
Either the data aren’t defined or collected in quite the right way. Or the outcome measures don’t precisely address our research questions, or perhaps too much time has passed for the original data to be of immediate use.
And so we end up concluding that it’s easier to just start from scratch. Not necessarily cheaper, or timelier, or of less burden to our participants. Just easier.
It takes a unique combination of humility and audacity to build on others’ work. To accept what might be less than perfect data and simply ask, how far can we go with what we’ve got? What questions could we answer, and with what degree of confidence?
This is exactly what Michael Sanders and colleagues at What Works for Children’s Social Care (WWCSC) have done with their reanalysis of the Education Endowment Foundation’s (EEF) data archive, published as What Works in Education for Children who have had Social Workers.
They’ve taken an extensive data set – covering 83 randomised controlled trials (RCTs) at the time of analysis, all designed to help answer questions about closing the attainment gap for socio-economically disadvantaged pupils – and asked how it could be used to answer a slightly different question about outcomes for children who have experienced social care in the six years prior.
To date, over 1.6 million children and young people have been involved in an EEF trial. This is a rich resource for researchers. The EEF itself is using it to conduct additional sub-group analysis on pupils eligible for free school meals (FSM), as well as pupils with English as an Additional Language (EAL) and Special Educational Needs and Disabilities (SEND).
But when the EEF started this archive of our trials’ data, we always hoped that it would grow to become a public good in its own right. We wanted any organisation with a shared interest in improving children and young people’s lives to be able to access the data we’d collected to support their own aims.
It’s a long way between hope and reality, though, and perhaps not surprising that it’s taken eight years for the archived data to become sufficiently extensive to help answer meaningful questions for some of the most dispersed cohorts in our system. This is, therefore, a moment worth marking – as our WWCSC colleagues note here, ‘This project represents the first time one WWCSC has systematically re-analysed the data from another, and reflects how far we’ve come in terms of designing research in such a way as to allow this kind of analysis later.’
As for the signs of potential identified in the analysis – those interventions that appear to have larger positive impacts for children and young people who have had a social worker than for their peers – there are some intriguing patterns in the findings.
Those projects with a specific focus on building effective partnerships with parents and carers emerge as potential good bets. As do a number of catch-up interventions focused on addressing gaps in pupils’ core literacy skills.
The EEF has previously identified a range of targeted catch-up interventions that yield differentially better progress for the disadvantaged pupils. But the findings regarding programmes focused on parents and carers hint at something new.
Parental engagement in children’s learning and the quality of the home learning environment has long been associated in the evidence base with improved academic outcomes at all ages. The research points to three key levers for change: supporting parents to have and fulfil high academic expectations for their children; developing and maintaining communication with parents about school activities and schoolwork; and promoting the development of effective reading habits.
Yet the theoretical promise of these levers has rarely been matched by tested interventions.
Which is why this new analysis is so useful as a starting point. It offers us the chance to both narrow and refine our lines of enquiry. It helps us reassess what we know without just throwing everything up in the air. To be smarter about where we invest our research effort and spend.
From the EEF’s perspective, it’s wonderful to see the initial vision for the data archive starting to be realised. The challenge of improving outcomes for children and young people can often feel monumental and no single organisation can tackle it alone. But a culture of collaboration gives us a much better chance of success. What Works for Children’s Social Care has now shown what is not necessarily easier, but nonetheless is achievable.
Our hope now is that this project can light the way for others, to highlight these new lines of enquiry so that we can test with renewed focus and ultimately offer better guidance and support to policy-makers, commissioners and, crucially, those working directly on the frontline with children and young people to improve outcomes and lives.
* Stephen Fraser is Deputy Chief Executive of the Education Endowment Foundation (EEF).