March saw the publication of an important study – a randomised controlled trial (RCT) carried out in the Netherlands testing the effectiveness of Family Group Conferences. This is important for two reasons.

First, Family Group Conferences, which bring family members together to discuss a case and to try and posit solutions around the time (in the UK) of a public law outline being issued, are popular and increasingly used in practice in the UK and elsewhere, as well as having been supported by the Department for Education through the Innovation Programme.

Second, because the evaluation method – an RCT, is relatively novel in children’s social care, and is widely held to be the ‘gold standard’ way of evaluating whether something works, and how well. They’re not appropriate all the time, and running them in complex areas like children’s social care is tricky, but at the What Works Centre we believe they’re an important part of having a vibrant evidence ecosystem.

When we’re considering an academic research paper (or indeed any research at all), we should ask ourselves two questions – Really? And So What? The first, because it’s important to check whether the claims made by research stand up to scrutiny, and so what, because even if a finding is true, it doesn’t mean that it’s useful.

Really?

So, what did this study find? At first glance, the picture isn’t encouraging for fans of Family Group conferences – it finds no effect on most of its outcomes – including perceived social support, empowerment of parents, and maltreatment of young people. Most concerningly, the study finds a positive and significant impact of FGCs on the likelihood of a child being taken into care. The statistical approach used in finding these approaches is the standard one, and is appropriate here. The researchers have taken some sensible precautions against false findings.

There are important caveats here. First, the study uses an ‘intention to treat’ analysis, so compares families that were randomly assigned to FGC – whether they actually had a conference or not -to those that were randomly assigned to business as usual. With a very low take-up of conferences (about a quarter of families), a real effect might be missed – although I don’t personally think this is likely.

Second, a lot of families were lost between the beginning of the study and the end, and their results have been ‘filled in’ with a statistical technique called multiple imputation. Again, this is a standard technique, but using it for as many data-points as has been done here (about a third of the treatment group), isn’t ideal, and based on the different dropout rates between treatment and control, it may introduce a bias into the results.

Finally, I’m much more skeptical about the taken-into-care result. The numbers of children in both groups, but particularly the control group, that went into care are so small that a false positive result is more likely. I’d take this finding with a pinch of salt.

So What?

This study was conducted in the Netherlands, which offers a different context for social services than the UK. We therefore need to be careful about extrapolating it to practice over here. This study is important, but by itself doesn’t suggest to me a need to change direction just yet. It does underline that Family Group Conferences aren’t a silver bullet, and certainly aren’t effective everywhere or for everyone. We need a better understanding of whether they do deliver in a UK context – something that the What Works Centre will be hoping to begin researching very soon.