Evidence standards

Rating the quality of research of ‘What Works’ in children’s social care

The evidence standards we use to rate and rank the quality of research of ‘What Works’ in children’s social care

We want you to be sure of the quality and reliability of the evidence we’re showing. We’ve used these standards to review existing social care research and they will also to shape new research carried out by the What Works Centre and other researchers.

The EMMIE framework

The summaries in the Evidence Store have been created by Cardiff University, who applied our evidence standards to systematic reviews.

We’ve chosen to use the EMMIE framework for our evidence standards. This was developed by the UCL Jill Dando Institute, for use by the What Works Centre for Crime Reduction in their toolkit. This work was co-funded by the College of Policing and the Economic and Social Research Council (ESRC).

EMMIE stands for:

Effect: What difference does the intervention tend to make, based on published evaluations?

Mechanisms: What do we know about what makes it work?

Moderators: What things, such as the context, might influence whether it works?

Implementation: What is the evidence about how to implement it?

Economic impact: What are the economic implications of using it?

EMMIE also provides a framework for judging the quality of systematic reviews (the EMMIE-Q process).

Why we used the EMMIE framework

Some evidence standards focus solely on whether an intervention works – its effect. But we believe that how an intervention is implemented and other contextual factors can make a big difference. An intervention or service that worked in one place may not be effective in another.

That’s why we’re adopting a broader approach to evidence that also looks at issues of implementation and context, as well as effectiveness.

We think the EMMIE framework is suitable for children’s social care research because:

• Children’s services are complex interventions. A careful approach is needed to address this, including an understanding of how services work and the context in which they are delivered
• Children’s social care professionals aren’t only interested in what works, but also who it works for, in what circumstances, and why
• We want to produce evidence that children’s social care professionals find useful, and that includes exploring why services make a difference and how to implement them in practice

Each of the intervention summaries in the Evidence Store aims to give you information relating to the five domains of EMMIE, and also reports the strength of the evidence available to answer these questions.

We’ve put this information into plain English, but the full EMMIE summary of each intervention is under ‘further information’ for you to download.