A review carried out by the University of Oxford’s Rees Centre and The Alan Turing Institute for What Works for Children’s Social Care finds that there are substantial reasons to be concerned about the ethics of using these techniques, and that these can only be mitigated through care and transparency in their use.
What Works for Children’s Social Care is pleased to be publishing a review of the ethics of using machine learning in children’s social care.
Machine learning is a general approach in computer science that allows algorithms to carry out tasks on the basis of data, without being explicitly and completely pre-programmed by designers. Machine learning has been used elsewhere in the public sector to improve the personalisation of services, the prediction and analysis of trends, organisational functioning, and resource allocation.
The review concludes that these techniques should not be used without proper ethical oversight; that there are serious risks of reinforcing biases, or risk aversion in the system; and that low data quality may mean either that risks are missed, or that families are subjected to assessment or interventions that they don’t need.
To ensure that these approaches are used as ethically as possible, the report makes several recommendations, including (among others);
- Mandate the responsible design and use of machine learning models in children’s social care at a national level. These standards should protect affected stakeholders against the misuse of data in social care settings and should provide local authorities with guidelines for designing, procuring, and implementing machine learning models fairly, ethically, and responsibly
- Institutionalise inclusive and consent-based practices for designing, procuring, and implementing machine learning models. Local authorities should actively pursue the creation of engagement processes, to ensure the consent-based involvement of affected stakeholders in the design, procurement, and implementation workflows
- Focus on individual- and family-advancing outcomes, strengths-based approaches, and community-guided prospect modelling. This starting point in an improved data landscape would call upon data scientists to develop novel approaches to these analytics that enable holistic considerations of developmental, physical, cognitive, social and emotional needs of affected individuals.
- Improve data quality and understanding through professional development and training.
In order to provide a rounded view, the research draws on a review of the existing literature, an examination of existing ethical frameworks in social care and machine learning, and roundtable discussions with families with experience of children’s social care, practitioners and other experts in the field.
This report is particularly timely, as these approaches are in use in a growing number of local authorities, with varying levels of deployment and transparency. What Works for Children’s Social Care commissioned this review of the ethics of the approach in response to public and sector concern about the absence of a bespoke ethical framework for children’s social care, as part of a broader programme of work around predictive analytics. The second component, a test of the efficacy of these approaches, is due to be published in the summer.
Michael Sanders, Executive Director of What Works for Children’s Social Care, said:
“At What Works for Children’s Social Care, we believe that we need to have an open and transparent debate about the use of predictive analytics and data science in children’s social care, and one that draws in the widest possible number of voices, armed with the best possible academic research. This report from the Rees Centre and The Alan Turing Institute is an important part of that debate and will help both the public, and local and national governments, to consider whether a particular course of action is the right one. ”
Dr Lisa Holmes, Associate Professor at Oxford University and Director of the Rees Centre, said;
“I am delighted to see this report published. I hope that our findings, and in particular our recommendations, can help to ensure that ethical considerations feature in discussions and decisions about the use of machine learning in children’s social care.”
Dr David Leslie, Ethics Fellow, The Alan Turing Institute, said:
“In undertaking this ethics review, we are extremely fortunate to have drawn upon the insights of practitioners, data scientists, and families with lived experience of children’s social care. As the use of machine learning systems in this sector continues to grow, it will become more and more important for those most personally and professionally impacted by them to come together to steer these technologies’ prospects and to navigate their challenges. More than anything else, this ethics review stresses the need for well-informed and inclusive participation to achieve this end and to steward the potential of machine learning systems to significantly advance public welfare and the social good.”
Watch a recording of our recent launch event for the report at The Alan Turing Institute