June 2022
Summary
This report presents findings from the evaluation of Affordable Maths Tuition (AMT) by Third Space Learning (TSL). AMT is an online tutoring programme that aims to reduce the maths attainment gap by recruiting and training maths tutors in India and Sri Lanka to make online one-to-one tuition more affordable to children in English schools.
This project builds on a report WWCSC released in February 2020 which revisited 63 trials funded by the Education Endowment Foundation to determine what works in education for children who have had social workers. AMT was one of the ten interventions that the research identified as showing signs of potential for children’s social care and which warranted further research.
AMT provides weekly online sessions in which tutors work one-to-one with pupils on maths topics. The virtual sessions are led by maths tutors in India and Sri Lanka trained by TSL to provide tuition based on the UK curriculum. Tutors provide feedback which informs regular reports that are available to pupils’ teachers, helping to guide them in class work.
This evaluation aimed to assess the effectiveness of AMT in improving maths attainment for children who have had a social worker in the past six years. The evaluation’s original design comprised both an impact evaluation and an implementation and process evaluation (IPE). However, the Covid-19 pandemic disrupted in-school activities and the primary outcome measure, KS2 SATs scores, was no longer available for the 2020/21 cohort, making the impact evaluation unviable. The IPE was retained as a means of understanding experiences of the AMT programme.
The delivery involved:
- Recruitment of tutors in India & Sri Lanka to deliver the tuition sessions
- TSL training provision to tutors of the UK curriculum
- Weekly online one-to-one tutoring sessions with pupils.
Eligibility Criteria
Year 6 pupils that in the past six years have been:
- A Child in Need
- Subject to a Special Guardianship Order
- Subject to a Child Protection Plan
- Have been Looked After.
Objectives
This evaluation aimed to assess the effectiveness of AMT in improving maths attainment for children who have had a social worker in the past six years.
Methods
The evaluation comprised several strands of work, employing both qualitative and quantitative methods in order to better understand and assess how AMT had been implemented and received. These work strands were:
- A review of tutor training materials and recorded online training sessions with tutors
- Observation of ten recorded AMT sessions
- Telephone interviews with maths leads/ teachers (20 at baseline and 11 at endline)
- Online paired/ grouped interviews with 14 pupils in the intervention group
- Online interviews with ten AMT tutors
- Five online/ telephone interviews with Virtual School Heads
- Two telephone interviews with stakeholders from TSL
- Collection of cost data from participating schools.
Qualitative data was compiled and analysed thematically using NatCen’s Framework method. Findings are mapped against the research questions and presented by evaluation domain.
The evaluation ran from August 2020 until March 2022, with AMT sessions delivered to pupils between October 2020 and July 2021. Significant disruption to delivery occurred between January and April 2021 due to the Covid-19 pandemic, which led schools to close to all pupils with the exception of provision for vulnerable children and children of critical workers.
Key Findings
The AMT programme offered a welcome approach to supporting disadvantaged pupils with maths learning. The format and nature of the intervention (specifically a tailored online one-to-one tuition service) presented a positive approach to tackling educational disadvantage. The programme was well received overall, with school staff and pupils often highlighting improvements in maths engagement, enjoyment and confidence.
There were, however, several barriers to delivery, limiting the potential of the programme to impact pupils’ outcomes. These barriers were predominantly practical, with technical difficulties, lack of available space, staff resource constraints and pupil absence being raised by stakeholders. This was compounded by issues with awareness and engagement among some school staff, which limited their use of tools available as part of AMT.
It is also important to note that Covid-19 was a key barrier to key elements of success of the programme, including fidelity, acceptability and mechanisms by which outcomes were achieved. This was primarily because school closures put many students with a lack of access to technology and parental support for learning at home at greater disadvantage to progressing with their learning via the AMT programme.
Conclusions and Recommendations
Findings from the evaluation suggest that the AMT programme offers promise for children with a social worker, with some limitations and caveats. The evaluation highlighted several issues with delivery which could helpfully be addressed. This might include:
- Improving the quality of equipment provided as part of the intervention (such as headsets)
- Greater reliance (and perhaps further consideration of rewards, such as effort points), in order to improve pupil engagement
- Ensuring that pupils have the same tutor week on week
- Reconsideration of the 24-hour cancellation/ rescheduling policy to give school staff greater flexibility to reschedule AMT sessions
- Clearer communication with teachers around eligibility criteria to ensure that pupils abilities are better matched to the programme, and, in turn, that they can benefit from it
- Activities to promote engagement of school staff and encourage teachers to make better use of tools available to them as part of AMT. This will help teachers build closer connections between AMT and maths teaching in the classroom.
Importantly, it is also recommended that the AMT programme is evaluated with an impact evaluation and IPE once the educational context is more stable, and Covid-19 less of a mitigating factor in delivery.