Getting real-time feedback from apprentices

We want to find out how well apprenticeships are working for apprentices. 

Industry moves fast and we want to make sure apprenticeships move with the times, and remain relevant and effective. But how?

We’re starting by getting real-time feedback from apprentices to see how their apprenticeships are going and if their course is moving them toward their goals.

Are apprentices up for this?

In our research sessions, apprentices have consistently told us they’re happy to give feedback regularly if it’s quick, easy and it has a clear purpose.

They wanted to give feedback by text message. When they start their apprenticeship, apprentices are asked for their phone number and for consent to be contacted by phone for surveys like this.

What we’re asking apprentices about

By answering our questions, apprentices are providing us with valuable feedback. For example, their answers let us know if they are getting:

• the skills they need to progress in their job?
• the support they need?
• time off the job to do their training?

When talking to apprentices, some told us that industry tends to advance faster than their apprenticeship, for example some aspects of their apprenticeship training are no longer relevant for their work. We can ask apprentices about this to help drive the quality of standards.

We’ve also been talking to the Institute for Apprenticeships. We can collaborate with them to tailor our text messages to ask specific questions about particular standards and help them review and improve apprenticeship standards.

How we’re doing it

You might think it would be time consuming to have a text conversation with a large number of  apprentices across England. But we’ve built a simple system using chatbot technology, which means the whole process is automated.

The system works intuitively. If apprentices give us positive responses to questions, we can reply with positive, supportive messages. If an apprentice’s response suggests that they’re not getting the support they need or they’re not happy with their apprenticeship, we can point them to useful resources.

The system is also very flexible. We can quickly create sets of new questions and send messages to new lists of numbers so that we can ask apprentices for whatever real-time feedback is needed.

Where we are now

We’ve just tested the system with a small, closed group of apprentices.

The aim of that wasn’t necessarily to capture data – it was more about testing how the system works. We use GOV.UK’s notify tool to trigger the messages and we rely on telecoms companies to deliver the messages – so we’re testing how this works.

We’re always getting new insights from users. For example, we’ve found that if we send messages from a short 5 or 6 digit number, it looks more professional, but some users are concerned that it could be a premium number and so they don’t reply.

What’s next?

Longer term, we think much of the information we gather could be helpful to users and we’d like to display it at relevant points in their user journeys. For example, on the Find An Apprenticeship website we think it could be really beneficial to applicants to see what current apprentices think of their course, and maybe include feedback from recently qualified apprentices on how the qualification has helped their career.

As with all ESFA’s digital products, we’ll test, iterate and improve as we go.

Paul Kelman, Product Manager, Education and Skills Funding Agency