A regulated approach to external quality assurance for end point assessments

I want to use this opportunity to explain how we are providing External Quality Assurance (EQA) when asked to do so by trailblazer groups. In doing so, I want to focus today on what Ofqual does in support of the Institute’s overarching responsibility for apprenticeships quality. I want to do this so that trailblazers understand what we do and how it might help them, how we can and do work effectively with professional bodies and to give you all a sense of our approach to achieving assessment validity.

I’d like to explain how we’ve approached our EQA responsibilities, working closely with the Institute and tailoring our regulation to meet the needs of employers, professional bodies, training providers and of course apprentices.

As you may know, Ofqual is currently one of the 4 options that employers can choose from to provide external quality assurance for apprenticeship end point assessments (EPAs). We are the only statutory regulator for non-degree apprenticeships, which does mean we are different from other EQA providers and it is important to understand what that means in practice. But I also want to make the point that we haven’t stuck to a rigid ‘one-size-fits all’ approach; I hope my remarks will help bring this to life.

Our credentials

If you know Ofqual at all, you will know that our expertise lies in assessment. Some observers have reached a conclusion that our experience and relevance is limited to GCSEs and A levels – that we are an examinations regulator.

I get the perception. Of course we know a lot about exams, and our work on GCSE and A levels gets high levels of attention every summer.

But our expertise is equally applicable to vocational and technical qualifications that represent the significant majority of the regulated market. We oversee well over 6 million VTQ certifications every year. While it’s true that Ofqual’s initial focus was on GCSE and A levels, given that these were subject to national reform, we have pivoted in recent years to VTQ regulation that now accounts for the majority of our activity.

Whether qualification or end point assessment, there is a judgement to be made of knowledge and skills. How the assessment is designed, developed, and delivered has to be right – for the learner, the subject, the training provider and the employer.

By way of example, earlier this week I was involved in discussions with an organisation that wants us to regulate safety critical qualifications. They told me that their pass rate was nearly 100% – which sounds odd at face value. But context is everything and this qualification is all about occupational competence for practitioners. As such, the course has no set length. An experienced and quick learner might take a few months to pass, while another might take years to reach competence. That’s OK! Our regulations allow this flexible approach, recognising the particular challenges and complexities of competency based assessment. Indeed, that was the subject of recent ground breaking research by our own Professor Paul Newton, which we recently published.

Let’s consider next, an example of where effective regulation of vocational qualifications has had a positive impact on public safety. We have been working closely with the Security Industry Authority, the body that oversees licences to practice for private security guards. Why? Because this is a high risk industry, where there can be tragic consequences if, for example, door supervision isn’t carried out effectively. We work very closely with the SIA to ensure that regulated qualifications provide this assurance of occupational competence.

So I would suggest that the regulation of assessment is just as applicable and important in vocational pursuits as in academic ones. And these examples also highlight the fact that we do engage with employers and professional bodies to help them get what they want.

A current example of this is our work with the Construction Industry Training Board (CITB). Amongst other things, they are keen that, when setting new national occupational standards, awarding organisations (AOs) don’t interpret them in different ways such that they might lack consistency and comparability. Clearly, this is important, particularly where these qualifications signal occupational competence to work in an industry.

Because these qualifications are offered by AOs, we can and will work with CITB, that represents many employers, to ensure that the qualifications offered by the 30 or so AOs support their needs and those of learners. We see this as just one example of working in support of professional bodies to ensure that assessments meet their needs. To provide the regulatory teeth to help them get what they want. This is regulation in support of professional bodies, not a replacement for them.

Regulation provides us with the means to improve quality and increase confidence in qualifications. It enables us to set high standards from the start and to use our powers to protect the interests of learners and bring things back on track when they go wrong – or when they look like they might go wrong.

Given this, you might expect that the organisations we regulate might not be very keen on us! But as one of them told me recently “regulation provides clarity, consistency and confidence for users in that market.” A cynic might suggest that an organisation that is already a member of the Ofqual regulated community would be keen to curry favour with the regulator! But recognition of the value of regulation is not confined to AOs. Several employers we have worked with have also voiced their support.

EQA – a regulated approach

So, how does that assessment expertise and those regulatory powers apply to our approach to EQA in practice? We have carefully created a new and tailored approach to ensure an effective quality assurance regime to secure the standards and safe delivery of apprenticeship EPAs, which dovetails with the roles of other agencies and, importantly, meets the needs of employers.

And that tailored approach reflects the fact that we can’t, and don’t, adopt a ‘one-size-fits all’ approach to EQA. We’ve worked hard to establish a flexible approach to EQA that recognises the varied apprenticeship assessment landscape.

Before we agree to provide EQA, we apply our assessment expertise to a technical review of the Assessment Plan – confirming that it is capable of supporting quality EPAs. We work closely with the trailblazer group of employers and the Institute, feeding back to them our advice, to ensure that the assessment plan meets everyone’s needs and supports valid assessment – that is, the delivery of EPAs that test the right things, at the right level, wherever and whenever apprentices take the test.

Professionals work with us even more closely when we technically evaluate the EPA materials – a process undertaken by our assessment specialists working with relevant employer and subject experts who know best what should be being tested. So my thanks to those professionals ranging from customer service and conveyancing, to healthcare and horticulture. All these and more have worked with us over recent months to evaluate EPA materials.

This collaborative work has seen changes made to assessment materials before apprentices reach their EPA. The outcome is that apprentices on an Ofqual-regulated apprenticeship can be confident that they will be undertaking a good quality assessment. And employers and training providers can be confident that the assessment is fit for purpose, meets the requirements of the assessment plan and is comparable and consistent whichever EPAO is used.

EQA and recognition

Now, of course we can only regulate those organisations that we recognise – that is, those who we are confident have the capacity and capability to deliver EPAs safely. We look at every EPAO in our recognition process.

We have heard concerns about this process. But we should all have high expectations. It is in no-one’s interest to allow organisations to deliver EPAs without confidence that they have the capacity and capability to do so. This would lead to a race to the bottom and the undermining of quality and we will not allow it.

Equally, we recognise that some EPAOs are a different beast to some of our existing organisations. We do understand that our regulatory approach for a niche EPAO might need to be different to an organisation delivering high volume qualifications.

So we streamlined our process and, since we did, we have seen a marked increase in organisations contacting us about becoming recognised. My guess is that those actively pursuing applications will all tell you that applying for recognition has challenged them, but they will also tell you that they feel appropriately supported.

We take a pragmatic approach to recognition. For example, we are currently working with an organisation operating in a niche sector – one that wants to offer EPAs against a single standard to around 40 apprentices per year. This established professional body may be small but they are certainly expert. They might not be familiar with our rules yet, but as long as they have sufficient resources and processes in place to deliver valid assessments, we will find a way to recognise them. We think that this flexible approach to recognition is an important feature of quality assurance that we provide.

Committed AOs, with capable assessors

So we have in place important up-front checks of the EPAO and the EPA. And once an EPA is live in the market, we regulate it on an ongoing basis, bringing to bear the full range of our regulatory powers.

This includes the power to undertake audits. Over the last 6 months we’ve undertaken audits with 15 AOs across 5 apprenticeship standards. We’ve looked in particular at whether EPAOs have sufficient qualified resource to deliver EPAs – that is, experienced assessors available when required.

We have found some good practice in terms of EPAOs’ commitment to employing competent independent assessors – including those that require assessors to undertake ongoing training and standardisation.

What’s evident to us is that these AOs are taking steps to know where and when they will need assessors, and are making quite substantial efforts to ensure that those assessors are able to do a good job. That’s not to say that there are no issues, and we’ve worked with AOs to ensure any necessary improvements are being made. And we will continue to keep this area under review as volumes of EPAs increase.

Our other regulatory powers

But audit is just one of our regulatory tools. As the AOs in the room will tell you we can and do take other actions, including issuing directions and penalties. We’ve already deployed the full suite of regulatory powers to protect the interests of apprentices and employers.

For example we are taking action following a series of events that saw apprentices undertaking an EPA that were unable to upload their work to the online platform which meant their work was lost. This should not have happened and should not happen again.

That’s why we are taking regulatory action to achieve 3 aims:

1. Most importantly, to protect the interests of those apprentices involved;
2. to ensure lessons are learned so that it doesn’t happen again;
3. and to make sure other EPAOs learn from this mistake.

So what?

To conclude, the messages I’d like to leave you with are:

We have a well-developed, tried and tested approach to EQA that provides employers, training providers and apprentices with confidence that the EPA is all that it should be: that there is quality, reliability, consistency and comparability.

We take action up front to ensure that the EPAO and the EPA are fit for purpose. And we have the levers to address issues as they arise and get things back on track.

This approach works. We are hearing positive feedback from employers and others in the system – and we welcome any feedback you want to give us.

We are keen to work with professional bodies; we see regulation as a support to, rather than replacement for, these important institutions. In this context, we’d encourage trailblazers not to think of this as a binary EQA choice between regulation or a professional body – you can have both!

And finally I want to stress our commitment to supporting the Institute in fulfilling its remit to ensure the overall quality of the apprenticeship system.

Thank you very much for listening, and I’m very much looking forward to hearing your questions.

Phil Beach CBE