Sally Collier

Sally Collier spoke at The Apprenticeships Conference 2019 about external quality assurance for end-point assessments: 

I’m delighted to be here today and to have the opportunity to explain some more about our regulation of end point assessments. I want spend some time this morning considering:

  • our approach to external quality assurance (or EQA I shall call it henceforth), and how we work closely with the Institute
  • and what I believe are the wider benefits of regulation to the wider apprenticeships programme as a whole

It is very important to me that everybody involved in this complex system understands what we do and how we work very effectively with employers and professional bodies. I want to start by setting out what we’re trying to achieve through regulation, how we go about delivering our intentions, and how we can help to resolve things when they don’t go to plan.

So, what is regulation in this context?

Well, it’s not about enforcing rules for the sake of enforcing rules. Regulation is about supporting the right behaviours. And for us, those behaviours mean high standards in qualifications. As a regulator, it is important that we operate objectively and consistently. We must set out our expectations clearly, so that those who play a part in developing, delivering and awarding qualifications understand very clearly what is expected of them.

It is also important to understand why we have those expectations. We regulate for quality not quantity. Good qualifications are trusted and their value is understood. They are a tool for signalling ability and are a recognised currency in the labour market.

A key objective of ours is that the public have confidence in qualifications. Our expectation is that those who rely on qualifications can trust that individual learners have the knowledge, skills and behaviours they need.

As a regulator, we are supported in our objectives through the legal tools and powers with which we are endowed. Of course, we hope we don’t have to use them, but when we do, we use them proportionately, and in combinations that will have the best effect.

Yes, that does mean that on occasions we may issue fines and direct End-Point Assessment Organisations – or EPAOs as I shall call them henceforth - to do certain things, but it is always with the intention of securing standards and promoting public confidence, which go hand in hand. In short, regulation enables us to set and maintain high standards, and to protect the interests of learners and those who rely on qualifications.

So, how does our regulation translate specifically into EQA?

As you may know, Ofqual is currently one of 4 options that employers can choose to provide EQA for apprenticeship end point assessments.

Like other EQA providers, we work extremely closely with the Institute, supporting its overarching responsibility for apprenticeship quality. But because we are the only statutory regulator for non-degree apprenticeships, it means we are also different to other EQA providers in a number of ways.

1. Treat end point assessments as qualification

First, we treat end point assessments as qualifications. This means that we approach them with the same rigour and focus as other qualifications that we regulate, and are able to draw on all of our regulatory tools. For example, it is absolutely essential that qualification users have confidence that the organisations delivering end point assessments have the right capacity and competence.

We achieve this by requiring EPAOs that offer end point assessments regulated by us to be recognised by us. That means they have to reach the high quality bar that we set for all regulated awarding organisations, and be compliant with our rules. We believe that all EQA providers should have similarly high expectations in this area. It benefits no one to allow organisations to deliver end-point assessments without being confident in their knowledge and understanding of assessments as well as their professional subject-specific expertise. If we don’t all have similar expectations there could be a race to the bottom and quality could be undermined. We cannot allow that to happen.

2. The need to be open and flexible

Second, one of the important ways that we have responded to the reforms is to recognise that we need to be open and flexible about the type of assessment organisation that we recognise, and to be responsive to employer needs. We know that some EPAOs are different to those that we already regulate. And we appreciate that our regulatory approach for niche EPAOs needs to be different to an organisation delivering a vast breadth of qualifications. That’s why we consider each application carefully, on its own merits. Indeed, we meet with every potential applicant before they commence applying for recognition to discuss their specific circumstances.

We recognise that applying for recognition can seem a daunting process. But those who have applied to us tell us that what they do as part of their application stands them in good stead for their ongoing operations. Circumstances can be very different, so we want to understand the specific needs of particular industries, and the context and scale in which they are operating. We know that for some industries, only a certain professional body or type of organisation will have the capacity and capability to meet the needs of that particular sector.

For example, we are currently working with an established professional body, who are small but long established and certainly an expert organisation. They are operating in a niche sector and want to offer end-point assessments against a single Apprenticeship Standard to around 40 apprentices a year. There is no reason why they cannot become recognised. If they can demonstrate to us that they have sufficient resources and processes in place to deliver valid assessments, and are capable of ongoing compliance, then we will find a way to recognise them and support them to develop a full understanding of our rules.

I know that about a third of the EPAOs we recognise now are niche, sector-specific, or professional bodies themselves, and this number is growing day by day. If you are this kind of organisation and have considered recognition in the past but decided it is not for you, do look again, we would be delighted to hear from you.

So, how do we ensure quality in the end point assessments we regulate?

Our regulatory approach is tailored to each type of qualification. We have carefully created a new approach to ensure effective quality assurance of end-point assessments that secures standards and their safe delivery, and which dovetails with the roles of other agencies and, importantly, meets the needs of employers. We have set end-point assessment-specific rules and published our EQA approach so that it is clear and transparent to all who choose to use it.

Before we agree to provide EQA, we apply our assessment expertise to a technical review of the Assessment Plan - confirming that it is capable of supporting quality end-point assessments. When we do that, we work closely with the Institute to feed back to the group of employers our advice, to ensure that the assessment plan meets everyone’s needs and supports valid assessment - that is, the delivery of end-point assessments that test the right things, at the right level, wherever and whenever apprentices complete their assessment.Whether its employers in high speed rail infrastructure or accountants who have designed the standard, we recognise that they know their profession and are committed to its future – they are the ones who know what good looks like. But they can often use a little help in knowing how to judge assessment success – knowing how best to assess an individual – and that’s where our expertise comes in.

We have now reviewed 118 Assessment Plans. Last year we published findings from 100 of them in order to share best practice. As a result, we gave advice about the detail of assessment procedures – do they appropriately match the knowledge, skills and behaviours that the apprentice needs to be able to evidence in order to do the job? For example, it wouldn’t be right to assess the practical skills a butcher requires through a multiple choice test. We also gave advice on grading and grading criteria, which is notoriously difficult to get right because the grade needs to send the right signal of competence and differentiate appropriately between learners.

The lessons from our Assessment Plan review are a key first stage from our perspective, in designing end-point assessment quality in upfront. Without a decent Assessment Plan, the EPAO has less chance of designing assessment materials that will work.

A second key stage in how we ensure quality is what we call the Technical Evaluation. This is where we work with industry professionals and our own assessment specialists to review the end-point assessment materials produced by EPAOs.

In essence, we are checking that the EPAO has interpreted the Assessment Plan correctly and designed assessment materials that are sufficiently valid. To assist with this, we bring in subject experts to ensure that we get an industry perspective. For example, our team are currently working with experts in land-based and heavy vehicle engineering, teaching assistants, and learning and development practitioners, to evaluate end-point assessment materials that have recently come on to the market.

Collaborative work like this strengthens assessment. It gives employers and apprentices’ confidence that end-point assessments are fit for purpose, and it gives EPAOs – particularly those developing end-point assessments for the first time – important feedback and guidance so that they can be sure their end-point assessments are meeting the needs of their customers.

These reviews have led to changes that range from points of detail through to major differences in interpretation between EPAOs. For example, in the customer service standard we found a wide range of different assessment approaches – from multi-choice tests to portfolio submissions – that, had they gone ahead, would have led to a standard of competence meaning something different in each case.

We consider it our job to ensure that there is a consistent approach being taken across the market, and that apprentices who take their end-point assessment against one standard with one EPAO are not susceptible to different testing regimes with any other. It is extremely important to us that the apprentice who passes in Coventry has reached the same standard as the apprentice in Carlisle. And similarly, that the apprentice who passes in 2019, has reached the same standard as the apprentice who passes in 2021.

The maintenance of standards is our core business as a regulator, and we consider it equally as important in apprenticeships as in other qualifications; upholding these standards is what will give employers confidence in the apprenticeships system.

We bake quality into regulated qualifications from the start, but it is equally important that we then monitor them over their lifecycle to ensure that quality is maintained. Just as with other sectors, developments can occur that mean intervention can become necessary. At these times, it is also important that others within the sector hear and take heed of any lessons learned.

For example, last year we became increasingly alert to risks regarding assessor capacity and capability across end-point assessments. So over the last 6 months we have undertaken audits with 15 EPAOs across 5 apprenticeship standards. Our focus for these audits has been whether EPAOs have sufficient qualified people to deliver end-point assessments and that they are available when they are required. We found that EPAOs are, in general, making great efforts to ensure they have a sufficient supply of capable assessors to meet demand. But that’s not to say that they are finding it easy. In particular, we know that they are finding it challenging to access the data they need to accurately forecast end-point assessment demand, and therefore to be able to arrange and resource assessment appropriately.

It is possibly no surprise that we found stronger performing EPAOs were those that:

  • First, have set up robust systems for forecasting demand…taking into account indicative Gateway dates, how long each end-point assessment takes to conduct, and the locations of apprentices and assessors;
  • Second, are keeping their forecast under regular review - making changes where necessary, for example to take account of things like the on-programme apprenticeship taking longer than expected and the rate of EPA resits; and
  • Third, have systems for early identification of instances where a shortfall of assessors is likely, enabling them to take action.

What’s evident to us is that these EPAOs are taking steps to know where and when they will need assessors, and are making quite substantial efforts to do so. We’ve worked with specific EPAOs who have encountered problems to ensure any necessary improvements are being made. We will continue to keep this area under review as volumes of end-point assessments increase. And we will continue to undertake proactive, thematic reviews wherever appropriate and share that intelligence.

Audits are just one of our regulatory tools. We can and will take other action, including issuing directions and penalties, where that is a necessary and appropriate course of action. The new Apprenticeship system is already sufficiently mature enough for us to have utilised the full breadth of our regulatory powers to protect the interests of apprentices and maintain the confidence of employers in apprenticeships.

For example, we are currently taking action where apprentices undertaking an end-point assessment were unable to upload the work they had done in a timed assessment task to an online platform. This resulted in lost work and data, and could have resulted in an invalid assessment. We directed the Awarding Organisation concerned to stop awarding in order to protect the apprentices, and directed them to fix the problem to ensure that the apprentices were not unfairly disadvantaged. This should not have happened and should not happen again. That’s why we are taking regulatory action, and we will be publishing an announcement on this specific case today. The situation for those learners has been resolved – their interests have been protected. We need to ensure that lessons here are learned by others too, so that such a situation does not arise again.

So, as you have heard, we have a well-developed, tried and tested approach to EQA that provides employers, training providers and apprentices with confidence that the end-point assessment is all that it should be: that there is quality, reliability, consistency and comparability.

We also believe our approach is having positive benefits for the wider system too. For example, our recognition process is an established bar for the competence and capacity of effective EPAOs. It is a standard to which many EPAOs are now looking to aspire. We also know that we regulate many EPAOs for which we are not the nominated EQA provider. Organisations we regulate deliver nearly 500 end-point assessments, for 200 (approved) standards – which is over 50% of the market. The good practice of these EPAOs’ regulated operations will naturally strengthen the end-point assessment market. And, as previously indicated, we are always sharing feedback from our experiences to benefit the wider system.

In summary, I think there are 4 areas where our EQA work is benefitting the wider system:

  • First, we believe that our approach has added value to employer involvement, and not replaced it
  • Second, we believe the feedback we have provided on assessment plans has benefitted more than just the EPAOs we regulate, helping to deliver assessments that are conducted in the most appropriate ways
  • Third, we know our approach is helping to ensure that there is consistency between EPAOs and has set a bar for their competence, so standards can be more easily maintained
  • And fourth, where we have identified challenges we have taken action and shared these concerns, so the wider system can be on the lookout for similar issues.

I hope that gives you a useful overview of our work in this important area.

You may also be interested in these articles:

Sponsored Video

Advertisers

Upcoming FE Events

Advertiser Skyscrapers

Newsroom Activity

NCH London added a new event 6 hours

New College of the Humanities is pleased to announce that...

New College of the Humanities is pleased to announce that we will be taking part in National Apprenticeship Week 2021 #NAW2021 On the 9th February...

  • Tuesday, 09 February 2021 09:00 AM
  • London - Online
Vanessa Skinner has published a new article: Supporting Student Wellbeing During a Pandemic 12 hours 21 minutes ago

Latest Education News

Further Education News

The FE News Channel gives you the latest education news and updates on emerging education strategies and the #FutureofEducation and the #FutureofWork.

Providing trustworthy and positive Further Education news and views since 2003, we are a digital news channel with a mixture of written word articles, podcasts and videos. Our specialisation is providing you with a mixture of the latest education news, our stance is always positive, sector building and sharing different perspectives and views from thought leaders, to provide you with a think tank of new ideas and solutions to bring the education sector together and come up with new innovative solutions and ideas.

FE News publish exclusive peer to peer thought leadership articles from our feature writers, as well as user generated content across our network of over 3000 Newsrooms, offering multiple sources of the latest education news across the Education and Employability sectors.

FE News also broadcast live events, podcasts with leading experts and thought leaders, webinars, video interviews and Further Education news bulletins so you receive the latest developments in Skills News and across the Apprenticeship, Further Education and Employability sectors.

Every week FE News has over 200 articles and new pieces of content per week. We are a news channel providing the latest Further Education News, giving insight from multiple sources on the latest education policy developments, latest strategies, through to our thought leaders who provide blue sky thinking strategy, best practice and innovation to help look into the future developments for education and the future of work.

In May 2020, FE News had over 120,000 unique visitors according to Google Analytics and over 200 new pieces of news content every week, from thought leadership articles, to the latest education news via written word, podcasts, video to press releases from across the sector.

We thought it would be helpful to explain how we tier our latest education news content and how you can get involved and understand how you can read the latest daily Further Education news and how we structure our FE Week of content:

Main Features

Our main features are exclusive and are thought leadership articles and blue sky thinking with experts writing peer to peer news articles about the future of education and the future of work. The focus is solution led thought leadership, sharing best practice, innovation and emerging strategy. These are often articles about the future of education and the future of work, they often then create future education news articles. We limit our main features to a maximum of 20 per week, as they are often about new concepts and new thought processes. Our main features are also exclusive articles responding to the latest education news, maybe an insight from an expert into a policy announcement or response to an education think tank report or a white paper.

FE Voices

FE Voices was originally set up as a section on FE News to give a voice back to the sector. As we now have over 3,000 newsrooms and contributors, FE Voices are usually thought leadership articles, they don’t necessarily have to be exclusive, but usually are, they are slightly shorter than Main Features. FE Voices can include more mixed media with the Further Education News articles, such as embedded podcasts and videos. Our sector response articles asking for different comments and opinions to education policy announcements or responding to a report of white paper are usually held in the FE Voices section. If we have a live podcast in an evening or a radio show such as SkillsWorldLive radio show, the next morning we place the FE podcast recording in the FE Voices section.

Sector News

In sector news we have a blend of content from Press Releases, education resources, reports, education research, white papers from a range of contributors. We have a lot of positive education news articles from colleges, awarding organisations and Apprenticeship Training Providers, press releases from DfE to Think Tanks giving the overview of a report, through to helpful resources to help you with delivering education strategies to your learners and students.

Podcasts

We have a range of education podcasts on FE News, from hour long full production FE podcasts such as SkillsWorldLive in conjunction with the Federation of Awarding Bodies, to weekly podcasts from experts and thought leaders, providing advice and guidance to leaders. FE News also record podcasts at conferences and events, giving you one on one podcasts with education and skills experts on the latest strategies and developments.

We have over 150 education podcasts on FE News, ranging from EdTech podcasts with experts discussing Education 4.0 and how technology is complimenting and transforming education, to podcasts with experts discussing education research, the future of work, how to develop skills systems for jobs of the future to interviews with the Apprenticeship and Skills Minister.

We record our own exclusive FE News podcasts, work in conjunction with sector partners such as FAB to create weekly podcasts and daily education podcasts, through to working with sector leaders creating exclusive education news podcasts.

Education Video Interviews

FE News have over 700 FE Video interviews and have been recording education video interviews with experts for over 12 years. These are usually vox pop video interviews with experts across education and work, discussing blue sky thinking ideas and views about the future of education and work.

Events

FE News has a free events calendar to check out the latest conferences, webinars and events to keep up to date with the latest education news and strategies.

FE Newsrooms

The FE Newsroom is home to your content if you are a FE News contributor. It also help the audience develop relationship with either you as an individual or your organisation as they can click through and ‘box set’ consume all of your previous thought leadership articles, latest education news press releases, videos and education podcasts.

Do you want to contribute, share your ideas or vision or share a press release?

If you want to write a thought leadership article, share your ideas and vision for the future of education or the future of work, write a press release sharing the latest education news or contribute to a podcast, first of all you need to set up a FE Newsroom login (which is free): once the team have approved your newsroom (all content, newsrooms are all approved by a member of the FE News team- no robots are used in this process!), you can then start adding content (again all articles, videos and podcasts are all approved by the FE News editorial team before they go live on FE News). As all newsrooms and content are approved by the FE News team, there will be a slight delay on the team being able to review and approve content.

 RSS IconRSS Feed Selection Page