Improving quality assurance in integrated end-point assessment

In this blog, SDN and Evoko’s university and end-point assessment specialist, Emily Hughes, gives some practical reflections on external quality assurance for integrated end-point assessment.

As universities move into the core of the academic years delivery, many are supporting apprentices as they progress to end-point assessments for their integrated degree apprenticeships.

I spend a lot of time talking to university staff about this topic. Some universities want to refine the approach and understand that it is different to on programme assessment. Others are simply thinking about it in terms of ‘needing a different assessor’.

The external quality assurance of the integrated end-point assessment has been a long-held discussion topic across the apprenticeships sector.  With the transition of the EQA arrangements to Ofqual for most non-integrated standards well underway, there has been more discussion about the rigour of the external quality assurance arrangement for the integrated apprenticeship equivalents.

Working across a wide range of universities, it is interesting to see the breadth of interpretations of the external quality assurance guidance to providers recently published by the Office for Students (OfS).  As this external quality assurance approach becomes embedded, all universities should be giving it consideration as they deliver the integrated end-point assessments this summer.

A great deal can be learnt from the Ofqual regulated approach to end-point assessment and the application of their conditions and criteria within assessment.

No matter how the end-point assessment is externally quality assured, there are some core principles and areas I often find underpinning discussion, not only because they are at the core of the Institute for Apprenticeships and Technical Education (IFaTE) and OfS external quality assurance guidance but also as it is at the heart of good end-point assessment.

How do you know that the assessment materials are relevant, reliable, efficient and accessible?

Each apprenticeship assessment plan provides guidance and information about the assessment methods, deployment and any additional requirements of the assessment activities.  The challenge for universities as the end-point assessment organisation, is the need to meet additional external quality assurance expectations that reach beyond their usual day-to-day activity.

The biggest area that many find a challenge is around the separation from the delivery team and the security of assessment materials with regards to tracking of access. How do you know that someone teaching content on programme doesn’t have access to the assessment materials for the end-point assessment, so they don’t “teach to the test”?

There is also the need to demonstrate refinement of assessment materials and genuine industry feedback before signoff and deployment to live assessment. This will often be new within a university to include feedback on the material prior to deployment. How do you make sure this is valuable and not just a tick box exercise? How can you show security of materials and assessment software testing?

How do you know the support materials are telling the right people what they need to know?

The need to provide support and guidance to students is nothing new within higher education. However, the need to also now support the employer who is equally front and centre within apprenticeships, can often be a new area to consider.

The need for an apprentice to understand not only what is in the assessment, but how it will be delivery, by whom, and the rules around retakes and results need to be clear.

It is important that an employer can understand the relevance of the support and assessment materials and how, as employers, they can also contribute to feedback loops to help improve and refine the assessments.

Support materials need to have absolute clarity around the difference between ongoing assessment and end-point assessment as this provides the launch pad for understanding and support.

How do you know the end-point assessment delivery is aligned to the requirements of the assessment plan? How are you assured the results will provide reliable and valid assessment outcome?

Planning for the delivery of an integrated end-point assessment may, in many ways, seem no different to that of any other degree assessment. However, alignment to the assessment plans and apprenticeship standard content and the production of reliable, comparable and valid results will need to be considered throughout.

It may also require some consideration of the alignment between degree requirements and that of the assessment plan. For example, could an apprentice pass a degree module with a mark of 45% but not the end-point assessment which requires a 50% minimum pass mark? If this happened, what would be the consequences and how can this be administered, recorded and processed?

The volumes of assessment delivery needs to be considered, alongside the accessibility of assessment methods that may be less familiar within a higher education setting.  Cross referencing policy and process for reasonable adjustments against the IfATE guidance can be a helpful starting point for this.

Is there a robust selection and recruitment process that makes sure the right people are undertaking the design and assessment activities? Will they be credible and make a judgement that employers and apprentices will respect?

I have seen many universities using the same process for the end-point assessment as for university wide external examiners – whilst there are areas of this that can be shared, the greater focus on occupational and assessment skills is often missed. The approach for the promotion and monitoring of ongoing assessor CPD tracking and the schedule for assessor training, specifically related to delivery and moderation of the end-point assessment, needs to be developed and deployed for each standard offered.

Consideration for the required volume of assessors is also often overlooked with an assumption one assessor is sufficient. However, this doesn’t take into account the needs for mitigation of any conflicts of interest, retakes and resits.  You also need to consider how you will cover basic events like sickness of the assessor on the assessment day. End-point assessment is not just paper based marking that can be pushed to another day.  It is often a live assessment activity the apprentice has prepared for and had booked for some time. How can you ensure it will happen as planned?

Conflict of interest declarations and tracking can also need refinement beyond what happens in other areas of the university.  A single declaration at the start of an appointment will not be sufficient. How is this monitored and updated over the lifetime of an appointment, against each assessment cohort and delivery cycle?

How do your quality assurance and feedback processes support continuous improvement in all areas of the end-point assessment cycle?

In considering policies and processes within the integrated end-point assessment area there are core areas that may need a different, more focused approach.  Policies will need specified ownership and accountability. Are they aligned and fit for purpose against apprenticeship and external quality assurance requirements?

There needs to be a specific assessment data management processes for end-point assessment activity. How is gateway documented and tracked and how is the outcome and certification evidenced and recorded?

How are you undertaking and documenting engagement with employers and apprentices in preparation for end-point assessment? How does this feed wider continual improvement?

Universities are highly experienced experts in delivering content and assessment. However, end-point assessment can bring a different level of rigour and oversight that, for some, will be less familiar and often requires changes to established practices.

Taking time to think about the journey of the assessment materials, apprentice, employer and assessor will make sure all areas are considered and addressed. This will make sure the end-point assessment is a rich and rewarding experience for all, whilst meeting the highest standards of external quality assurance.

By Emily Hughes, Senior Associate at SDN and SDN + Evoko’s university end-point assessment specialist