The Benefits, Challenges and Best Practice of Remote End-point Assessment

During the summer of 2022 I ran a survey with end-point assessment organisations on their experiences with remote assessment. The purpose of the survey was to identify the benefits, challenges, and good practice in remote end-point assessment. The survey, completed by 19 EpAOs, provided a fascinating insight into the post covid world of remote assessment. Some EpAOs had only ever delivered remotely, whereas others have transitioned to remote, either as a result of Covid, or as a result of the decision by IfATE to retain many of the flexibilities that were introduced during Covid. A summary of the flexibilities that have been retained can be found on the IfATE website.

Before I delve into the detail of the survey, I thought it would be useful to summarise the assessment methods across apprenticeship standards, as the assessment method impacts its ability to be delivered remotely.  The most common EPA activity is an interview or professional discussion, used in 65% of standards, test/exams are used in 38% of standards, projects in 29% of standards, observations in 23% of standards, practical demonstrations in 17% of standards, and presentations in 9% of standards. Observations, practical assessments and tests /exams are most commonly used at levels 2 and 3; Projects are most commonly used at level 4 and above.

The Benefits

Starting with the overarching benefits of remote assessment, every EpAO I interviewed mentioned the positive environmental impact through reduced travel. Outside of the environmental impact, the benefits reported can be categorised into 4 key areas: timeliness, efficiency, flexibility, and quality. 

EpAOs reported that the ability to deliver remotely made it easier to schedule and reschedule assessments, and therefore deliver a more responsive, and scalable, service. Some assessment plans require assessments with more than one assessor and/or expert in attendance, and for the EpAOs that operated these assessment plans, they reported that the ability to delivery remotely made the arranging of such assessments much easier, and far less costly than having to arrange a face-to-face assessment. 

A couple of EpAOs reported that they have been able to reduce their cancellation timeframes because of the increased speed at which they can arrange things when delivering remotely as compared to face to face. Some EpAOs also reported being able to open up to a wider geographic as they were no longer restricted by assessor or apprentice travel.  Many also reported a greater flexibility in delivery times, as remote assessment enabled them to deliver outside of standard office/site hours, making it easier to work around apprentice shift patterns and busy periods at work. EpAOs also highlighted that a lot more could be carried out in a lot less time, for example, what was previously travel time, could now be used as assessor prep and write up time.

EpAOs also reported a positive impact on quality. When delivering remotely most (not all) assessments are recorded. The recording of assessments has enabled IQA to take place afterwards whereas previously the IQA would have travelled to attend in person. One EpAO also mentioned that previously, when delivering face to face, the attendance of an IQA to observe made the apprentice more nervous, so the move to remote and recording helped take the pressure away. 

Some EpAOs also reported higher quality assessor reports as assessors have been able to focus on capturing high quality details knowing that there is a recorded back up, rather than writing down everything the apprentice says. Many EpAOs also reported using the recording as part of their standardisation work (subject to the appropriate permissions), which has been helpful at generating debate and bringing consistency across their assessors and assessments. There was also mention of incidental learning as assessors (and apprentices) have learnt to adapt to new ways of doing things and new technology, which can only be a good thing for future self-development.

The Challenges

Moving on to challenges, the most common assessment methods facing challenges, when delivering remotely, were the observation and the practical demonstration. Some EpAOs undertook some innovative approaches to enable them to continue to deliver observations, whereas for others, the logistics and safety concerns were too great to enable the observation to take place, hence why some IfATE flexibilities permitted alternative assessment methods during covid. Back in 2020, I wrote some articles, following discussions with EpAOs on how they have adapted to Covid – “the art of the possible” and “key considerations when delivering remotely”. Two EpAOs reported that they were starting to see employers request a move back to face to face for observations and practical demonstrations/tasks, even where it can still be delivered remotely.

Unsurprisingly, the main area of challenge, reported by all EpAOs, was that of technology. This broadly fitted into 3 areas – internet capability, security, and technology.

1. In relation to internet capability, many reported challenges with the internet dropping out or freezing which resulted in assessments being paused, delayed or cancelled.
2. In relation to security, there were 4 keys areas. The first related to business security, such as firewalls preventing certain apps or remote assessment tools from operating. The second related to commercial security, such as commercially sensitive or confidential information being on video and/or recorded. The third area related to safeguarding, when using video or recording equipment when there are young or vulnerable people. Finally, some reported forgetfulness, where the apprentice forgets their password to log into the assessment systems (for example, they set their password access at gateway and by the time of the assessment they had forgotten their password access details). Indirectly linked to security, one EpAO reported challenges with ID checks at the start of the assessment because the apprentice’s camera did not focus clearly enough to see the details of, for example, a passport.
3. When it came technology, EpAOs used a variety of platforms for the running of remote assessments. Some used whatever system was used by the employer/apprentice (subject to a range of checks), others used their own systems. This created challenges on 2 fronts, firstly compatibility with customer systems (see above example of being blocked by firewalls), and secondly the ability of assessors and apprentices to use the systems, or a system, that they are not familiar with.

Another area often mentioned by EpAOs related to the ability to operate remotely whilst remaining aligned to the assessment plan. For example,

• (a) if an exam-based assessment allows the apprentice to use notepaper to make notes during an assessment, how can they be securely be disposed of when the remote assessment is completed?
• (b) most assessment plans provide time allowance of +/-10%, if it is a group assessment how can that be manged remotely?;
• (c) if an exam is open book how can a remote invigilator have sight of both the apprentice and the books?.

One survey respondent reported that remote assessment presented an increased risk of malpractice, but unfortunately did not provide examples, so I am not able to expand on that.  Most assessment plans also refer to suitable rooms for assessment such as quiet and free from distractions, some EpAOs reported that some employers simply did not have suitable rooms to cater for remote delivery, for example, constant background noise, or very weak internet signal.

One EpAO reported that their EQA provider was very slow to respond to queries, and gave poor advice, which resulted in delays to the delivery of assessments. Another questioned where all the decisions sat, for example, was it the IfATE or EQA provider that agreed flexibilities?  This is now much clearer as the IfATE website shows the permitted flexibilities, and details which flexibilities must be approved by the EQA provider.

A final area of challenge related to professionalism. This covered both the apprentice and assessor. Some EpAOs reported that remote assessments were not being treated as seriously by the apprentices as when they were delivered face to face. Other EpAOs reported that their standardisation work had identified that some assessors may not be operating as effectively when delivering remotely, such as only half their face on screen, or constantly looking down to take notes without warning the apprentice. One thing I did find interesting was that only 2 of the EpAOs that took part in the survey (10%) reported training to their assessors in the use of the remote assessment technology, but they did all report that their assessors and apprentices had adapted and transitioned well.

Another interesting area of feedback, that straddled both benefits and challenges, related to the impact on equality and inclusion. Although the survey did not provide a lot of detail, some EpAOs reported that remote assessment had been great for inclusion and had seen some introverted apprentices shine, whereas others reported that it impacted the ability of some apprentices to perform well as they were nervous of the technology.  It would be interesting to find out if apprentices receive training on the use of the remote assessment platforms.

Overcoming challenges

I have listed a range of benefits and challenges, but what I find most inspirational is the innovation that has been displayed to overcome such challenges.  Each EpAO has approached this differently and there is no one size fits all, but here are some highlights, alongside some thoughts of my own:

1. Always test the systems before an assessment – the best practice identified from speaking with EpAOs is to adopt a range of methods to ensure everyone is prepared. This may cover written guidance and instructions, gateway checks for compatibility, tests of the system with assessor and apprentice a few days before the assessment, or to be logged in no less than 15 minutes before the assessment begins.
2. Provide training – only 2 of the EpAOs reported providing training to assessors. None had reported issues with assessor ability, but if you want to deliver the best assessment experience, your assessors need to be confident in using technology and in dealing with any challenges it may bring. This equally applies to the apprentices undertaking the assessment.
3. Update your contingency plans – you will always have had contingency plans in place, such as what to do if there is a fire, or transport delays. Make sure the contingency plan, and associated testing of the plan, is updated to cater for remote assessments, such as what to do if the internet cuts out, if the assessor or apprentice can’t gain access to the system, or if the on-line assessment software fails.
4. Update your reasonable adjustments and special consideration policies. With the introduction of remote assessment, the types of requests for reasonable adjustments and special considerations will have changed.  Make sure the policies and those involved in its implementation evolve with the changes to delivery methods and technology.
5. Provide protocols for assessor conduct.  A few EpAOs reported that, since the introduction of remote assessment and its associated recording and viewing by IQAs and at standardisation meetings, they have identified areas where quality can be improved when delivering remotely, such as protocols on the positioning of an assessor on screen, body language end eye contact, and set scripts to introduce and conclude the assessment.
6. Update your data protection and sharing policies. Delivering remotely usually brings with it the ability to record.  It is absolutely critical that your data sharing policies and protocols cover this and who can view the recordings, and to re-check these permissions at the start of each remotely delivered assessment. Remember there are some employers or standards where the use of video equipment/cameras or recording is not possible for security, sensitivity or safeguarding reasons, so you should check on or before gateway whether cameras or recording can be used. Linked to this, make sure you have sufficient electronic storage and associated security; recordings take up a lot more space than word or pdf documents.
7. Engage the training provider and employer – you need to be sure that remote delivery is possible and that all involved are able to provide suitable premises, equipment, and are able to operate it.  The ESFA funding Rules and ESFA Conditions for EpAOs highlight the need for readiness, and readiness for remote assessment is no exception. 

I hope this article provides a useful summary. Remember, all my previous articles to support the EpAO sector can be accessed via the FE News channel and via my LinkedIn news feed.