Tim Chewter, Head of Research and Communications, Strategic Development Network (SDN)

If you’ve had chance to look through some of the new apprenticeship assessment plans, you will see some assessment methods that are new, and others that are familiar. However, even if they are familiar, the high-stakes nature of end-point assessment means the way you conduct the assessment, or prepare your apprentices for it, will be different.

Over the last few weeks, we’ve published a series of articles looking at different assessment methods specified in the new assessment plans, and started to unpick what they look like in practice in the context of high-stakes end-point assessment.

(If you’ve missed our earlier posts, here’s our introduction on how end-point assessment is shaping up, and insights into the professional discussion, presentation/showcase and practical assessments. The Institute for Apprenticeships is also starting to release early guidance on assessment methods too.)

So, this week, let’s tackle the Multiple Choice Test, Welcome to Article #4 ...

The basics

56% of all assessment plans include written or online knowledge tests, of which half are multiple choice questionnaires. If you’ve been a trainer or assessor in the sector for a while, multiple choice tests will be nothing new. But let’s quickly cover off the basics…

Multiple choice tests comprise a set of questions where the candidate needs to select the correct response from a range of presented answers. The number and complexity of possible answers is used to increase the degree of difficulty for the assessment.

Multiple choice tests form only part of the end-point assessment, and sit within a suite of assessments to measure the competence of the apprentice overall. Multiple choice tests are used to measure breadth and depth of knowledge. They can also be useful in validating (but not directly assessing) practical skills and can help to formulate judgments on behaviour.

The validity, reliability and robustness of the test depends on the quality of questionnaire design. High quality tests have a range of plausible responses in a narrow range of possible answers, and require the apprentice to think through and carefully distill their knowledge of the subject, to select the right answer. Tests will be significantly less valid where low quality or leading response options are given.

Advantages of this assessment method

Multiple choice tests generate numerical scores that can be mapped to a grading profile. It provides a rapid “check” score that can rank candidates across a peer group or provide supplementary evidence of performance as part of a range of assessment methods.

For those conducting the on-programme training, multiple choice tests can be a helpful way to measure the apprentice’s progress and underpinning knowledge, as well as preparing them for the end-point assessment.

Once developed, these tests are easy and cost effective to administer, particularly when using technology platforms that allow you to automate grading and moderation.

Risks of this assessment method

With these forms of assessment, it can be time consuming to develop question banks, which will also need to be updated regularly. It is also difficult to construct a valid and reliable suite of questions. The range of responses needs to be plausible and the correct responses should not be immediately obvious.

Multiple choice tests can be susceptible to random chance. Even in complex questionnaires, up to 15% of the final achievement can be attributable to chance, which should be accounted for in pass and grading thresholds.

Those who have conducted some of the first end-point assessments have reported that apprentices are often anxious and sometimes unprepared when they come to sit the test. End-point assessment is high-stakes and multiple choice tests may be invigilated by an assessor the apprentice has never met, sometimes in unfamiliar environments. Training providers have an important role in preparing apprentices for the end-point assessment, and making sure they are familiar and comfortable with such tests. They may also have a role to play in helping to invigilate the end-point assessment test (particularly where being conducted remotely by the EPAO)

What it means in practice…

  • EPAOs will need to put in sufficient time and resource to design high quality and robust questionnaires. Where done well, this assessment can be highly sophisticated. Where done badly, it can be the weakest form of assessment and lead to inaccurate grading. Assessment specialists, like SDN or others, can work with you to help get this right. As a training provider, the EPAO may not provide practice question banks, so you may need to carefully design your own practice assessment questions too.
  • This form of assessment is prone to statistical error and inherent chance, so it’s important that your grade thresholds and moderating processes reflect this
  • Multiple choice tests lend itself to remote testing – looking at ways to automate and conduct tests online can help to make the assessment cost effective
  • Compared with other assessment methods, end-point assessors will have very little opportunity to put the apprentice at ease, so it’s important the training provider prepares the apprentice well so they are comfortable and familiar with this form of assessment

 

Tim Chewter, Head of Research and Communications, Strategic Development Network (SDN)

Introduction

Article #1 The Professional Discussion

Article #2 Presentation/Showcase

Article #3 Practical Assessments

To get to grips with end-point assessment in more depth, places are now available on our Level 3 Award course in Undertaking End-Point Assessment. SDN are also producing a set of recorded presentations covering the main end-point assessment methods and critical areas of practice.

These will be available at the beginning of March. Find out more here: www.strategicdevelopmentnetwork.co.uk/sdnevent

More updates and practice can be accessed on our new End-Point Assessment LinkedIn Group and through our mailing list – feel free to join!

Let us know if you have any comments at the bottom of this page.

You may also be interested in these articles:

Register, Login or Login with your Social Media account:


Advertisers

Advertiser Skyscrapers

Video Advert

Latest Education News

Further Education News

The FE News Channel gives you the latest education news and updates on emerging education strategies and the #FutureofEducation and the #FutureofWork.

Providing trustworthy and positive Further Education news and views since 2003, we are a digital news channel with a mixture of written word articles, podcasts and videos. Our specialisation is providing you with a mixture of the latest education news, our stance is always positive, sector building and sharing different perspectives and views from thought leaders, to provide you with a think tank of new ideas and solutions to bring the education sector together and come up with new innovative solutions and ideas.

FE News publish exclusive peer to peer thought leadership articles from our feature writers, as well as user generated content across our network of over 3000 Newsrooms, offering multiple sources of the latest education news across the Education and Employability sectors.

FE News also broadcast live events, podcasts with leading experts and thought leaders, webinars, video interviews and Further Education news bulletins so you receive the latest developments in Skills News and across the Apprenticeship, Further Education and Employability sectors.

Every week FE News has over 200 articles and new pieces of content per week. We are a news channel providing the latest Further Education News, giving insight from multiple sources on the latest education policy developments, latest strategies, through to our thought leaders who provide blue sky thinking strategy, best practice and innovation to help look into the future developments for education and the future of work.

In May 2020, FE News had over 120,000 unique visitors according to Google Analytics and over 200 new pieces of news content every week, from thought leadership articles, to the latest education news via written word, podcasts, video to press releases from across the sector.

We thought it would be helpful to explain how we tier our latest education news content and how you can get involved and understand how you can read the latest daily Further Education news and how we structure our FE Week of content:

Main Features

Our main features are exclusive and are thought leadership articles and blue sky thinking with experts writing peer to peer news articles about the future of education and the future of work. The focus is solution led thought leadership, sharing best practice, innovation and emerging strategy. These are often articles about the future of education and the future of work, they often then create future education news articles. We limit our main features to a maximum of 20 per week, as they are often about new concepts and new thought processes. Our main features are also exclusive articles responding to the latest education news, maybe an insight from an expert into a policy announcement or response to an education think tank report or a white paper.

FE Voices

FE Voices was originally set up as a section on FE News to give a voice back to the sector. As we now have over 3,000 newsrooms and contributors, FE Voices are usually thought leadership articles, they don’t necessarily have to be exclusive, but usually are, they are slightly shorter than Main Features. FE Voices can include more mixed media with the Further Education News articles, such as embedded podcasts and videos. Our sector response articles asking for different comments and opinions to education policy announcements or responding to a report of white paper are usually held in the FE Voices section. If we have a live podcast in an evening or a radio show such as SkillsWorldLive radio show, the next morning we place the FE podcast recording in the FE Voices section.

Sector News

In sector news we have a blend of content from Press Releases, education resources, reports, education research, white papers from a range of contributors. We have a lot of positive education news articles from colleges, awarding organisations and Apprenticeship Training Providers, press releases from DfE to Think Tanks giving the overview of a report, through to helpful resources to help you with delivering education strategies to your learners and students.

Podcasts

We have a range of education podcasts on FE News, from hour long full production FE podcasts such as SkillsWorldLive in conjunction with the Federation of Awarding Bodies, to weekly podcasts from experts and thought leaders, providing advice and guidance to leaders. FE News also record podcasts at conferences and events, giving you one on one podcasts with education and skills experts on the latest strategies and developments.

We have over 150 education podcasts on FE News, ranging from EdTech podcasts with experts discussing Education 4.0 and how technology is complimenting and transforming education, to podcasts with experts discussing education research, the future of work, how to develop skills systems for jobs of the future to interviews with the Apprenticeship and Skills Minister.

We record our own exclusive FE News podcasts, work in conjunction with sector partners such as FAB to create weekly podcasts and daily education podcasts, through to working with sector leaders creating exclusive education news podcasts.

Education Video Interviews

FE News have over 700 FE Video interviews and have been recording education video interviews with experts for over 12 years. These are usually vox pop video interviews with experts across education and work, discussing blue sky thinking ideas and views about the future of education and work.

Events

FE News has a free events calendar to check out the latest conferences, webinars and events to keep up to date with the latest education news and strategies.

FE Newsrooms

The FE Newsroom is home to your content if you are a FE News contributor. It also help the audience develop relationship with either you as an individual or your organisation as they can click through and ‘box set’ consume all of your previous thought leadership articles, latest education news press releases, videos and education podcasts.

Do you want to contribute, share your ideas or vision or share a press release?

If you want to write a thought leadership article, share your ideas and vision for the future of education or the future of work, write a press release sharing the latest education news or contribute to a podcast, first of all you need to set up a FE Newsroom login (which is free): once the team have approved your newsroom (all content, newsrooms are all approved by a member of the FE News team- no robots are used in this process!), you can then start adding content (again all articles, videos and podcasts are all approved by the FE News editorial team before they go live on FE News). As all newsrooms and content are approved by the FE News team, there will be a slight delay on the team being able to review and approve content.

 RSS IconRSS Feed Selection Page