Matt Wingfield, Chief Business Development Officer at Digital Assess

A group of UK technologists and academics have demonstrated that a new method of assessment, similar to that used in optometry testing, can improve exam grading accuracy by up to 30% from the current reported average of 56.2%.

The model developed by education technology company Digital Assess, with consultancy from exam research experts Professor Richard Kimbell (Goldsmiths, University of London) and Dr Alastair Pollitt (Cambridge), is based on Adaptive Comparative Judgement (ACJ), and allows examiners to more precisely identify the standard of an exam script. It has been proven to significantly increase marking accuracy in open ended assessment.

Based on the Law of Comparative Judgement, ACJ proves that people are better at making comparative, paired judgements rather than absolute ones. This logic is utilised by opticians to accurately assess the required prescription of a patient. Rather than asking patients to state precisely how much clearer the lens makes their sight, they are instead asked to repeatedly compare one lens against another and determine which helps them to see more clearly. After a number of comparisons an extremely accurate conclusion is reached.

Professor Richard Kimbell explains: “In the current exam system, examiners make an absolute judgement by marking every paper according to a mark scheme. We have shown that asking the same markers to compare one paper to another side-by-side and declare which is better can produce more consistent results. Repeating this process multiple times with multiple assessors enables a mathematical algorithm to produce a reliable ranking of all candidates. Grade boundaries can then be applied against this.” 

ACJ has been proven to significantly improve accuracy in as few as 8-10 sets of paired comparisons. A recent Ofqual report showed that in 2016, 43.8% of all reviewed marks for GCSE, AS and A-level exam scripts were returned with a different mark. Grade changes were more prevalent in subjects where the nature of assessment is more subjective than others.

Soft skills such as ‘critical thinking’ or ‘original interpretation’ are difficult to measure consistently, and mark schemes are less applicable to these than to hard skills such as factual or theoretical knowledge, leaving them open to the discretion of one marker.

Matt Wingfield, Chief Business Development Officer at Digital Assess, explains: “Mark schemes lend themselves well to regurgitating facts and ticking boxes, but assessing soft skills is much harder, as they are inherently more subjective and mean different things to different markers. This failure to accurately assess soft skills is of little use to employers, who actively seek these when recruiting.”

“It is unrealistic to assume all examiners will form the same judgements, and ACJ takes this requirement away by automatically incorporating lots of expert opinions into the final ranking. It also makes it easier for examiners to clearly see what good work looks like, because they can compare an entire cohort of work, instead of looking at one in isolation and using only a mark scheme as a gauge.”

“The outcome of an eye test is by definition ‘high stakes’ - where people’s health is involved, it needs to be highly accurate - but exam results can determine people’s future, and should be no less accurate. Why not apply the logic of one to the other? The traditional assessment method has been historically unfair, and no doubt we will again see lots of exam appeals following results day this year.”

ACJ has consistently proven its accuracy in tests with awarding bodies and leading institutions worldwide, and is also applicable to coursework or portfolios of work, as well as end point assessment.

You may also be interested in these articles:

Sponsored Video

Register, Login or Login with your Social Media account:


Advertisers

Upcoming FE Events

Advertiser Skyscrapers

Newsroom Activity

Latest Education News

Further Education News

The FE News Channel gives you the latest education news and updates on emerging education strategies and the #FutureofEducation and the #FutureofWork.

Providing trustworthy and positive Further Education news and views since 2003, we are a digital news channel with a mixture of written word articles, podcasts and videos. Our specialisation is providing you with a mixture of the latest education news, our stance is always positive, sector building and sharing different perspectives and views from thought leaders, to provide you with a think tank of new ideas and solutions to bring the education sector together and come up with new innovative solutions and ideas.

FE News publish exclusive peer to peer thought leadership articles from our feature writers, as well as user generated content across our network of over 3000 Newsrooms, offering multiple sources of the latest education news across the Education and Employability sectors.

FE News also broadcast live events, podcasts with leading experts and thought leaders, webinars, video interviews and Further Education news bulletins so you receive the latest developments in Skills News and across the Apprenticeship, Further Education and Employability sectors.

Every week FE News has over 200 articles and new pieces of content per week. We are a news channel providing the latest Further Education News, giving insight from multiple sources on the latest education policy developments, latest strategies, through to our thought leaders who provide blue sky thinking strategy, best practice and innovation to help look into the future developments for education and the future of work.

In Jan 2021, FE News had over 173,000 unique visitors according to Google Analytics and over 200 new pieces of news content every week, from thought leadership articles, to the latest education news via written word, podcasts, video to press releases from across the sector, putting us in the top 2,000 websites in the UK.

We thought it would be helpful to explain how we tier our latest education news content and how you can get involved and understand how you can read the latest daily Further Education news and how we structure our FE Week of content:

Main Features

Our main features are exclusive and are thought leadership articles and blue sky thinking with experts writing peer to peer news articles about the future of education and the future of work. The focus is solution led thought leadership, sharing best practice, innovation and emerging strategy. These are often articles about the future of education and the future of work, they often then create future education news articles. We limit our main features to a maximum of 20 per week, as they are often about new concepts and new thought processes. Our main features are also exclusive articles responding to the latest education news, maybe an insight from an expert into a policy announcement or response to an education think tank report or a white paper.

FE Voices

FE Voices was originally set up as a section on FE News to give a voice back to the sector. As we now have over 3,000 newsrooms and contributors, FE Voices are usually thought leadership articles, they don’t necessarily have to be exclusive, but usually are, they are slightly shorter than Main Features. FE Voices can include more mixed media with the Further Education News articles, such as embedded podcasts and videos. Our sector response articles asking for different comments and opinions to education policy announcements or responding to a report of white paper are usually held in the FE Voices section. If we have a live podcast in an evening or a radio show such as SkillsWorldLive radio show, the next morning we place the FE podcast recording in the FE Voices section.

Sector News

In sector news we have a blend of content from Press Releases, education resources, reports, education research, white papers from a range of contributors. We have a lot of positive education news articles from colleges, awarding organisations and Apprenticeship Training Providers, press releases from DfE to Think Tanks giving the overview of a report, through to helpful resources to help you with delivering education strategies to your learners and students.

Podcasts

We have a range of education podcasts on FE News, from hour long full production FE podcasts such as SkillsWorldLive in conjunction with the Federation of Awarding Bodies, to weekly podcasts from experts and thought leaders, providing advice and guidance to leaders. FE News also record podcasts at conferences and events, giving you one on one podcasts with education and skills experts on the latest strategies and developments.

We have over 150 education podcasts on FE News, ranging from EdTech podcasts with experts discussing Education 4.0 and how technology is complimenting and transforming education, to podcasts with experts discussing education research, the future of work, how to develop skills systems for jobs of the future to interviews with the Apprenticeship and Skills Minister.

We record our own exclusive FE News podcasts, work in conjunction with sector partners such as FAB to create weekly podcasts and daily education podcasts, through to working with sector leaders creating exclusive education news podcasts.

Education Video Interviews

FE News have over 700 FE Video interviews and have been recording education video interviews with experts for over 12 years. These are usually vox pop video interviews with experts across education and work, discussing blue sky thinking ideas and views about the future of education and work.

Events

FE News has a free events calendar to check out the latest conferences, webinars and events to keep up to date with the latest education news and strategies.

FE Newsrooms

The FE Newsroom is home to your content if you are a FE News contributor. It also help the audience develop relationship with either you as an individual or your organisation as they can click through and ‘box set’ consume all of your previous thought leadership articles, latest education news press releases, videos and education podcasts.

Do you want to contribute, share your ideas or vision or share a press release?

If you want to write a thought leadership article, share your ideas and vision for the future of education or the future of work, write a press release sharing the latest education news or contribute to a podcast, first of all you need to set up a FE Newsroom login (which is free): once the team have approved your newsroom (all content, newsrooms are all approved by a member of the FE News team- no robots are used in this process!), you can then start adding content (again all articles, videos and podcasts are all approved by the FE News editorial team before they go live on FE News). As all newsrooms and content are approved by the FE News team, there will be a slight delay on the team being able to review and approve content.

 RSS IconRSS Feed Selection Page