Does AI Assessment Benefit English Language Learners?

Despite the huge buzz around AI and the opportunities it presents, when it comes to English language assessment, Dr Evelina Galaczi, Director of Research – English at Cambridge University Press & Assessment, says humans must remain in the driving seat to strike a balance between innovation and preserving quality. 

A recent YouGov poll found that 39% of people were concerned AI-based tests might not be assessing relevant language skills, potentially disadvantaging those taking exams to work, live and study in the UK. It also found that just over one quarter (26%) of respondents expressed unease at the prospect of limited human interaction when assessing ability. 

For me, the poll highlights two critical points. One – English learning is a fundamentally human skill, so we must think critically about the role AI should play when assessing this ability. And two – if the skills required are not adequately assessed with an AI-only approach, we risk misinformation about an individual’s ability, which could fundamentally affect a person’s educational and career goals. 

A Misplaced Fear? 

This is not to say that AI should not play its part. We simply need to understand what it can do and, more importantly, what it currently can’t. UCL’s Professor Rose Luckin refers to this in her work as “getting AI ready”. 

I find that the common fears around AI are often a little misplaced, because it’s not as simple as AI being either good or bad – it’s more about how it’s used. Just because some students or test takers can use AI to cheat, it doesn’t mean AI is going to increase instances of cheating. To become AI ready, it’s important that language assessment professionals focus on how we integrate AI into assessment rather than how we ban it. This creates an increasingly key role for humans skilfully using AI. 

The Evolving Role of the Human

Humans won’t be replaced by AI in the assessment space anytime soon, but their roles will change. People will need to stay in the driving seat at all stages of assessment, starting with test design. So, in the future when tests are designed, humans will be embedding AI in the test development process. 

If we look at test delivery it paints a similar picture – we can’t just delegate everything to a machine. In some contexts, we will need human examiners or teachers to step in and deliver the speaking tests. The same goes for marking and overseeing test security, too. 

And of course, if we dig deeper and look at communication more generally – there are many human aspects of communication that will not disappear because of the advent of AI. Only last week I tried using translation technology to have a conversation. I was speaking Bulgarian, my mother tongue, and the other person was speaking English. Whilst it was an impressive piece of tech, communication is not just a simple transaction of words. It has lots of additional aspects. For example, something seemingly insignificant, like little gestures and nods to show you’re following the conversation, make such a difference. 

What can AI do Well in English Language Assessment? 

In the classroom, AI can provide 24/7 opportunities for practicing language and gaining instant feedback. It’s something that a teacher cannot compete with, and it also frees up their time to focus on the more social and emotional aspects of learning. The way AI quickly produces data is a bit like how my running app tells me how long I’ve run and how fast. We can do the same for learning, and AI can provide insights about an individual’s performance, which is really useful data for teachers, parents and policymakers. 

If we turn our attention to assessment – humans and AI can also have a significant impact on helping to ensure tests are inclusive and fair. Although we need humans to understand the nuances of inclusion and accessibility, AI can be a big help in adapting tests to the needs of the test taker. For example, it can allow listening tests to be delivered at the pace of the test taker – through pausing and changing the speed as needed. So, AI can help us do more in terms of accessibility and fairness than we have done before. 

What Does AI Struggle with in Assessment? 

But of course, there are challenges! I often get asked: What specific language skills do AI-based assessments struggle to evaluate? The short answer is: all of them, in one way or another! For example, listening and reading skills are all about comprehension – what have test takers understood from reading a text? What have they understood from listening to someone?  One of the challenges that AI faces is that you need to have well-calibrated content to accurately measure reading or listening ability. We find that AI might give you a reading text and some comprehension questions, but it doesn’t do it consistently well. 

We see this in speaking, too. For example, we’ve probably all had a conversation (or possibly an argument!) with Siri, ChatGPT or Alexa. Whilst it’s impressive, AI doesn’t truly understand anything. It’s just an interface with data behind it. That’s not intelligence or knowledge!  Even if it sounds like it understands, AI cannot reliably adapt its speech well, especially when it comes to the different ability levels of learners and test takers. 

What are Some of the Big Risks and Concerns Around AI? 

When developing a strategic approach to the use of AI in education, the bottom line is about asking the question: what value does AI add? The biggest risk is using AI blindly without fully understanding the purpose and the impact it can have on learners and test takers. In other words, we need to ensure AI doesn’t lower assessment standards, but complements what human beings can do.

A big topic of discussion is the ethical considerations around using AI. It’s so important for everyone involved in assessment – from governments to teachers to individual users – to be aware of the risks. We need to be acutely aware of inaccurate content, bias, copyright, and issues around test integrity. And we mustn’t ignore the sizeable environmental impact of AI when evaluating the ethics around its use. Lastly, it is particularly important that, in the assessment space, we ensure that when we use AI, the model is trained on the right data, has been evaluated appropriately, and has a very well-defined purpose for its use.

What’s Next for AI and Assessment? 

I believe that a lot of immediate focus will be on understanding how we can improve current innovations in AI and language education and assessment. If I look to Cambridge, the English group I work in is highly focused on the role of AI in English learning and assessment, and we are continually talking to teachers and other stakeholders to find out how best to help them. In the assessment space specifically, we are already delivering a popular English test powered by AI and computer adaptive technology to offer fast and accurate testing of English language ability. For example, marking this test uses a hybrid approach where both examiners and AI play a role. This allows us to optimise their unique strengths. 

Looking ahead, we must continue to look critically at the ways in which AI can meaningfully boost test development, delivery and marking, test adaptivity for different types of test-takers, as well as test security. But perhaps most importantly, we need to continue creating an environment where AI can be used meaningfully to deliver fair and accurate assessments and is not just a shiny new gimmick. That’s what really excites me!

By Dr Evelina Galaczi, Director of Research for English, at Cambridge University Press & Assessment.