New findings from Jisc highlight the benefits of a Collaborative Approach to AI in Assessment 

By using a pilot model participants gain access to a safe space to explore, varied perspectives and support from sector experts 

Initial insights from Jisc’s year-long AI in Marking and Feedback Pilots outline the challenges and opportunities faced by educators and students from 38 UK colleges and universities when using education-based AI tools from Graide, Keath and TeacherMatic in assessment settings. 

The use of a pilot model allows educators to examine how AI tools might support a reduction in workload while preserving academic standards and enhancing the student experience. The collaborative nature of the pilot, bringing together users of differing levels of skill and experience, also helps participants develop a better understanding of where caution may be required.  

With support from experts at Jisc, a community of cohorts from UK further and higher education (FE and HE) and with input from product developers, the pilots provide a safe space for participants to explore how the application of AI in marking and feedback might impact their institution. 

By gathering different stakeholder perspectives on AI in marking and feedback, these pilots will support the development of a holistic overview of the use of AI in marking and feedback across UK tertiary education. 

Initial insights: 

• Keeping the human in the loop when using AI can be hard to maintain in practice, but throughout the pilot methods such as parallel marking were identified to more effectively bake-in this approach. 

• Honest and open dialogue and clear channels of communication were found to be beneficial when gaining student buy-in for the use of AI in marking and feedback. 

• AI should not be used as a ‘catch-all’. Evidence-based rationales should be developed for the use of AI tools in marking and feedback where efficiency gains can be demonstrated whilst maintaining high levels of academic integrity. 

• The delivery of clear, actionable and timely feedback across the pilots identified formative assessment as the most effective and low risk use case for AI in marking and feedback. 

Additional outcomes: 

• The pilot created opportunities for ongoing collaboration between participants and developers, with user feedback directly informing improvements to the tools and their suitability for real-world educational settings.  

• One of the shared challenges faced by participants was how best to communicate the use of AI in marking and feedback to staff and students. Through cohort sessions, shared resources, such as a ‘student communication pack’ have been developed collaboratively. 

• Another key finding was the link between the design of marking criteria and assessment frameworks and how well AI tools performed. Participants found that when expectations were clearly defined and explicitly written down from the outset, the AI tools produced more accurate and consistent outputs. 

Tom Moule, senior AI specialist at Jisc said:  

“Assessment is a particularly sensitive area, so we focused the AI in marking and feedback pilot on facilitating informed discussion and supporting a collaborative, discerning approach to deciding where AI can help, and where caution is needed. 

“What struck me was how quickly conversations shifted from the AI tools to much deeper questions about assessment itself. Giving staff the space to explore those questions together, and to reflect openly on what they were hearing from students, has been a crucial part of supporting a more careful, collaborative approach to AI in assessment.” 

Hilary Reid, digital learning team leader, Ayrshire College said: 

“At Ayrshire, we found that access to peer support from other institutions was one of the main benefits of taking part in the Jisc pilot. Being able to share our experience and learn from others who are further along in this area has been really valuable in shaping how we approach the use of AI for the purposes of marking and feedback. 

“Participating in the pilot has given us a voice and a direct channel of feedback with our chosen product developers, TeacherMatic, resulting in some developments that have directly benefited our users.  

“Introducing the use of AI for marking as part of a structured sector-wide pilot, as well as access to the additional support that goes along with being a pilot participant, has given teaching staff at Ayrshire the permission and confidence to explore using AI tools in their own practice, resulting in a huge increase in the use of TeacherMatic at the college.” 

Supporting the transition to scalable AI approaches 

As the pilots continue, Jisc’s AI team will use the findings to support UK further and higher education as they collectively transition towards thoughtful, scalable approaches that harness AI’s benefits while safeguarding trust, quality and professional judgement.