The Evolution of the Campus Chatbot
Bolton College’s Ada chatbot has been supporting students since April 2017.
With the support of IBM’s Watson Assistant platform and the College’s business intelligence and software application layers the chatbot can respond to student questions across three broad categories.
- Firstly, general enquiries from students about the College regarding semester dates, library opening hours, the location of the exams office, campus activities, deadline for applying for university and more.
- Secondly, the chatbot can respond to specific questions from students about their studies; such as: what lessons do I have today/this afternoon/tomorrow? who are my teachers? what’s my attendance like? when is my next exam? when and where is my work placement? what qualifications do I have? give me a list of all the courses that am I enrolled on etc.
- And thirdly, the chatbot is being taught how to support subject specific enquiries from students. Bolton College is teaching Ada to respond to questions relating to GCSE Maths, GCSE English, Hairdressing and the employability curriculum.
A chatbot’s ability to respond to a large volume of questions and at all hours of the day is welcomed.
Nevertheless, we need to remember that these early wins represent the start of a much longer evolutionary journey in the use of cognitive services on the campus.
This article seeks to explore the evolution of the campus chatbot and other cognitive services which are designed to support students, teachers and support teams.
One of the most intriguing and promising aspects of using chatbots and cognitive services is their potential to support three facets of teaching, learning and assessment (TLA); namely its
- Creation,
- Consumption and
- Management.
These early wins represent the start of a much longer evolutionary journey in the use of cognitive services on the campus.
Content Curation
Traditionally, the curation of content on adaptive learning environments was done by teachers and online instructional design teams. The progress that students made through an online course determined the learning and assessment materials that were presented to them; and this was done in a deterministic manner.
The introduction of machine learning within the education sector enables adaptive learning environments to take advantage of probabilistic modelling which allows them to behave in a more informed, intelligent and contextualised fashion – all the traits that are demonstrated so well by our teachers in their classrooms.
The key difference of course is that adaptive learning environments which are informed by cognitive services can do this effortlessly and at scale. The cognitive nature of these services means that teachers can deliver personalised and contextualised learning and assessment materials to each and everyone of their students – a task that was not possible before the introduction of cognitive services on the campus.
In the near future it will be inconceivable for an EdTech learning management system company to offer its products and services without a cognitive service offer.
Figure 1: Curating adaptive content using probabilistic modelling.
In figure 1 we see an example were the positioning of a slide in an online tutorial has been governed by a machine learning model.
It’s place is influenced by numerous variables; such as the academic level of the student, the topic being studied, the student’s current performance on the course, the student’s target grade on the course, historic test scores following the completion of the tutorial, the learning preferences of individual students, engagement statistics for the slide, the weighting attributed to the slide and more.
The nature of an adaptive learning platform when it is coupled with machine learning means that countless permutations can be presented to large numbers of students; and it can be done with ease.
This will alter how instructional design teams curate content for online tutorials and assessment activities. Traditionally, online instructional designers would have worked with teachers to design a detailed storyboard before curating each slide.
However, when cognitive services are coupled with adaptive learning environments the instructional designer simply has to curate individual slides and add them to a database.
The decision to place any given slide is governed by a machine learning model – the new agent in the instructional design and delivery process. Overtime, the data on each slide steadily builds up; enabling the model to perform with increasing reliability.
The use of explainable AI will mean that individual teachers and course teams will be able to govern and explain the behaviour of their machine learning models. This is an important element that needs to be taken into consideration by institutions when employing cognitive services to support TLA.
Teachers should be in a position to know why their students are presented with differing TLA slides as they progress through a given online tutorial or assessment activity.
Likewise, students, parents, quality assurance teams and education inspectors should be able to query the algorithms that govern the behaviour of online TLA materials.
Consumption
With regard to the consumption of online learning materials, the introduction of video, interactive content, social media, video conferencing and the emergence of augmented, virtual and mixed-reality environments have made the learning experience richer for students.
Online assessment practices have tended to be narrow because learning management systems have been unable to capture the intelligence that is embedded in peer-to-peer assessment, classroom presentations, oral answers and long form answers.
The advent of conversational services gives teachers, instructional design teams and publishers the opportunity to curate and present learning and assessment materials in a manner that offers many affordances.
For example, imagine students conversing with an animated avatar or a chatbot – and then imagination an adaptive learning management system logging the learning and assessment data associated with all of these conversations.
Traditionally, online assessment practices were seen as being rather narrow; and online instructional designers could only offer teachers a limited set of options which typically meant the use of multiple-choice questions or drag-and-drop activities.
Last year Bolton College attempted to address this issue with its trial of conversational tutorials. Online conversational tutorials enable teachers to take advantage of natural language processing to facilitate online learning and assessment. They provide teachers the opportunity to ask open ended questions that require the student to answer in short form.
For example, a teacher may ask her students to respond to the following question in an online tutorial: “Could you tell me why good customer service is important to businesses and other organisations?”
The ability to record, analyse and assess a wide range of possible responses to such a question is a key attribute of conversational tutorials. The teacher can also pose follow-up questions to garner further insight from the student and she can offer feedback that is tailoured to suit the needs of the student.
When a conversational service is coupled with learning analytics the questions, the follow-up questions and the feedback that is presented to each student offers further personalisation.
A cognitive service can augment a teacher’s capacity to support his or her students.
The online assessment opportunities that are afforded to teachers goes much further when natural language processing is coupled with natural language understanding and natural language classification services.
Bolton College’s ILT Team is currently developing a service that takes advantage of machine learning that will automate the marking, grading and feedback of free form text by students.
The service is being developed as part of a wider online package that will support students with their work placement programme. All students are asked to complete an evaluation of their work placement programme which is in free form text; which could be a few hundred words in length.
Students typically evaluate four or five aspects of their work placement programme; which may include comments on how they worked in a team, how they applied their problem solving skills, how they developed their communication skills and so on.
The service that is being developed by Bolton College will read, comprehend and review the evaluation that is written by each student across all these aspects of their work placement programme.
The solution will also grade and offer feedback to the student in real time. The feedback to the student will suggest possible areas for improvement; enabling the student to amend his or her work placement evaluation.
Overtime, the use of machine learning will mean that as each student completes a work placement evaluation it will improve and enhance the accuracy of the machine learning model that is being used to assess subsequent student work.
A key objective of the project is to raise the overall quality of the work placement evaluation that each student submits at the end of their work placement programme. If successful, new marking, grading and feedback machine learning models will be developed to support other contexts; which will be numerous and wide ranging.
This is another example were a cognitive service can augment a teacher’s capacity to support his or her students as well as enabling students to achieve higher outcomes during the course of their studies.
Every student will have access to a personal cognitive assistant; be that a teacher, a tutor, a teaching assistant, a librarian, a careers advisor, a mentor and many more besides on an on-demand basis.
At a simple level, Bolton College’s chatbot is being used to support students with subject specific enquiries; especially when they engage with content that is held on Moodle, the College’s learning management system.
For example, a student can ask Ada questions regarding the keywords, topics or themes covered in an online employability tutorial. The ability to use the chatbot in this manner enables teachers and course teams to manage and improve student understanding of key concepts within the syllabus.
When students are away from the classroom there is also a degree of comfort knowing that their questions will be answered immediately by their course chatbot. As the service becomes embedded into courses teachers may evaluate the types of questions that are being asked by their students; allowing them make timely adjustments to their lessons and tutorials.
If the Ada service is to become pervasive Bolton College needs to enable access to the service across multiple distribution channels. The College has already completed a project which embeds the service onto the student home page.
The service is also being embedded into all new applications that are created by the College’s ILT Team; such as the College’s work experience, social action and CV builder apps. The development of an iOS and Android app for the Ada service during 2018-19 will mark a major milestone for the College because it promises to deliver a cognitive assistant to every student on the campus.
The advent of the Ada chatbot on desktop, tablet and mobile means that every student will have access to a personal cognitive assistant; be that a teacher, a tutor, a teaching assistant, a librarian, a careers advisor, a mentor and many more besides on an on-demand basis.
A student simply has to pick up his or her phone and talk to Ada to access information, advice and guidance; or to complete a simple transaction that supports their studies; such as handing in an assignment, renewing library books, booking an appointment or applying for the next course.
Management
When we observe and reflect on the behaviour of chatbots and other cognitive services on the campus it becomes apparent that these services support students and teachers in a manner that has no precedent within the education sector.
When examining the broader range of cognitive services that are to be found on a campus; it is important to remember that they are composed of numerous narrow AI applications or agents that cooperate with one another to support the needs of students, teachers and support teams. The nature of cooperation enables cognitive services to act and behave with a degree of agency.
In many ways they can be seen as additional virtual members of course teams and support teams; augmenting their capabilities so that they can better support every student; regardless of place and time.
Supporting a student at every point on the student life cycle is the job of numerous individuals and teams across the campus. The task of supporting every student becomes impossible; especially in an institution with tens of thousands of students who could be geographically dispersed and who study asynchronously.
However, cognitive services are designed to operate at scale; which enable teachers and support teams to deliver tailoured, personalised and contextualised services at a one-to-one level through the medium of a chatbot. Let’s examine how a chatbot could be used to support students at various points during their time at a school, college or university.
During induction, teachers and support teams can leverage the capabilities of the chatbot to support new students as they embark on their studies. Colleagues can construct intents, entities and dialogues so that students can ask for information on a whole manner of college services; including the time and place of their next class.
Similarly, colleagues can create other intents, entities and dialogues to support students as they enquire about future courses or training opportunities. Responses can be tailoured to individual students if the chatbot is linked with multiple datasets and business intelligence layers within the institution. The following video from the IBM Watson Assistant team provides an overview regarding the basic tooling behind a chatbot.
The benefits of online learning are well documented; however, the transactional distance between student and teacher remains wide.
The advent of the education chatbot offers teachers the opportunity to bridge this gap; enabling teachers to engage with their students in real-time through the medium of the chatbot.
When students are away from the classroom or the campus their questions about classroom topics can be readily answered by their course chatbot. The chatbot enables the teacher to be virtually present with his or her students; and at all times of the day or week. Cognitive services can also be used proactively by teachers and support teams.
For example, a student could have secured a good grade on a recent assignment. The chatbot could praise the student and show the student where he or she is with regards to achieving the target grade for the course. If the student has applied for an undergraduate course the chatbot could act on behalf of teachers and support teams to advise the student about the remaining grades required to secure a place at the university.
One of the most interesting projects that Bolton College is running over 2018-19 is the introduction of the Ada chatbot on the institution’s staff home page; especially for teachers.
Once the chatbot has been connected with the College’s key datasets, its business intelligence layer and it has been trained by a broad range of teams around the campus; teachers will be in a position to ask Ada a wide range of questions about the courses that they are managing and about the students that they are supporting.
As the teacher facing Ada service develops we envisage that teachers will ask Ada about the academic progress that is being made by individual students. For instance, Ada could answer the following questions from a teacher: what’s the average grade profile for student X, show me the students who are currently performing below target levels, could you show me the students in my class who are doing a GCSE retake in Maths, who is going out on a work placement next week, is my course on target to meet its value added score and more.
The introduction of the teacher facing Ada service promises to change how teachers engage with student data and how they gather insights regarding the progress made by their students.
Bolton College’s Ada service for teachers will also be used to support colleagues with enquiries about the College’s teaching, learning and assessment standards. Teachers may ask Ada about differentiation in the classroom, how to improve feedback to students, questioning techniques in the classroom and so on.
When the teacher facing Ada project gets underway the answers to these questions will be curated by the College’s Quality Unit, its team of Advanced Practitioners, the Staff Development Team and by teachers from across the College.
In addition to this service colleagues from across the College will have the opportunity to ask Ada about the availability of places on staff development workshops that are being held on the campus and they will have the option to book places via the chatbot.
Just as the student facing chatbot prompts and offers advice and guidance to students; the staff facing chatbot will also offer similar support to teachers and support teams.
Summary
If you are introducing a chatbot to your campus it is likely to evolve over three distinct stages. The chatbot in stage one is connected to all the major datasets on the campus. The chatbot recognises who is asking the question and is able to respond contextually.
For example, when a student enquires about her average grade profile on the course, the chatbot can also advise the student about the grades needed on subsequent assignments if she wishes to gain the entry qualifications required to get to university.
The chatbot in stage two can help students, teachers and support teams with day-to-day transactions and workflows.
For example, students are able to book appointments with their tutors or support teams; they can search and apply for courses, they can submit assignments, they can change their network passwords, and they can amend their profile and contact details.
Likewise, teachers and support teams can make room bookings, they can book annual leave, they can search and apply for professional development courses, they can transfer or withdraw students from courses, they can do internal transfers from one college budget to another or they can add support interventions for students.
There is no doubt that conversational services can make simple transactions and workflows easier to manage. When they do, it mitigates the need to engage directly with software applications and their graphical user interfaces.
These services are still in their infancy but as cognitive services become more able they will offer individuals the opportunity to carry out complex transactions and workflows that were previously in the domain of the traditional graphical user interface.
Stage three represents the point when chatbots and the cognitive services that underly them conduct themselves as active agents on the campus to help students, teachers and support teams.
As mentioned earlier, the main campus chatbot should be seen as an amalgam of multiple agents. Each agent specialises in performing tasks and activities that fulfill specific objectives for teachers and support teams across the campus.
For instance the library agent gathers context specific resources to support each student with his or her studies; the careers agent presents appropriate information, advise and guidance to students to support their progression beyond their immediate studies; the university agent supports students with their desire to secure a place at university; the grading agent aims to support students achieve the highest grade possible on each of their courses; and so on.
When these agents cooperate with one another in a desirable manner they become very powerful aids for supporting students, teachers and support teams.
For example, teachers on a course may get together for their weekly team meeting. During the meeting they discuss the academic progress that is being made by their students. Teachers start to engage in dialogue with their chatbot or cognitive assistant; even asking for advice and guidance about how best to support particular students on the course.
It is important to remember that the course team work alongside the suite of cognitive services on the campus. These services can access a copious amount of data; and they can analyse it and find insights with ease.
When used well the course team is more capable of supporting each student on their course; and when this happens it will increase the number of students who successfully complete their studies and they will also graduate with higher grades. This is only one example were individuals work with cognitive services on the campus. Now repeat this across all contexts with students, teachers and support teams on the campus.
As you can see, the advent of the personal cognitive assistant and other cognitive services on the campus have the potential to improve how we design, deliver and manage education services.
However, they also pose numerous challenges for the sector. At a simple level they challenge how education services have always been delivered. At a more complex level they challenge the need for certain habits and routines that have been established over the last century or more.
Furthermore, they have the potential to ignite new ideas which will shape the future of the education landscape in the UK and beyond.
Aftab Hussain, Strategic ILT Lead, Bolton College
Responses