Featured Voices

Derek Newton, Education Writer in NYC
Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

ProctorU, one of the largest and oldest remote test proctoring and security providers, announced this past week that it was doing away with its AI-only proctoring service. Going forward, the company said, they’d use only trained, human test proctors - a service they always offered. 

They are eliminating the proctoring process that records a student during their test, uses AI and other technologies to analyze the session, flagging or scoring it for suspicious or abnormal events and behaviors such as routinely looking off screen or talking to someone.

That’s a pretty standard start for remote proctoring. But what made this process “AI-only” was that, after the test recordings were flagged or scored, they were turned over to faculty or the test-providers. No human on the proctoring side ever reviewed it, leaving faculty or assistants on the school side to review the flags and make decisions about what was or was not going on during the test.

In ditching the AI-only process, ProctorU said that it put too much burden on teachers to review and police test sessions - a big job for which they were often unprepared and unable to manage given their already hefty commitments. In one of the press reports about the decision, the company said that it would take a professor more than nine hours to review just the flagged portions of a single one hour exam in a class of 150 students.  

Further, ProctorU executives said that the AI systems used in proctoring are sensitive by design which could flag innocent activities as suspicious. This makes sense. If the objective of a first screen is to catch things for people to review, you want to catch most anything that could be something. In other words, better to over-flag than under-flag. The problem is, this creates more flags - more work when the flags are just passed along to teachers.

As a result, ProctorU and independent audits show that very few of the flagged test sessions were being reviewed at the school. An Iowa study showed 14% were being watched by teachers. ProctorU said 11% were. Few, either way.

This pathway - sensitive AI, lack of time for review and no review - made the process nearly pointless.

Worse, we know that some professors were relying on the flags, the scores of deviation, as proof of misconduct. Given what the AI does, that’s impossible. Without a human review, a determination of any kind - cheating or insignificant - is impossible.

It’s possible that simply having an AI-only system deterred some misconduct, given the possibility for detection. But with review rates so low, it’s unlikely the deterrence was much of one. For deterrents to work, someone has to be caught once in a while.

This significant marketplace decision has a few implications for instructors and administrators. Here are five things faculty and other schools should do:

Understand the Difference

 When tests or assignments are done online, the choice is not whether to proctor or not to proctor. For the record, not proctoring is a very bad idea. Research shows that not protecting a test invites academic misconduct. 

If a remote assessment is to be proctored, faculty and administrators should know there are three different models - the AI-only, scan and pass along option, a record, scan and review by trained company proctors option, and an exam with live human proctors during the exam. The first type is what ProctorU is dropping and the type that has caused the most controversy.

Because they’re not the same, they don’t cost the same. Responding to the ProctorU decision to use human proctors only, an executive with Respondus, a ProctorU competitor, said, “A typical university using Respondus Monitor spends less than 25 cents per proctoring session. Live, human proctoring often costs $25 per exam. The price points and services are entirely different.”

They are very different. Understand how they’re different is important.

Be Sure You Know What Kind of System Your Proctoring Company Uses and What it Means for You

The names of proctoring companies can be confusing - ProctorU, Proctorio, Respondus, Honorlock. Their product names can be even more so.

If you’re not sure what type of proctoring your chosen company is using for your exams - ask.

If the answer is the cheaper, record, AI-scanned and passed along version, understand that the burden for reviewing the recorded and flagged sessions is probably yours. The company may be making a recording and telling you what to look at, but you’re going to have to look at it and make a decision about what’s happening.

Review Your Flags

 Whatever method your proctor company uses, videos of the sessions will be passed along or made available - either flagged, flagged and reviewed or flagged and reviewed in real time. You are still going to need to review those sessions and decide what to do.

The benefit of having humans review the session before they get to you is that you’ll get far fewer things to review. One estimate was that human reviewed proctored tests cut the number of reviews required by faculty by 85% - cutting nine hours of review to just one.

Either way, review is necessary. The system does not work unless a teacher or a similar authority at the school is watching, engaging students where there are incidents and elevating misconduct incidents where necessary.

Ask for Help

 If reviewing flagged test sessions isn’t possible, and it may not be, tell someone. Ask for help. Whatever it costs, remote test proctoring loses nearly all of its impact if no one on the school side is reviewing the sessions. A system in which just 11% or 14% of sessions are reviewed is a broken system.

Moreover, if it’s not possible or reasonable to review the sessions, school administrators should know the burden that’s being created.

Ask for Better

People are unquestionably better at discerning attempts to be dishonest from common behavior. Computers can flag things that are different, things that may be misconduct. Only a human can decide what actually is.  

If your department or school is using a system that bypasses humans, ask for one that doesn’t. Ask for it especially if you cannot or should not be reviewing hours of computer-flagged test sessions. The better options are probably more expensive. But getting it right - avoiding false accusations, freeing up teachers to teach and mentor, having two humans involved instead of none - is probably worth it.

Derek Newton, Education Writer in NYC

Sponsored Video

Advertisers

Upcoming FE Events

Advertiser Skyscrapers

Latest Education News

Further Education News

The FE News Channel gives you the latest education news and updates on emerging education strategies and the #FutureofEducation and the #FutureofWork.

Providing trustworthy and positive Further Education news and views since 2003, we are a digital news channel with a mixture of written word articles, podcasts and videos. Our specialisation is providing you with a mixture of the latest education news, our stance is always positive, sector building and sharing different perspectives and views from thought leaders, to provide you with a think tank of new ideas and solutions to bring the education sector together and come up with new innovative solutions and ideas.

FE News publish exclusive peer to peer thought leadership articles from our feature writers, as well as user generated content across our network of over 3000 Newsrooms, offering multiple sources of the latest education news across the Education and Employability sectors.

FE News also broadcast live events, podcasts with leading experts and thought leaders, webinars, video interviews and Further Education news bulletins so you receive the latest developments in Skills News and across the Apprenticeship, Further Education and Employability sectors.

Every week FE News has over 200 articles and new pieces of content per week. We are a news channel providing the latest Further Education News, giving insight from multiple sources on the latest education policy developments, latest strategies, through to our thought leaders who provide blue sky thinking strategy, best practice and innovation to help look into the future developments for education and the future of work.

In Jan 2021, FE News had over 173,000 unique visitors according to Google Analytics and over 200 new pieces of news content every week, from thought leadership articles, to the latest education news via written word, podcasts, video to press releases from across the sector, putting us in the top 2,000 websites in the UK.

We thought it would be helpful to explain how we tier our latest education news content and how you can get involved and understand how you can read the latest daily Further Education news and how we structure our FE Week of content:

Main Features

Our main features are exclusive and are thought leadership articles and blue sky thinking with experts writing peer to peer news articles about the future of education and the future of work. The focus is solution led thought leadership, sharing best practice, innovation and emerging strategy. These are often articles about the future of education and the future of work, they often then create future education news articles. We limit our main features to a maximum of 20 per week, as they are often about new concepts and new thought processes. Our main features are also exclusive articles responding to the latest education news, maybe an insight from an expert into a policy announcement or response to an education think tank report or a white paper.

FE Voices

FE Voices was originally set up as a section on FE News to give a voice back to the sector. As we now have over 3,000 newsrooms and contributors, FE Voices are usually thought leadership articles, they don’t necessarily have to be exclusive, but usually are, they are slightly shorter than Main Features. FE Voices can include more mixed media with the Further Education News articles, such as embedded podcasts and videos. Our sector response articles asking for different comments and opinions to education policy announcements or responding to a report of white paper are usually held in the FE Voices section. If we have a live podcast in an evening or a radio show such as SkillsWorldLive radio show, the next morning we place the FE podcast recording in the FE Voices section.

Sector News

In sector news we have a blend of content from Press Releases, education resources, reports, education research, white papers from a range of contributors. We have a lot of positive education news articles from colleges, awarding organisations and Apprenticeship Training Providers, press releases from DfE to Think Tanks giving the overview of a report, through to helpful resources to help you with delivering education strategies to your learners and students.

Podcasts

We have a range of education podcasts on FE News, from hour long full production FE podcasts such as SkillsWorldLive in conjunction with the Federation of Awarding Bodies, to weekly podcasts from experts and thought leaders, providing advice and guidance to leaders. FE News also record podcasts at conferences and events, giving you one on one podcasts with education and skills experts on the latest strategies and developments.

We have over 150 education podcasts on FE News, ranging from EdTech podcasts with experts discussing Education 4.0 and how technology is complimenting and transforming education, to podcasts with experts discussing education research, the future of work, how to develop skills systems for jobs of the future to interviews with the Apprenticeship and Skills Minister.

We record our own exclusive FE News podcasts, work in conjunction with sector partners such as FAB to create weekly podcasts and daily education podcasts, through to working with sector leaders creating exclusive education news podcasts.

Education Video Interviews

FE News have over 700 FE Video interviews and have been recording education video interviews with experts for over 12 years. These are usually vox pop video interviews with experts across education and work, discussing blue sky thinking ideas and views about the future of education and work.

Events

FE News has a free events calendar to check out the latest conferences, webinars and events to keep up to date with the latest education news and strategies.

FE Newsrooms

The FE Newsroom is home to your content if you are a FE News contributor. It also help the audience develop relationship with either you as an individual or your organisation as they can click through and ‘box set’ consume all of your previous thought leadership articles, latest education news press releases, videos and education podcasts.

Do you want to contribute, share your ideas or vision or share a press release?

If you want to write a thought leadership article, share your ideas and vision for the future of education or the future of work, write a press release sharing the latest education news or contribute to a podcast, first of all you need to set up a FE Newsroom login (which is free): once the team have approved your newsroom (all content, newsrooms are all approved by a member of the FE News team- no robots are used in this process!), you can then start adding content (again all articles, videos and podcasts are all approved by the FE News editorial team before they go live on FE News). As all newsrooms and content are approved by the FE News team, there will be a slight delay on the team being able to review and approve content.

 RSS IconRSS Feed Selection Page