Exam boards, regulators and institutions largely agree that artificial intelligence detectors produce false positives and lack evidentiary standing. Such detectors are also proven to be biased against non-native English speakers. So the essay is no longer viable as a way to assess what a student understands – if it ever was (which we doubt).
The recommended alternatives may make misconduct harder to conceal, but holistic review, direct supervision, corroborating evidence and vivas are really just more forms of detection. And few would dispute that detection and discipline are a poor foundation for genuine academic integrity.
Our answer has been to try to make AI detection beside the point by making it irrelevant to assessment. And we wanted to enlist AI as a friend rather than an enemy in the learning process.
Specifically, we have developed an AI platform that makes teaching and learning a two-way exchange, such that assessment happens simultaneously throughout, rather than being a verdict delivered at the end.
That’s the premise behind Duologue, a platform we’ve built and piloted at City St George’s, University of London, in collaboration with scientist and entrepreneur Timo Hannay and tech philosopher and author Tom Chatfield. Instead of the student asking the AI questions, the AI leads a Socratic conversation probing what the student already knows and where the gaps are in their AI literacy and critical and computational thinking. It’s less digitised viva and more a learning journey that’s tested in real time as it unfolds.
There are also game-like “labs” that support learners along the way – often two or three students around a screen – built around understanding AI itself: how prediction algorithms work, how AI actually arrives at its answers. They’re part of building AI literacy, so students learn to question and understand the tools they’re using rather than accepting AI output uncritically. The gamified element is more quiz-like than playful – testing how accurate a student’s understanding is – but delivered through the same Socratic back-and-forth as the rest of the platform.
The labs are low-pressure interactions designed to spark curiosity in the mechanics of AI – from algorithms to tokens and classifiers – via guessing games, pattern spotting and “seeing behind the curtain” of coding.
For individual modules on the platform, strategic partners including the Raspberry Pi Foundation, Sage Publishing and the software company Wolfram also contributed staff time, expertise and intellectual property. But Duologue’s development was primarily funded through the university’s Digital Innovation and Transformation Office and drew on huge input from City St George’s academic experts, learning designers, technologists and infrastructure and intellectual property experts, as well as the students who supported prototyping through workshops and feedback.
It was very important to us to build the platform inside the university, rather than outsourcing it. We’ve seen so many products coming up in the past two years that promise to plug into existing systems and take them to the next level but turn out not to work as you need them to – not because the technology is bad necessarily, but because they’re often superficial (a student can get a mark out of 10 without their understanding ever really being tested or built on) or are built for generic needs and contexts. A team embedded in a university can adapt the tech to its students, staff and curriculum.
Take our safeguarding layer, which watches for concerning language and directs people to support. It highlighted a reference to a juicing diet as a red flag. Juicing can be slang associated with steroid or other drug use, but in this case it turned out to be an innocent reference to a juice-based diet. Recalibrating it took background knowledge that only people at the institution could bring.
Nor was red flagging the only issue we’ve faced – but that is exactly what you would expect. A system that continuously assesses judgement is itself an experiment in what judgement means, and we’ve worked to bring on board staff sceptical that a chatbot can stand in for a seminar, for instance.
We readily concede that Duologue isn’t teaching in the traditional sense – there’s no avatar lecturing students. Rather, it guides learners through what they need to grasp and pushes them further based on how they’re doing, rather than delivering content downwards. And we think that is potentially more useful than a seminar.
Admittedly, the platform is still in pilot, and we’re wary of overclaiming what one project at one university proves. But we think we’ve hit on something, and we are getting a fair bit of attention. Conversations with other universities about running their own pilots are under way. Government bodies and healthcare organisations have been getting in touch, too.
The key point is that academic integrity won’t be protected by becoming better at guessing whether a machine wrote an essay. That’s only likely to get even harder. But even if it became possible, the essay would still be a warped yardstick: it was always possible to produce a technically competent document without any real, deep understanding.
Rather, we will best protect academic integrity by becoming better at demonstrating whether a student can think.
Baba Sheba is professor of digital education and transformation and director of digital transformation at City St George’s, University of London. Mike Bond is strategy partner at Bond & Coyne, a brand and technology agency.
Leave a Reply