Back to blog

What breaks when universities grade AI-era essays?

Thắng Đoàn
Thắng Đoàn

Your students can now generate a passing essay in seconds, so the essay no longer tells you who learned anything. Universities worldwide are rediscovering a very old answer, and hitting a very old wall.

The written essay stopped proving anything

When ChatGPT arrived at the end of 2022, universities scrambled to tell whether generative AI could pass their assessments, The Conversation's researchers recall. Early evidence suggested models performed poorly on tasks like critical legal analysis, so cheating fears were initially overblown. But their newer research shows AI is getting better at exams, and headlines already boast about AI passing US bar exams. That means any take-home written work now carries a question mark you cannot remove.

The reflex answer: watch the student talk

Institutions are converging on oral assessment. Times Higher Education reports universities weighing a return to in-person exams to combat AI cheating. At Hebrew University, a committee now recommends requiring students to present seminar papers and using more oral examinations, per Calcalistech. VnExpress reports professors in Vietnam pushed back toward old-style oral exams for the same reason. Talking to a student for ten minutes reveals understanding that no detector software can infer from text.

The scaling wall nobody budgeted for

Oral exams fail at scale for two reasons, according to the Illinois researchers. Faculty face fatigue examining students one by one, and different evaluators frequently award divergent marks to identical answers. A department of 500 students cannot oral-examine its way out of this. So the honest version of the oral-exam comeback is a labor problem wearing an assessment costume.

Illinois hands the examiner to machines

University of Illinois researchers Geoffrey Challen and Craig Zilles released a preprint on a two-agent LLM pipeline for chat-based exams. Students complete live typed dialogues where they must support an argument, not just produce text. The architecture splits the work between an examiner agent and a grader agent, probing reasoning the way a human oral would, without examiner fatigue. It is a blueprint for testing reasoning rather than static outputs.

What this does not solve

An LLM examiner can itself be gamed, confused, or inconsistent across students, and the preprint stage means the evidence is early. None of these approaches measures skills students will use alongside AI, like prompting or verifying outputs, The Conversation argues for redesigning assessment around how graduates will actually work. Oral exams also test articulateness, which is not the same as competence, and they disadvantage students who fear speaking. The tool changed; the deeper redesign is still on you.

Which of your program's learning outcomes could only be verified by watching a student reason live?
If AI graded your oral exams tomorrow, what would you still insist a human check?

Pick one course you run and rewrite its final assessment as three questions you would ask a student out loud.

Sources

Times Higher Education

Calcalistech

VnExpress

CASRAI

The Conversation

Share:

Recommended for you

Enjoyed this article?

Subscribe for new articles. No spam. Unsubscribe anytime.

By subscribing you agree to receive the newsletter. No spam, and you can unsubscribe anytime.