My Teacher Thinks the World Is Flat! Can AI Essay Graders Be Fooled?
Can AI essay graders be fooled?
New research shows many automatic essay scoring (AES) systems reward “word soups” over real writing. Using interpretability tools, the authors found that a few keywords drive most of the score, while flow, grammar, and coherence matter far less.
The wild part: because these models aren’t grounded in world knowledge, adding confident but false lines—like “the world is flat”—can even boost a grade.
- Scores barely drop when context around “important” words is removed.
- Coherence, content relevance, and common sense are underweighted.
- Simple adversarial edits can game the system.
Bottom line: Before using AES for high‑stakes tests, we need models that understand meaning—not just keywords—and rigorous validation across multiple writing skills.
Paper: http://arxiv.org/abs/2012.13872
Paper: http://arxiv.org/abs/2012.13872v1
Register: https://www.AiFeta.com
AI NLP EdTech Assessment MachineLearning ExplainableAI Ethics
