← back
Build AI Systems for Discernment, Not Approval - Angel Ortmann Lee, Duolingo
Takeaway
Human review works better when the workflow explicitly demands independent evidence rather than approval of AI output.
Summary
- Duolingo tested automation bias by inserting false copy-typing alerts into historical English Test sessions with no cheating.
- Experienced proctors with calibration accuracy above 90% accepted half of the fabricated alerts; no test-taker results were affected.
- Revised guidance emphasized that AI alerts are preliminary and require independent video evidence before being upheld.
- Correct rejection of false alerts rose from 50% to 71% after the guidance change, without modifying the detection model.
automation-biashuman-in-the-loopduolingo
Original description
The human-in-the-loop paradigm promises automation's efficiency without sacrificing safety or nuance. But it hides an underexamined assumption: that human involvement produces genuine discernment, not just a rubber stamp. In practice, human-AI interaction often occurs in environments where throughput or business incentives crowd out critical thinking. This talk examines why human oversight so often falls short in practice, and how deliberate interaction design can close the gap. We've grown comfortable delegating reasoning to machines. We follow GPS down unfamiliar streets and accept AI coding suggestions with minimal inspection. When Shaw & Nave (Wharton, 2026) studied human-AI interaction, they found people accepted AI answers over 80% of the time, even when it was wrong. They call this cognitive surrender: when humans forgo deliberation and adopt AI output with minimal scrutiny. At the Duolingo English Test, a controlled experiment revealed that experienced proctors shown fabricated AI cheating alerts confirmed cheating at near-chance rates. But coin-flip accuracy is unacceptable when college admissions and visas are on the line. The model wasn't the problem, the signals were fake. But skilled reviewers were still showing systematic confirmation bias. A single change to decision framing improved accurate rejections by 21%. The fix isn't better models or more human oversight. It's engineering the interaction itself. What reasoning patterns do you need from the human, and how does the interface elicit them? This talk covers practical design principles for building AI systems that improve human judgment, produce more reliable review behavior, and generate higher-quality training data. Every AI system trains its users, the question is whether you're doing it deliberately. Speakers: Angel Ortmann Lee (Duolingo): Angel is a Software Engineer at Duolingo, building AI security systems for the Duolingo English Test to help ensure online testing is secure, trustworthy, and accessible for learners around the world. LinkedIn: / angel-ortmann-lee-7494b9201