Overview
- A Stanford-led paper published May 27 found professors preferred AI-generated answers to contract-law office-hour questions in roughly 75% of nearly 3,000 blind head-to-head comparisons.
- The study used 40 representative 1L contract questions written by faculty, calibrated AI outputs to match human answers, and had 16 professors from 14 U.S. law schools evaluate anonymized pairs to reduce bias.
- Professors flagged AI answers as pedagogically harmful about 3.5% of the time versus 12% for peer-written answers, meaning reviewers judged human responses more often likely to mislead or confuse students.
- Performance varied across systems — several models outscored instructors on average — but authors warned the research measures answer quality only and does not assess long-term learning, instructor fit, or citation accuracy.
- The finding arrives as law schools wrestle with policy: some are experimenting with AI tutoring while others, including UC Berkeley Law, have imposed strict limits on student AI use for graded work and exams, raising questions about supervision and professional risk.