Quick Take
- Students scored higher on proctored exams after instructors introduced AI tools tailored to applied STEM courses.
- The trial enrolled 7,200 students across 19 universities, with 4,768 students in the experimental exam analysis.
- The trial compared earlier and later rollouts, leaving open how course-specific tools would perform against generic chatbots.
Students in applied STEM courses scored higher on proctored exams when instructors introduced AI tools tailored to their classes, compared with classes still waiting for their tools, according to a randomized trial across 19 universities reported in a Nov. 14 preprint.
The research led by Jun Li of the University of Michigan estimated a student-level gain of 0.32 standard deviations, with a 95% confidence interval of 0.06 to 0.59 standard deviations.
The estimate puts the improvement at roughly 33% of the typical spread in scores within a class and leaves uncertainty about the size of the gain. Standardizing scores within each class gave researchers a common scale for comparing results across courses that used different exams.
The preregistered trial involved 7,200 students in 55 applied STEM classes over three semesters, from spring 2025 through spring 2026. The study counted each course-and-instructor pairing as one class.
The experimental part of the exam analysis covered 4,768 students across 44 course offerings. Researchers used results from earlier terms alongside that sample to account for how classes had performed before the rollout.
The exam analysis included only proctored, in-person exams, mostly on paper. The remaining exams used strictly monitored computers or restricted-access computers at university testing centers.
The researchers added that screening rule after preregistration and excluded course offerings where exams could allow AI assistance.
How the trial tested classroom AI
One group introduced course-specific AI tools a term earlier. Researchers measured learning on proctored exams that excluded AI assistance.
Relative to delayed rollout, the estimated student-level gain was 0.32 standard deviations (95% CI 0.06 to 0.59), using earlier exam results as a baseline.
The trial focused on applied STEM classes. The exam-security filter was added after preregistration, and the timing comparison does not isolate which tool designs worked best.
What instructors built with AI tools
Researchers grouped classes by subject and academic level, then randomly assigned them to introduce their tools immediately or one teaching term later.
Instructors in both groups received design and technical support, including training sessions, examples of how to write instructions for the tools, and access to a platform to build and deploy them.
Once instructors brought the tools into their courses, all students in those classes had access during the term, before sitting the exams used to measure learning.
Some instructors built tutors that gave students targeted practice and feedback, while others built coaches that guided students through problems by asking questions. Instructors also designed interactive simulations and ran role-play exercises that let students question AI-generated stakeholders.
The study detected no statistically significant change in engagement or satisfaction. Engagement measures varied by class and included records such as attendance and assignment completion, and satisfaction came from end-of-term course evaluations.
The trial compared rollout timing, leaving open how course-specific tools would fare against generic chatbots. How much each design contributed to the gain and whether it extends beyond applied STEM remain open questions.


