Black-Box Tutor: An AI chatbot for Scaffolding Mechanistic Reasoning and Self-regulation in Chemistry

Funding period: 01/2026 - Present Supported by the College of Education at Michigan State University

One of the goals of science education is to build students’ knowledge and reasoning about how and why things happen, i.e., mechanistic explanations. Chatbots, such as large language models (LLMs), have become increasingly integrated into higher education chemistry courses to support students’ explanations; however, it raises questions about how LLMs can support students’ learning about complex topics during mechanistic explanations about science. Self-regulated learning (SRL) is an essential skill for generating explanations, where students take agency over their learning process by setting goals and evaluating their progress in relation to their goals. Studies find that students who use SRL productively often outperform their peers across contexts; however, most students struggle to use SRL productively, particularly when using GAI tools. Presently, LLMs operate through surface-level linguistic patterns rather than genuine understanding of students’ explanations or their SRL processes in real-time, which could offer insights into adaptive scaffolding.

The Black-Box Tutor is an AI chatbot (OpenAI’s ChatGPT EDU Edition) situated at exactly that decision point, though it is deliberately limited in scope: it does not enact adaptive pedagogical moves in the manner of a tutoring system that escalates through hints, prompts, and assertions, but instead offers hints aimed at supporting students’ mechanistic reasoning as they work through chemistry content. Its hints are grounded by a retrieval-augmented generation (RAG) pipeline, drawing on topic-relevant chemistry material, rather than on an internal pedagogical model of the learner. The motivating question is not whether the chatbot improves performance, but how reasoning is distributed between an information query and a student’s own developing understanding of the mechanism underlying the phenomenon: when a student turns to the chatbot for a hint or information, rather than continuing to reason independently, what does that shift reveal about where the cognitive work of mechanistic explanation is actually occurring?

Data collection, targeting a sample of 75 undergraduates students enrolled in introductory chemistry courses at MSU, is currently underway.