In a bold classroom experiment, researchers at the University of Illinois let ChatGPT enroll in an aerospace engineering course—and the results reveal both promise and peril for AI-assisted learning.
Key Points at a Glance
- ChatGPT earned a B overall in a semester-long control systems course.
- It excelled at structured math questions but struggled with analytical reasoning.
- The study shows ChatGPT’s limitations in higher-level understanding despite quick and mostly correct outputs.
- Findings suggest educators must redesign curricula to foster deeper critical thinking.
What happens when artificial intelligence goes to engineering school? A team of researchers at the University of Illinois Urbana-Champaign’s Grainger College of Engineering decided to find out. They enrolled the free version of ChatGPT in a full semester of AE 353: Aerospace Control Systems, a demanding undergraduate course, and treated it like just another student. The result? ChatGPT passed the course with a B—but its grade told only part of the story.
The study, led by Ph.D. student Gokul Puthumanaillam and advised by Professor Melkior Ornik, set out to evaluate how ChatGPT performed across a range of assignments. On straightforward, structured math problems, the AI performed impressively, often earning an A. However, when faced with open-ended questions that required synthesis, context, or abstract reasoning, it faltered—scoring a D-level performance.
The final average for ChatGPT across the course was 82 percent, narrowly below the human class average of 84.85 percent. But the implications of the AI’s performance were more revealing than the numerical grade. “It’s possible for a student who puts in minimal effort—essentially just copying ChatGPT’s answers—to pass with a B,” said Puthumanaillam. “But they won’t have actually learned anything.”
In essence, the AI mimicked a lazy student’s dream—fast, efficient, and reasonably accurate—but offered little in the way of true understanding or engagement. Its answers were sometimes peppered with strange technical jargon or inaccuracies, like referencing “quasi periodic oscillations” that were never mentioned in the course materials or lectures. Even when fed all necessary material, the model occasionally hallucinated, producing information that sounded plausible but was completely fabricated.
Professor Ornik emphasized the role of AI as an educational tool rather than a replacement for thinking. “Like calculators in math class, ChatGPT is here to stay,” he said. “What this study tells me as an educator is that I need to adapt. That might mean designing more project-based or analytical assignments where critical thinking is unavoidable.”
One critical insight from the research was that while ChatGPT could mimic learning—adjusting its answers when corrected—it lacked long-term progression. When given a multiple-choice question, corrected, and then asked a variant, it might answer better. But in general, its performance plateaued quickly, hovering around 90 percent for math-based homework and showing little evolution over time.
The study carefully controlled conditions to mirror those of real students. There were no group assignments, and ChatGPT received the same prompts, at the same time, as its human classmates. “ChatGPT was just another student in the class,” Puthumanaillam quipped. “But unlike students, it doesn’t get smarter by struggling through a concept.”
The research was supported by the Grants for Advancement of Teaching in Engineering and benefited from collaborative work by other Ph.D. students and faculty, who provided course materials and the digital infrastructure. The full study, titled “The Lazy Student’s Dream: ChatGPT Passing an Engineering Course on Its Own,” will be presented in June at the 14th IFAC Symposium on Advances in Control Education.
For educators, the message is clear: generative AI is powerful, but its strength lies in speed, not wisdom. As such tools become more embedded in learning environments, instructors may need to rethink what it means to teach—and to truly learn.
Source: University of Illinois Grainger College of Engineering
