Every time you glance at a new environment, your brain instantly reads not just what’s there—but what you could do there. Scientists have just uncovered how our minds map out opportunities for action in a way no artificial intelligence can match, revealing a powerful edge of human perception.
Key Points at a Glance
- Human brains automatically and rapidly process “affordances”—possible actions an environment allows—using unique patterns in the visual cortex.
- MRI scans reveal these action-oriented brain processes happen even without conscious thought or explicit instructions.
- AI models like GPT-4 and advanced image recognition still fall short, failing to naturally see action possibilities in a scene.
- Understanding human efficiency could inspire more sustainable, smarter, and human-like AI in robotics and beyond.
It seems almost effortless: step into a room, look at a photo of a mountain trail, or stroll past a city canal. Instantly, you know where you could walk, run, climb, or swim—even if you’ve never been there before. But what if you tried to get an AI to do the same? A new study led by researchers at the University of Amsterdam reveals just how different our brains are from even the best artificial intelligence when it comes to “reading” the world.
The research team, guided by computational neuroscientist Iris Groen and PhD student Clemens Bartnik, set out to uncover how our brains process these so-called “affordances”—the hidden opportunities for action that leap out at us from any scene. Using MRI scanners, participants viewed photos of all kinds of environments and indicated which actions the scenes invited: walking, cycling, driving, swimming, boating, or climbing. Meanwhile, their brain activity was meticulously recorded.
The findings were striking. Parts of the visual cortex lit up in unique ways that couldn’t be explained just by recognizing objects or colors in the images. “These brain areas not only represent what can be seen, but also what you can do with it,” explains Groen. Even more fascinating, these patterns appeared even when participants weren’t asked to consider actions, suggesting the brain automatically registers affordances below the level of conscious thought. The concept, long a staple of psychology, is now proven to be a real, measurable process in our neural hardware.
But can artificial intelligence—trained on oceans of data—catch up? The team put state-of-the-art AI systems, including GPT-4, to the test. Even after training specifically on action recognition, AI models still struggled. Their internal calculations diverged sharply from human brain patterns, and their answers were often inconsistent or lacked the intuitive, physical connection we take for granted. “Even the best AI models don’t give exactly the same answers as humans, even though it’s such a simple task for us,” says Groen. “This shows that our way of seeing is deeply intertwined with how we interact with the world. We connect our perception to our experience in a physical world. AI models can’t do that because they only exist in a computer.”
This gap isn’t just an academic curiosity—it has huge real-world implications. As AI becomes ever more present in fields like healthcare, autonomous vehicles, and robotics, it’s vital that machines go beyond just recognizing what’s in a scene. They need to understand what can be done. Imagine a rescue robot navigating a disaster zone, or a self-driving car distinguishing a bike path from a driveway. These are tasks where human brains still reign supreme.
There’s also a lesson in sustainability. Today’s AI demands massive computational power and energy, accessible mostly to tech giants. But the human brain achieves its remarkable speed and flexibility on a fraction of that power. By studying the brain’s tricks—how it so quickly and efficiently spots action possibilities—researchers hope to guide the next wave of AI toward more human-friendly, energy-conscious designs.
For now, the human edge remains clear: perception isn’t just seeing, but understanding—and acting on—what’s possible. And that’s something even the smartest AI has yet to truly master.
Source: University of Amsterdam
Enjoying our articles?
We don’t show ads — so you can focus entirely on the story, without pop-ups or distractions. We don’t do sponsored content either, because we want to stay objective and only write about what truly fascinates us. If you’d like to help us keep going — buy us a coffee. It’s a small gesture that means a lot. Click here – Thank You!
