What is Fundamental Intelligence? Can AI achieve it?

World-Class Neuroscientists Debate the Future of Intelligence.

“Everything that we do… can be explained in terms of resolving uncertainty through information seeking.” - Karl Friston

Intelligence is not a neutral or objective property. It is, as Summerfield puts it, a normative concept, constructed by psychometricians to value certain behaviours over others, and therefore always shaped by what society has decided to prize.

This is an uncomfortable starting point for a field that increasingly treats intelligence as something to be engineered. If we cannot agree on what intelligence is, how are we expected to build and experience it? And yet that is precisely what the AI industry is attempting, at extraordinary speed and scale, with over $700 billion committed to AI infrastructure in 2025 alone.

“Information Is All You Need?” — Friston vs Summerfield

It was against this backdrop that the British Neuroscience Association convened a debate between two of the world’s most influential neuroscientists: Karl Friston and Christopher Summerfield.

The question on the table — is information all you need to create intelligence? This article will explain the two fundamentally different accounts of what intelligence is, where it comes from, and whether artificial systems can ever truly possess it.

Karl Friston is the most cited neuroscientist alive, and creator of the Free Energy Principle — a mathematical framework that attempts to explain all of perception, action, learning, and even evolution under a single unifying idea. He arrived at the debate representing what he calls the “neat” position, the belief that a single governing principle underlies intelligence at every scale.

Christopher Summerfield, Professor of Cognitive Neuroscience at Oxford and Research Scientist at Google DeepMind, took the opposing “scruffy” view, that intelligence is not the expression of one grand principle, but a set of practical solutions shaped by the specific demands of the world.

The tension between these two positions determines how we should build AI, what we should expect from it, and whether the systems we are already deploying deserve to be called intelligent at all.

“Karl has spent the last 20 years arguing that the brain is best thought of as a generative model. For 19 of those years, everyone thought this was an eccentric position… and then large language models happened.”

— Christopher Summerfield

This remark is a useful entry point. It captures something important: the divide between these two arguments is not about whether neuroscience is relevant to AI. Both sides agree that it is. It is about the level at which that relevance operates. Is intelligence best understood from the top down, through first principles, or from the bottom up, through the accumulated solutions that evolution and engineering have independently converged upon?

The Free Energy Principle (FEP)

Friston’s central claim is that the brain is an active prediction machine. At every moment, and at every level of the nervous system, the brain is generating a model of the world and comparing that model against incoming sensory signals. It is perpetually trying to minimise the gap between prediction and reality - a quantity Friston calls surprise, and more formally, variational free energy. The word surprise here is not the everyday concept; it is a precise mathematical concept measuring how improbable a given sensory signal is, given our current model of its causes. On this account, the brain has one fundamental drive: to make the world less surprising by continuously updating its model of what is out there.

This idea has deep roots in theoretical neuroscience. Rao and Ballard’s landmark 1999 study of the visual cortex showed that higher brain areas send predictions downward to lower areas, while lower areas send prediction errors back upward — displaying the residual mismatch between what was expected and what arrived. Friston’s contribution was to generalise this into a unified principle that operates across the entire brain, and across all timescales. Perception here is the named process of updating predictions to reduce error, and learning is the slower process of updating the model itself. Even attention can be understood as the brain adjusting how much weight it gives to prediction errors from particular sources.

What makes Friston’s framework radical is the extension of this logic from perception to action. If the brain can resolve surprise either by updating its model or by changing the world, then action becomes a form of inference. Rather than selecting behaviours because they lead to reward, an agent acting under the free energy principle selects behaviours because they are predicted. They correspond to the future states the agent’s generative model expects to inhabit. This is what Friston calls active inference, and it is the move that makes the FEP a key theory of the whole of intelligent behaviour, not just cognition. As he put it during the debate: “Everything that we do, we perceive, we are, can be explained in terms of resolving uncertainty through information seeking.”

However, the FEP is not a theory in the falsifiable sense. It is a principle, closer in character to the principle of least action in physics. It does not, on its own, make specific testable predictions. Friston acknowledged this distinction explicitly in the debate. This unfalsifiability at the principle level is one of the most contested aspects of the framework, and it sets up the sharpest point of disagreement with Summerfield.

What the World Demands

“If we want to get down and dirty and build a brain, we need to think about how the world is.”

— Christopher Summerfield

Summerfield does not begin with the brain. He begins with the world. His argument is that intelligence is a set of solutions that evolved in response to the specific, non-accidental structure of the environment. The world has properties, and those properties are not arbitrary. Time flows in one direction and cannot be reversed; hence, any agent navigating the world must carry information about the past into the present. Space is three-dimensional and roughly continuous, so agents must build representations not just of objects but of the spatial relationships between them. The world generalises; what was true before tends to be true again, but it also has exceptions, singular events that resist generalisation and must be stored specifically. We are also social animals, which means our brains have evolved dedicated computational machinery for modelling other minds. And our bodily needs recur with regularity, to perform actions like showering, eating, etc. This means intelligence cannot be purely intellectual; it must also be motivational.

Summerfield turns this analysis onto the AI systems that currently exist and argues that they succeed because their architectures have independently arrived at solutions to the same problems. For example, the transformer, the architecture underlying every major large language model, relies on a mechanism called self-attention, which functions as a form of content-addressable memory: it learns associations between all tokens in a sequence and retrieves relevant context dynamically. Summerfield notes that this is not dissimilar to how the medial temporal lobe and neocortex interact in biological brains to arbitrate between generalisation and the recall of specific episodes. Transformers were not designed with neuroscience in mind, but in solving the engineering problem of handling sequences over time, they arrived at a computational structure that rhymes with the brain’s own solution. This convergence, Summerfield argues, is what happens when you solve the right problems.

Where both arguments differ most is in what this implies going forward. Friston’s framework suggests that if you get the principle right, the architecture follows. Summerfield inverts this: understand the demands of the world first, then ask whether your system genuinely meets them, not just in principle, but in practice. The implication is that the gap between current AI and genuine intelligence is not a matter of the wrong equations, but of missing components: embodiment, active information-seeking, richer memory, and the capacity to model other agents. These are all human traits which are hard to replicate.

---

Where They Clash

I have structured the debate below in a playwright style, so it is easier to follow. This is a paraphrased version of what was discussed.

Summerfield: The problem with the Free Energy Principle is not that it explains too little. It may explain too much. It can describe how an intelligent system updates itself in response to uncertainty, but it struggles to tell us what that system should care about in the first place.

Intelligence is normative. We call certain behaviours intelligent because we value them. Curiosity. Planning. Cooperation. Survival. But if those values are never specified, then minimising surprise alone gives no moral direction at all.

Friston: But those values are specified. They are encoded in the priors of the generative model. An organism expects itself to occupy certain kinds of states and avoid others. What we call value is simply the statistical structure of the states an agent is disposed to inhabit.

Summerfield: That creates another problem. If any value can simply be inserted into the prior, then the framework itself provides no guidance about which values matter. A system minimising surprise may optimise perfectly for its own predictions, while remaining completely indifferent to human well-being.

And for artificial intelligence, that distinction becomes critical.

Friston: The mistake is assuming reward itself is fundamental. Reinforcement learning treats reward as the primitive force driving behaviour. But reward is not primary. It is secondary. It reflects prior beliefs about preferred states.

Summerfield: Yet reinforcement learning is currently the paradigm producing the most transformative AI systems we have. AlphaGo. Modern agents. Large-scale optimisation systems. These systems work because they maximise reward successfully in practice.

Dave Silver, the architect of AlphaGo and one of the intellectual fathers of the reward-maximisation framework, raised over £1 billion for a company built on precisely that principle earlier this year.

Friston: Enough for engineering, perhaps. But not enough for understanding intelligence itself.

Summerfield: And this is where the disagreement becomes deeper than AI architecture. The question is no longer just how intelligent systems operate, but what intelligence fundamentally is.

Is intelligence a universal property of self-organising systems, reducing uncertainty?

Or is it a goal-directed capacity that only makes sense relative to what an agent is trying to achieve?

---

The debate never fully resolved that question, but it revealed something more important: beneath the technical language of priors, rewards, and generative models lies a philosophical divide about the nature of mind itself, one that may ultimately determine how we build the next generation of artificial intelligence.

What Does This Mean for AI?

Friston’s diagnosis of current AI is unsparing. Large language models run on what computer scientists call von Neumann architectures, chips in which computation and memory are physically separated, and where roughly 99% of energy is consumed simply moving data between them. Real biological intelligence does not work this way. Neural computation is in-memory: processing happens where the data lives. This is the principle underlying neuromorphic computing, where hardware architectures are modelled on the brain’s own organisation. It is where Friston believes the future of genuine AI lies: small, efficient, embedded systems that operate based on active inference rather than passive pattern-matching on vast datasets.

“I don’t think the current direction of travel is sustainable,” he said plainly, pointing to the data centres consuming hundreds of billions in investment as a symptom of building intelligence the wrong way.

Summerfield does not entirely disagree with the sustainability concern, but his prescription is different. Where Friston sees LLMs as architecturally wrong in principle, Summerfield sees them as incomplete. The transformer architecture already embodies several of the right computational solutions: content-addressable memory, generalisation over structured representations, socially-steered objectives through reinforcement learning from human feedback. What it lacks is embodiment, active information-seeking, and the kind of efficient world-modelling that comes not from scale but from constraint. His argument implies that the path forward is not to abandon current AI paradigms but to extend them: give these systems bodies, goals, and richer memory, and watch what they become.

---

Can AI Achieve Fundamental Intelligence?

Neither speaker gave a clean answer to the title question, and that irresolution is itself the most honest finding of the debate.

From Friston’s perspective, genuine intelligence requires active inference: a system that does not merely respond to inputs but acts on the world to bring about the states it predicts. Current AI does not do this. It is, as he put it, fundamentally passive, sophisticated at prediction, but not genuinely engaged with the world in the way that would make its predictions self-fulfilling.

From Summerfield’s perspective, the question is whether a system meets the demands of the world it inhabits: whether it has the memory to carry the past into the present, the spatial representations to navigate structure, the social models to operate among other agents, and the motivational systems to sustain itself. Current AI meets some of these demands partially. It does not meet all of them fully.

What the debate made clear is that the question “Can AI achieve fundamental intelligence?” cannot be answered until we have resolved what fundamental intelligence is. The audience voted, at the end of the debate, for Summerfield’s view that the world’s structure is the right starting point for building intelligence. However, the more significant convergence was the one neither speaker had to argue for: that intelligence, whether in neurons or in silicon, is fundamentally about managing uncertainty in a structured world. The question that remains, and that the next generation of AI researchers will have to answer, is whether understanding that principle is enough to build intelligence, or whether the act of building will always reveal what the principle alone could never anticipate.

The next article will be: How is knowledge organised in the brain? Do LLMs work the same way? This will focus on insights from Tali Sharot’s Research.

Thank you for reading this and subscribing to The Agora Collective Substack.

References

Clark, A. (2016). Surfing Uncertainty: Prediction, Action, and the Embodied Mind. Oxford University Press.

Friston, K. (2010). The free-energy principle: a unified brain theory?. Nature Reviews Neuroscience.

Rao, R. & Ballard, D. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra- classical receptive-field effects. Nature Neuroscience.