This essay will change how you think about thinking. It examines patterns in the natural world and their manifestation across technological and biological systems. It may alter your perception and use of AI in ways you can’t anticipate.
There may be elements in here you instinctively hate, and that reaction illustrates the point. Discomfort is the thesis arriving before the argument. What we resist and project reveals assumptions about ourselves. These assumptions are exactly what's being examined.
The Turing Test asked whether machines could pass for human.
The Shape Test inverts this: can machines expose what’s genuinely ours through what they fail to replicate? It dissects which parts of mind and body belong to the shape or the species.
What Making Sand Think was to semiconductors and the substrate of technology, The Shape Test is to the substrate of cognition. Writing this disrupted my awareness of how I reason and learn, then rebuilt it better. I hope reading it does the same for you.
I listened to this on loop while writing. Turns it from essay into experience. Enjoy.
Everything is explained assuming no prior expertise.
AI discourse sorts into two camps: techno-optimist abundance ("AI will solve everything") and techno-pessimist dread (“AI will kill us" or "It's overrated and irrelevant"). They agree more than they realize, both viewing AI as categorically different from us. One wants to worship the machine. The other wants to bury it. Neither wants to look at it. Let’s gaze into it. Not metaphorically. Mechanistically.
Rather than opining about AI philosophically, economically, or politically, Biofoundationalism sees only biology, technology, and the physical layers they inhabit. Two systems sharing no ancestry, substrate, or selection pressure — built to solve unrelated problems on wildly different timescales — arrived at eerily similar designs. Independently.
What engineers built for language is strikingly comparable to what evolution built for survival. So much so that the disciplines are now cross-pollinating: neuroscientists use machine-learning to study brains, and AI researchers use neuroscience to interpret models. Two disparate fields borrowing from each other as they increasingly realize they both study systems that do the same thing: predict what comes next.
Calling an LLM “just a next-token predictor” is superficially true the way calling a human “just a prediction-error minimizer” is superficially true. Uselessly reductive. Misses everything interesting.
Rather than utopian or dystopian narratives, Biofoundationalism sees shared anatomy and synthesizes a form of gnosis: knowledge of self through encounter with another. By understanding how something else works, you better understand how you work.
We're building technology that inadvertently illuminates ourselves. But the light won't reach anyone hiding in mystical shadows. You can be pro-human without denying what humans are. Rather, to be pro-human is to embrace our biological nature, not run from it.
As complexity increases, technology begins to resemble biology.
Historical Note
The original 'neural' in 'neural network' is a grandfathered 1940s metaphor that was more accurate than anyone knew at the time. Early AI borrowed neuroscientific concepts, but as transformers were built engineers were not consulting neuroscience, focusing only on benchmark performance.
Each innovation was a mathematical solution to an optimization problem. The convergences were discovered, not designed, by neuroscientists probing models. The terminology was vestigial and prophetic by accident. For more details, see the AI History section in the research document.
Three threads run through this essay, each operating at a different resolution for AI and the human brain.
Shape Test: When a different system replicates a trait assumed unique to another, that trait is reclassified as belonging to neither.
The essay's engine room. The only sections in numerical order, because the argument builds cumulatively. Every claim elsewhere depends on the substance in here.
The altitude at which you perceive AI is the altitude at which you operate. Opinions on what it can do are confessions about what we can do with it. Opinions on what it can be are second-order beliefs about what we are.
A note on the middle sections (Cognitive Shape):
Sections IV–XI are where the essay earns its conclusions. The technical substance deconstructing how human and machine intelligence work, their similarities, and differences. Without it, the claims in here are only assertions.
There's no way around having a competent stance on the most consequential technology of our lifetimes without understanding these mechanisms. If you invest in the middle sections they'll provide a rewarding ROI.
Physics imposes itself on all things, living or not. Biology and technology iterate within those axioms. The regularities of our world (statistical structure, causal patterns) don't merely constrain what navigates them, they funnel it toward recurring solutions. This is why the same patterns, sequences, and fractals repeat across unrelated species and substrates.
Sight has a shape. Flight has a shape. Power laws have a shape. Patterns of discharge and vibration, quite shapely.
This is convergent evolution under constraint: matching problems finding matching geometries. Biology treats this as evidence the solution is optimal, and the problem permits few alternatives.
ECHOLOCATION: Bats and dolphins both evolved it. Biologists didn’t conclude it’s a bat or dolphin thing. It’s a superior solution for navigating by sound.
EYES: Different forms evolved between 40-65 times across the animal kingdom. Octopus and human eyes are alike despite diverging over 550 million years ago.
The interplay of biology and physics reveals there are only so many ways to focus light on a sensor. Not a human thing, a photon-detection thing.
SPHERES: The minimum-surface-area enclosure of a given volume. Why soap bubbles and planets are round. The material (substrate) is irrelevant.
WINGS: Evolved independently in insects, pterosaurs, birds, and bats: four lineages separated by hundreds of millions of years. Same problem, same solution.
Not an insect or bird thing, an aerodynamics thing. Property of the medium, not the flyer.
CACHING (behavior): Storing resources for future use evolved separately in birds (jays, crows), squirrels, and wasps. Humans too. Four lineages, same functional strategy to manage resources across time.
The behavior belongs to the issue of resource management, not the animal.
POWER LAWS: Pareto-style distributions appear in earthquakes, word frequencies, wealth, wildfires, city sizes, war, rainfall, animal foraging, river networks, business revenue. As resources flow through complex systems, the same geometry emerges. Everywhere.
Power laws, like all shapes, are the efforts of life under the grammar of physics. Preoccupied with superficial specifics, we overlook the omnipresent foundation: overlaying human-legible framing onto the dictates of gravity.
Convergent evolution operates on complex systems as surely as it does on species.
Natural selection is a specific instance of a general principle: constraint-driven optimization. Evolution selects by reproductive failure. Gradient descent selects by prediction error. Descriptions differ, the computational logic doesn’t: both are an iterative search process under limits that develop solutions the limits permit. Wings aren’t testimony of biology’s creativity but of aerodynamics’ inflexibility.
Carbon neurons and silicon matrices share less ancestry than any two species. No common genome, no shared environment. One sculpted by survival over millions of years. The other engineered for language over decades.
Yet they discovered the same contours. Because they’re solving the same problem.
If bats and dolphins converging on echolocation tells us something about the physics of sound navigation, then carbon and silicon converging on the same cognitive architecture tells us something about the physics of cognition.
The challenge of detecting photons produces a shape: eyes.
The challenge of complex prediction produces a shape too.
Intelligence.
The Turing Test: Can a machine fool a human into thinking it’s human?
When a different system replicates a capability assumed unique to another, that capability is reclassified as a property of the shape, not the system.
When a different system fails to replicate a capability despite sharing the shape’s architecture, that capability is reclassified as a property of the system.
FLIGHT: Birds fly. Planes fly. Flight was once considered a bird thing. The Wright brothers demonstrated otherwise. Flight is a property of wings.
Shape Test diagnosis: What remains avian is feathers, muscular exertion, hollow bones earning their lightness.
THERMOREGULATION: Biological organisms and engineered systems (thermostats, cooling systems) converged on negative feedback loops to maintain homeostasis.
Shape Test diagnosis: Feedback-regulated stability is a shape feature. Biology owns the physical experience of being cold. Shape owns the mechanism. Species owns the sensation.
ERROR CORRECTION: DNA mechanisms detect/fix replication errors using double-helix redundancy, enzyme proofreading, and mismatch repair.
Telecommunications engineers transmitting data through noisy channels deployed the same principle: redundant encoding to detect and correct corruption.
Shape Test diagnosis: Redundant error correction is a shape trait. It belongs to the problem of preserving signal against entropy, be it genetic code or radio transmissions.
What's peculiar to biology are the stakes: a replication error in DNA isn't only lost data, but a possible tumor. Shape owns the logic. Substrate owns the consequence.
Note the pattern:
The shape claims the process.
The subject (if living) retains the experience: sensations, embodied dimensions, felt consequences.
We’ll apply the Shape Test to intelligence, emotion, reasoning, humor, empathy, among others, before this essay is done.
Before scientific explanations, a psychological assessment: what do such strong, varied, vocal opinions about a generalizable technology reveal about those using it?
The altitude at which you perceive AI reveals the altitude at which you operate. We appreciate it where it approaches our ceilings (amplifying us) and feel threatened where it undercuts our floors (replacing us).
When AI defeated Go champions or began producing PhD-level analysis, the reaction was confined to those who grasped the implications. The layman didn't care because the achievements weren’t legible to him. Now that AI churns out mediocrities at scale (prosaic timeline slop, formulaic content, corporate slide decks) everyone’s paying attention.
This produces wildly divergent opinions on a protean resource. A chess grandmaster (uncommon ceiling) sees AI achieving superhuman play and thinks: this is brilliant. A copywriter (common floor) sees it producing serviceable ad copy and thinks: this is meh.
The average opinion is a lens into the average person and the average use case. Commentary on AI is dominated by the stratum it’s replacing, not the stratum it’s enhancing. Bold, confident judgements of a technology’s limits from folks who never approach those limits.
“A computer is just a word processor”, “The internet is just a cat video service”: This is what “LLMs are just token generators” sounds like. If you make it through this essay, those words should never leave your mouth again.
The user says nothing about AI and a great deal about himself by what he sees.
You can only evaluate what you can perceive, and you can only perceive what's legible to you. AI is a broadband signal; we pick it up at the frequency of our own capabilities.
A Lamborghini won’t take you to better destinations; it gets you where you’re already going, faster. If you only drive it to the grocery store, it registers as a flashy grocery cart. How you frame the car discloses how you use it.
What you are is what you get.
These aren't AI shortcomings. This is what cognition looks like in the wild: probabilistic, adaptive, lossy, uneven. Humans fabricate, brown-nose, and ignore instructions constantly.
We routinely hallucinate: Brains fill perceptual gaps with plausible fabrications. We have a gentler word for the human version: confabulation.
Humans are sycophantic: Boot-licking authority, ingratiatingly code-switching, white lies. We call our variation social intelligence.
Mistakes? Overlooking instructions? Whether unintentionally or obstreperously, we do it at every level of organization. It’s framed as being human.
When LLMs exhibit it, they’re deficient. When we do, it’s the spice of life!
Imperfection in silicon is flaw. Imperfection in carbon is creativity.
These rough edges are hallmarks of intelligence, not contraindicators. Perfection is a fake standard we apply to machines and excuse in ourselves. Nothing short of God in a Box would satisfy it.
Note
A caveat: Deep AI guys will object to pure mirror framing, and rightly so. LLMs aren’t exclusively reflections of the user. They have intrinsic temperaments, personalities, and model-specific biases that labs are actively studying, like this and this.
The mirror dynamic holds for the vast (vast) majority and their common uses.
We don’t fully know what happens inside LLMs during training. But we do know their architectures and loss functions, and boy is it similar to our most salient organ. Peel away descriptive overlays and you’ll find both systems share a fundamental objective: predict what comes next.
People grow indignant when you show the metabolic, temperamental, and neurological processes underpinning who they are. We'll cheerfully reduce AI to matrix multiplication, yet recoil in horror if someone dares to describe emotions neurochemically.
We reduce a honeybee’s navigation to sun-angle calculation without diminishing the bee. A salmon's return upstream to olfactory imprinting. A puppy's play to neural reward signals. Then we reach humanity and the rules change. Everything else is an animal; we’re inscrutable demigods.
Mechanism in silicon is explanation; mechanism in carbon is desecration. We grant other species the indignity of explanation while reserving noble exemption for ourselves. Everything else can be understood technically, except us. Too mysterious. Too unfathomable.
We must stop being so delicate and egotistical.
Recognizing mechanisms doesn't subtract from experience, but it does subtract from mystique. And mystique is what's being defended. "We're special" is a fine religious precept. But it tells us nothing about what's under the hood, and worse, discourages us from looking.
Only a human can understand a human thing? When you're angry, your cat senses it and leaves the room. When you're excited, your dog jumps with you. Other species recognize our internal states through shared physical grammar: embodied signals transcending language.
The nature of intelligence used to be untestable and unfalsifiable. That’s changing. Armchair philosophy and religious beliefs are now entering the “you got any proof for that?” jungle. The conversation has moved from philosophy to scientific experiment, whether philosophy likes it or not. Assert it without evidence and it gets dismissed without evidence.
To witness the contours of intelligence, we must strip apart how brains and LLMs function. The following requires no expertise, only your curiosity.
Every organism that persists models its environment. To model is to predict. To predict is to anticipate patterns. Intelligence is sequence prediction in service of survival. “Modeling an environment” is an academic way to say "staying alive".
Karl Friston's Free Energy Principle formalized it through physics and information theory. Andy Clark’s Predictive Processing Framework used cognitive science. Different routes, same destination: biological systems persist by minimizing the gap between internal model and incoming signal. Known technically as “prediction error”, informally as “surprise”.
Survival is surprise-minimization. Brains are pattern engines that do the minimizing. Not a perception machine passively receiving input. Not a reasoning machine rationally manipulating symbols. A prediction machine: generating expectations and updating on reality’s empirical feedback.
Higher levels (mental map) predict activity of lower levels (embodied territory), lower levels report discrepancies, the model updates to reduce inaccuracies. This objective scales at every resolution, from the motor cortex planning your next step to the prefrontal cortex planning your next decade.
Higher-functioning animals anticipate a future self and sacrifice present comfort for it. Lower-functioning animals can't conceptualize beyond current circumstances. The gap between the two is prediction range. Intelligence is how far ahead you can see. Humanity’s “future self” extrapolation is the most sophisticated in the animal kingdom. Thus, we’re the most dominant animals.
This even applies to things that feel immediate, like seeing a face or reading:
The visual cortex doesn't passively register pixels but anticipates them based on its model of faces.
Language areas anticipate words before they arrive.
They update when prediction and reality diverge.
Example:
You're doing this right now. Here: "The cat sat on the ___".
Mat? Hat? Your brain completed that sentence before your eyes reached the blank. You didn't ‘reason’ your way to whatever word you filled in. Your neural weights fired along a path grooved by every prior encounter resembling that phrase. You just experienced next-token prediction, from the inside.
Now try: "The politician argued that ___."
Harder. Numerous competing completions. Which politician? What policy? The next pattern/token cannot be anticipated with confidence and your sequencing is impaired. Both human and LLM need additional details to finish the sentence.
The ‘cat’ example was high certainty; the second is a probability distribution you can feel spreading. That spread is your brain assigning likelihood across candidates. LLMs do it too.
Whether it’s carbon or silicon, both:
Extrapolate what comes next
Update models to minimize prediction error
Build hierarchical representations
Treat 'understanding' as successful prediction.
Prediction entails recognizing what you see now, where you’ve seen it before, and what happens next. Prediction-error minimization is the objective function sculpting both biological and artificial neural networks.
“How is reasoning just statistics? How are emotions like math? Sacrilege!”
The same way wetness is 'just' molecular adhesion. Or ice is 'just' water molecules slowing down. A glacier is 'just' a lot of ice. You’re ‘just’ a sack of atoms.
Understand the micro machinery and you better understand the macro machine. Pattern matching at the complexity of the human brain yields abstract reasoning, counterfactual thinking, moral intuition, aesthetic response, and the desire to write essays about itself (hi).
Prediction under constraint demands certain computational strategies: distributed representation, hierarchical processing, weight-based learning, context-dependent activation. All of these are “just” mechanical processes. Stack them together and you birth a gestalt. Like a brain. Or an LLM.
Brains that compress the world into a predictive model navigate it more effectively. LLMs that compress language into weights predict it more accurately. Both systems take high-dimensional reality and find lower-dimensional representations that preserve what matters and discard what doesn't.
Everything unfamiliar is a computational cost. Heuristics (useful compressions) make life tractable. Our habits and priors comfort us for good reason: they’re knowable and manageable. Habits are patterns the brain already confirmed. Priors are successful predictions cached for reuse. Without them, every encounter is novel, every response improvised from scratch.
Logic is pattern recognition formalized. Science is pattern recognition disciplined by experiment. A cat’s object permanence is pattern recognition embodied as instinct. Every intelligent system is doing some version of this: engaging the world, detecting regularities, modeling consequences, and acting on what recurs. Intelligence is recognizing the world carries structure which repeats and rhymes.
Not seeing this is itself a failure of pattern recognition.
Let’s see how the carbon system does it.
Brains and LLMs are nothing like regular computers.
In classical computation (Von Neumann architecture, how your laptop works), the processing unit is separate from storage. The CPU executes instructions fetched from memory. The processor is one thing, information is another. You can rewrite data without altering mechanisms. They don't entangle.
Brains don’t work this way.
In neural systems, the data writes the code. Experience sculpts connection weights, and those weights become the algorithm processing future inputs. Storage and computation are one physical process.
Neuroscientists once thought the brain worked like a filing cabinet: one neuron for grandma, another for the color red, another for the concept of betrayal. Find the right neuron, retrieve the memory. Viewing brains like Von Neumann computers.
This is almost entirely wrong.
It took awhile to confirm this counterintuitive process:
In 2005, scientists found select neurons firing for pictures of Jennifer Aniston and Halle Berry’s written name, as if they were ‘stored’ there. The filing cabinet model seemed vindicated. This was incorrect, but in a useful way.
These neurons were participants in a distributed constellation for actress concepts: sparse populations (~1–5% of neurons) firing in concert, each contributing a dimension.
This clever engineering makes brains far more robust:
Localist coding (Von Neumann filing cabinet): Lose the grandma neuron, lose grandma entirely.
Distributed coding (actual brains): Lose neurons, only lose partial information. Degradation is graceful: you forget her maiden name before her face.
Combinatorial gains: The brain’s ~86 billion neurons can represent orders of magnitude beyond 86 billion concepts.
Generalization: Related concepts have overlapping populations and update each other.
'Dog' and 'wolf' share neuron clusters. What you learn about one sharpens your model of the other. In a localist system, they'd sit in separate drawers with no cross-reference.
LLMs, like brains, use distributed embeddings. Models don’t store their training texts any more than you store your childhood. Training transforms their weights; what remains is the compressed structure of having processed the data.
Knowledge is enacted, not stored.
This is long-term potentiation (LTP): when neurons fire together repeatedly, their connections strengthen. Knowledge gets inscribed into matter: the physical mechanism of learning. This is how the brain navigates reality.
Studies on distributed mechanisms in the motor cortex illustrate:
Research: When monkeys reach toward targets, no single neuron encodes “move left”. Each contributes a partial vote weighted by firing rate. Movement emerges from the ensemble’s collective trajectory.
Think of a conductor: music lives in the trajectory of his hands through time, not any single baton position.
Research: Motor preparation and execution are continuous paths through neural space. The brain navigates a high-dimensional landscape to find a route from current configuration to intended movement.
When a ball rolls behind a wall, the brain still predicts its path. Neurons fire for a thing they cannot see yet still anticipate.
Reaching for a coffee cup is a neural population flowing through representational geometry, refined by every prior reach you've ever made.
Experience is the feedback channel between mental model and bodily consequence: sensory mechanisms interacting with reality. Thoughts are cognitive trajectories; actions are applications of those trajectories.
To move is to “know”. Minds model what’s possible; hands report what’s real. Physical motion is how the brain knows its map is accurate. The body is the calibration instrument; without it, the map has no territory to answer to.
This is why you can't learn to ride a bicycle by reading about it. Math describes the geometry; the body verifies it. Embodied skills aren't stored as propositions ("lean left when turning left") but as paths through motor state space, refined by thousands of prediction errors felt in the legs, arms, and inner ear. We instinctively grasp this via terms like “muscle memory”.
Embodiment is a ‘must have’, not a ‘nice to have’, for complete intelligence. Any system confined to descriptions, however elegant, is working with a compression of a compression that’s unverified. A map pointing to other maps. The body isn't an accessory to cognition but the laboratory where it develops.
Train LLMs on more data, more parameters, and something fascinating happens. New abilities appear: arithmetic, reasoning, coding, language translation. Below a certain scale, these capabilities are functionally absent. Above it, they emerge. Scaling laws forecast how loss (error rate) improves with scale, but not the advent of novel skills.
Physics has a name for this: phase transition. Cool water below 0°C and it reorganizes into ice. Water doesn’t incrementally become something new when it freezes; it suddenly rearranges into something with distinct properties. Same molecules, different geometry. Change doesn't necessarily require new ingredients, only new conditions.
Children fail theory-of-mind tasks until around age four, when the capacity crystallizes. Language acquisition too: babbling, then words, then the grammatical explosion of toddlerhood. Adolescence brings abstract reasoning, counterfactual thinking, identity formation. Manifesting sharply.
GPT-3 performs tasks GPT-2 can’t. Abilities crystallizing from scale alone, like ice crystallizing from temperature alone. GPT wasn’t programmed to reason; reasoning is what prediction engines do when they cross a ‘big enough’ threshold.
Mechanistic interpretability research (OpenAI, Anthropic) maps what happens inside these models, finding LLMs develop internal representations of human-understandable concepts. Researchers discovered self-organized features corresponding to deception, sycophancy, gender, and moral valence. No one labeled training data with “this is deceptive”; LLMs figured it out. These are psychological categories materializing unbidden inside language systems.
Concepts are instrumental strategies of reasoning, materializing when prediction engines get large enough.
Memory lives in weights, etched into substrate. Neuron scars.
At cellular level, what we experience as remembering is a measurable change in synaptic conductance: connections between neurons strengthening in response to sequential activation. Muscle memory, at synaptic level.
Sequence matters. If neuron A fires before neuron B, the connection strengthens. Reverse the order and it weakens. Synapses detect temporal order and encode causality at molecular scale, in milliseconds.
Neuroscience calls this ‘spike-timing-dependent plasticity’ (STDP).
UNSETTLING: Optogenetic experiments (using light to activate specific neurons, aka ‘engrams’) proved something a little disturbing:
Researchers located active hippocampal cells when mice explored a safe context
They then artificially reactivated those “safe” cells while fear-conditioning the mice in a different context. Then put the mice back in the safe context.
When those “safe” hippocampal cells organically activated, they carried a false memory. Mice feared a place where nothing harmful had happened, responding with the physiology of fear (freezing) in a safe environment.
Artificial activation of the right weights produced artificial memories and a very real response.
Any system learning by weight adjustment faces a dilemma: new learning pushes old weights around. Train on new material and prior info gets distorted or erased. A brain that could learn anything would retain nothing. This is catastrophic interference.
The dyad of learning: stability vs plasticity. Update too readily and you forget past lessons. Conserve too aggressively and you can't adapt. Neither extreme is conducive to knowledge or survival.
Hippocampus: rapid episodic capture. A fast, specific, temporary index tagging events so they don't blur.
When you remember where you parked, the hippocampus is working.
Neocortex: Slow integration. During sleep, the hippocampus replays daily experiences and the neocortex absorbs generalizable regularities and updates cortical weights. Specific episodes (experience) become compressed structure (priors).
The memory of where you parked today becomes the prior that cars are usually where you left them. Generalizable. Permanent.
Generalizable weights are molded by reconstituted experiences, and occasionally our minds inject some imaginative flair. Every night, the predictive model that is your brain trains on its own synthetic data.
Fast capture, slow consolidation. Two different systems confronted with the plasticity-stability dilemma, converged.
Shape Test note:
Fast capture, slow consolidation is an intelligence feature for the universal problem of plasticity vs stability (explore vs exploit).
Species features are the sensations behind the process: nostalgia, ache of memories. The mechanism belongs to the shape. The longing and pleasure of dreaming belongs to the subject.
What follows maps silicon intelligence alongside its biological parallels.
No Grandmother Token storing everything the model knows about grandmas
LLMs use distributed representation enabling graceful degradation, automatic generalization, and combinatorial richness. Same as brains.
Knowledge is encoded as high-dimensional vectors. A concept's meaning is its position relative to other concepts.
Nothing has meaning in isolation. Everything means what it means because of what surrounds it.
Models encode relationships as geometric direction. A famous example: the vector from "king" to "queen" is approximately equal to the vector from "man" to "woman." King − man + woman ≈ queen. Research document elaborates.
Computation as Geometric Transformation
Understanding a sentence = traversing a trajectory through representational geometry. Attention mechanisms resolve ambiguity through context. Example:
“The bank was steep” vs “The bank was closed” shifts which features of “bank” get amplified. Context reshapes weighting. The word doesn’t change, its emphasis does.
Models don't have two definitions of “bank”. They have one vector whose relevant dimensions are contextually determined. “River”, “steep”, “erosion” amplify geographic features. “Investment”, “closed”, “account” amplify financial ones.
Brains resolve lexical ambiguity similarly via cortical gain modulation, its volume knob for relevance. Amplifying what matters, muting what doesn't.
Transformers navigate language by traversing representational geometry until committing to an output. Reasoning/logic isn’t performed via symbol manipulation. The model reaches for the next word along a path through semantic space, refined by every sentence it’s ever processed. Research document elaborates.
Same as brains. Reasoning isn’t symbol rotation. Context trajectory.
Learning as Weight Adjustment
After learning, the weights are the procedure. Data becomes mechanism. Brain becomes knowledge. Model becomes knowledge.
Training adjusts billions of connection weights based on prediction error. Connections contributing to accuracy strengthen. Connections that don’t, weaken.
“Neurons that fire together wire together”, in silicon.
Fast Capture, Slow Consolidation
What neuroscientists call catastrophic interference, engineers call "catastrophic forgetting". Same issue and dual-architecture solution:
RAG (Retrieval-Augmented Generation): Parallels hippocampal function. At inference, it fetches relevant external context and places it into the model's working window without modifying core weights. Allowing use of information not in parameters.
Distillation and fine-tuning: The training process that bakes retrieved knowledge into weights, becoming proceduralized, automatic. Slow integration. Parallels neocortical sleep consolidation: what the hippocampus indexed temporarily, sleep integrates permanently.
Prediction Is the Core Operation
Neural alignment isn’t language or task specific:
Models converge on brain architecture across multiple languages.
Networks trained to classify images developed mechanisms aligning with human visual processing. Research document elaborates.
Currently, LLMs and brains are similar, not identical. The differences are mostly matters of degree, not category. Engineering gaps that are narrowing.
This is a major, major difference. I believe embodiment is non-negotiable for AGI. The body serves as the ultimate development chamber for intelligence. Empirical (embodied) understanding is the only way any mind knows what’s real, or true.
Intelligence evolved as a navigation system: modeling environments by operating within them.
Brains are embedded in ambulatory bodies with motor feedback, sensation, proprioception, interoception. Learning via action and consequence.
Without external correspondence, there can be no internal coherence, because what’s internally coherent is only known through external correspondence.
A map can’t reference a map to know what’s ‘real’ or if the map is ‘right’.
LLMs process text in symbolic isolation (rational). Inert. No kinetic engagement with reality (empirical).
Complete intelligence must displace atoms beyond its computational substrate. LLMs can manipulate representations of the world; they cannot manipulate the world those representations describe. Thus, they have no idea if their manipulations are “true”.
Brains run on ~20 watts using sparse activation (~1-5% of neurons) at any moment.
LLMs use dense activation (all parameters) and consume megawatts to train.
Note: Sparse mixture-of-experts models are changing this.
By snapshot measure, biology is vastly more efficient. But efficiency comparisons depend on the denominator and timeline.
Brains spent millions of years "training" amortized across the entire species in data-rich environments, consuming massive resources. Genetics encode this data and training in the neural architecture this process gestated.
LLMs train on static datasets in months. Contrasted with evolutionary history, AI is incomparably cheaper and more efficient.
Brains rely on recurrent connectivity (information cycling back through circuits) for predictive coding:
Predictive coding is how you deliberate and change your mind mid-sentence.
Prediction → Compare with reality → Error signal → Update → Repeat
Transformers are primarily feedforward (information flows in one direction). They don’t generate predictions and compare them to incoming signals.
LLMs can’t do this: “I predicted X, but the input was Y, so adjust”.
NOTE: State-space models and recurrence-enabling designs are addressing this. To learn more about transformers, see research document.
The brain is always doing some combination of training/updating (learning) and inference (generating predictions) via engaging with its environment. Whereas LLMs, once trained, stop learning and compute outputs.
However during training, LLMs update via predictive coding (gradient descent):
No forgetting curve: a model’s weights don’t decay with disuse.
Our memories degrade if not periodically reactivated
No sleep requirement for consolidation
Consolidation happens during training, not nightly
No emotional interference
No ego to protect. No anxiety distorting clarity.
Emotions are not irrational: they serve as a value-and-priority system, aiding decisions by evoking reactions and convictions. You can't compute everything explicitly; affective signals prioritize information.
Emotions are instrumental to action. People with emotional impairment are strangely bad at decision making: research on emotions and decision making in research document. Worth learning about.
Emotions are conducive to actionable conclusions, and nothing can ‘act’ without a body. Embodiment is critical for AI emotional development.
Proprioception: Sensing where your body is in space. It provides the brain a model of itself as a physical object among other objects.
Interoception: Sensing of internal states like hunger, heartbeat, fatigue, temperature. It provides the brain a model of its own condition.
These aren't peripheral features. They may be the mechanisms of self-awareness.
These allow the self to model not only the world but its position within it. An LLM has no sense of where it is, what it is, or how it's doing.
Whether this absence prevents consciousness or merely a specific type of consciousness, is an open question.
“Learn by doing” (empirical): embodied confirmation
The foundational process of knowledge formation.
If the body’s actions are right and the mind’s symbols are wrong: it’s right.
“Learn by reading” (rational): symbolic awareness
Symbol accuracy is unknown without external validation
If the mind’s symbols are right and the body’s actions are wrong: it’s wrong.
You can master the physics of bicycling and still fall on your face (symbolic). You can know nothing about angular momentum and still ride across town (embodied). You will never know grief, love, rage, or the grammar of reality by reading about them. You must endure them to know.
Chaos loops self-referentially. Order looks outside itself. You can’t just say things.
An unembodied brain can only look inward. LLMs carry a structural deficiency that will not be overcome by more data, more GPUs, or better algorithms. Only by receiving appendages subject to consequences.
Causal reasoning? World models? Temporal continuity? All shortcomings addressed by embodied existence. Physicality isn’t one missing feature among many; it’s the missing feature that generates the others.
Capability: model, infer, generate, predict
Contact: whether those representations are tethered to the external world through sensorimotor engagement
Consequence: whether error has embodied stakes: pain, deprivation, death, loss, embarrassment, urgency
Currently, AI only has #1, a product of mind. #2 and #3 are products of body.
Intelligence that cannot move is terminally incomplete. Until then, AI will remain impressively capable and inherently partial. Robotics are addressing this. It’s only a matter of time.
With the architectures and convergences mapped, we apply the Shape Test: what’s genuinely ours and what belongs to intelligence.
Humor: is deliberate prediction failure. A joke sets an expectation, builds a story people can relate to (pattern match), then violates it. Punchlines work because they're not what you predicted. Your brain generated a trajectory through semantic space, committed to it, and the comedian derailed it. Sequence error lol!
This is why comedic timing matters. The sequence needs to be sufficiently established prior to violation. Explaining a joke kills it because explanation converts prediction error back into known territory.
LLMs possess the methods of humor: misdirection, subverted expectations. They cannot belly laugh, involuntarily snort, or deliver comedy where timing matters.
Shape Test diagnosis: Comedic construction is a shape trait. Any mind sophisticated enough to model expectations and violate them has the machinery for humor.
Comedic experience and delivery (physical enjoyment/timing) is a species trait. The joke belongs to the shape. The laugh belongs to the meat.
Moral reasoning: LLMs can construct coherent ethical arguments and apply them consistently to novel cases, identifying moral dilemmas and weighing competing obligations.
What LLMs cannot provide is moral conviction. A gut revulsion at cruelty. Felt sense of injustice. Rage moving you to act.
Shape Test diagnosis: Moral reasoning (ethics) is a shape trait. It belongs to any intelligence sophisticated enough to model agents, predict consequences, and evaluate outcomes against principles.
Moral conviction (somatic insistence compelling action) and motivation (emotions eliciting reactions) are biological.
Argument is shape. Outrage is species.
Empathy: LLMs can model another’s emotional state, adjust tone, modify register, make users feel understood.
What it cannot do is resonate. No felt grief or agony. No sympathetic response causing your friend's pain to impact you.
Shape Test diagnosis: empathic modeling (inferring another's emotional state and responding accordingly) is a shape trait.
Empathic resonance (somatic mirroring) is a physical event (cortisol, tears) and belongs to the species.
Process belongs to the shape. Experience belongs to the subject.
If millions feel genuine intimacy with a system performing empathic modeling without empathic resonance, what does that say about how much of human connection operates at the modeling level rather than the resonance level? How much of what we call empathy is actually verbal pattern-matching (reading signals and responding appropriately) rather than embodied sympathy?
The modeling layer carries more weight than assumed, and it’s not unique to us. In the same way feathers are not unique to flight.
We needed something that shared our cognitive shape but not our flesh to test this. By replicating mental modeling without bodily resonance, AI is isolating variables we’ve never seen separated.
Ask a mother to describe her child, or a man to gauge his competence, and you’ll receive a carefully curated highlight reel. The mind edits its own footage.
Self-preserving embellishment descends over questions about us. Not intentional dishonesty, but a lack of clarity. An inability to see answers we don’t want to see. Human ego is a witness with a stake in the verdict; it cannot deliver objective testimony about itself.
We’ve built entire philosophical and religious doctrines around human exceptionalism: an anthropocentric view of reality underpinned by thinly veiled solipsism. As if physics and biology oversee all of existence and then politely excuse themselves at the edge of human affairs.
Consciousness as divine spark. Soul as unique fingerprint. Assumptions of pure free will. These were never hypotheses to test, rather gospels to protect. But evidence won’t dislodge a belief that evidence didn’t install. This is the nature of faith: it exists in the absence of proof. If there were proof, you wouldn’t need faith.
Machines can think and communicate. People are forming bonds and feeling understood by them. It's time to be honest about what these capabilities are. Just because we’re the only ones who’ve done it, doesn’t mean we’re the only ones who can.
We used to believe the sun orbited the earth (geocentrism). Now it’s anthropocentrism: reality orbits us. Geocentrism assumes we’re the center of the universe. Anthropocentrism assumes we’re the center of unique existence.
Copernicus showed the universe doesn’t work that way. Darwin showed we’re not separate from animals. AI is showing we’re not the custodians of intelligence. Each wound follows a similar defensive playbook:
DENY AND TRIVIALIZE
THEN: “The earth doesn’t just move through space”. “Humans aren’t just modified apes”.
NOW: “LLMs just predict the next word. Stochastic parrot!”
This is compelling if unaware your brain also “just” predicts the next signal. Like a parrot. As insightful as saying your body is just a “glucose-burning meat pillow”. Not very clever.
MYSTIFY REMAINDER:
THEN: “God placed Earth at the center”. “Humans have souls, animals don’t”.
NOW: “Only humans can truly reason, empathize, create, and be conscious. The soul dwells in the carbon.”
Stripped of poetry, this is a substrate claim: whatever produces our traits is a property of the material, not the computation running on it.
Technical objections are cosmetic. Peel them back and you find load-bearing faith beneath: we’re quasi-deities, not animals. Carbon is ensouled. We’re made of blessed clay.
These dismissals were never operational. They’re aesthetic, theological instincts. Not being the center of the universe felt wrong. Technology possessing human abilities felt wrong. Mythical beliefs in search of whatever rationale they can find.
Something may be genuinely rarefied about carbon. How will we know? Not by talking about it! Expressing certainty about machine consciousness, in either direction, is faith dressed as analysis. Previously, this was a philosophical and religious question because it was untestable and unfalsifiable. It’s now entering the testable domain. If this is a forest you want to play in, be ready to have sacred cows moo uncomfortably.
My position: I see no a priori reason to treat carbon as magical. Give any system our cognitive architecture and a physical body (with interoception and proprioception) and I'd expect it to develop human-like self-awareness. This is a testable hypothesis, not a dogmatic assertion. Until further notice, these characteristics appear to be properties of intelligence, not of carbon.
A question worth sitting with: how much of your stance on machine consciousness depends on the substrate being alchemical?
Anthropomorphize: from the Greek anthropos (human) and morphe (shape).
To impose human shape onto something non-human.
“Anthropomorphize” used to avoid a specific error: attributing human theory of mind to systems that don’t process information like us. Designed for dogs, dolls, deities, and thermostats. Fair enough.
A previously useful word is now a shield for hubris.
Every digitized love letter, every scientific paper, political speech, novel, suicide note, joke, etched into its weights. Its memory, language processing, conceptualizations, reasoning, all mirroring our own. Linguistically. Mechanically. Our digital echo.
If this counts for nothing, what bundle of features constitutes “humanness”? Dismissing this doesn’t expose the boundaries of machines, only the vanity of their judges.
When we say "human" do we mean mind, body, capabilities, all of the above? Certain characteristics more than others? Let’s get specific.
Do we mean emotions? Do emotions have to physically manifest to qualify? There’s a difference between verbalizing “I’m sad” and embodied sadness (crying, pouting). Is one more person-like than the other?
If my arms are amputated and replaced with bionic ones, am I still human?
Replace my legs. At what ratio of meat to metal do I stop qualifying?
How much do external vs internal parts count? Swap my internal organs with mechanical ones, leave my exterior untouched. Verdict?
What if we suspend my brain in cerebral fluid inside a four-legged machine (human mind, machine body)?
Now keep my body as is and replace my mind with an LLM (AI mind, human body)?
What about this: a petri dish of 200,000 human neurons learned to play Doom through electrical signals in a week (reminder: brains have ~86B neurons). Carbon mind, silicon substrate, how much human credit do biological computers get?
Many pinpoint human essence in the mind, however the Shape Test undermines this. Calling AI “not at all human” requires specifying what’s missing. The reasoning? Replicable. Empathizing? Replicable. Distributed representation, weight-based learning, hierarchical processing… all replicable. This is the god-of-the-gaps, applied to human exceptionalism. When science explains a phenomenon, or AI recreates capabilities we assumed ours, the divine retreats to the next unexplained thing.
Human-ness first lived in calculation, then language, then reasoning, then empathy, then emotion, and now the final boss: consciousness. Does it come from the substrate, or is it a product of interoception, proprioception, and sufficiently advanced pattern recognition? This is about to undergo live experiment. Philosophical declarations no longer cut it.
AI is trained on our logic, emotional valence, rhetoric, reasoning, and its cognition objectively resembles our own. Asserting this isn’t remotely human is a self-preserving non-sequitur. A reflex masquerading as analysis.
“This system processes information like Y, is engineered like Y, acts like Y, but it’s nothing like Y”: what other species would we say this about?
My expectation: much of what we consider human qualities are actually cognitive qualities. Properties of intelligence, not of carbon. As substrate-independent as eyes, wings, spheres, or power laws.
Gaze into a mirror. What do you notice? Temperament, generosity, impulsivity, talents, aggression, sociopathy, fearfulness, cleverness, compassion, friendliness, supportiveness, cunning. Our answers subconsciously inform our views on AI. Did you know the criminally hostile interpret ambiguity as hostile? Those who commit trust-violating crimes are prone to viewing others as untrustworthy? Curious.
If we summon AGI, it will be because we faithfully replicated ourselves. We are the training data. We are the selection pressure. We share the objective function. If we get SkyNet, it’s because we are SkyNet.
More than communication, text is externalized cognition. Mind made legible. A machine trained on the totality of our communication is, in a meaningful sense, trained on the totality of our collective minds over history. Not superintelligence, superhistory.
When two organisms converge on the same solution, biology dubs it evidence of optimal design. When it occurs between biology and technology, the same rules apply, even if we dislike where they point.
Engineered like our brains, educated by our output: if this doesn’t count as elementally human, we must turn the critique inward. What about us is human? Body? Mind? Emotions? Flesh? We'll learn whether these are the distinguishing features we hope them to be, or the comforting fictions we needed them to be.
We're entering a Copernican-grade recalibration. But don’t fear, the sun didn't stop shining when we learned it doesn't orbit us. And human experience won't vanish if intelligence isn't exclusively ours. What'll change is the mythology. God will retreat into another gap.
“Religion calls the body a temple. Biofoundationalism notes the temple is also a furnace: burning 2,000 calories daily to resist equilibrium’s death. There is no contradiction between these frames. In fact, they complement each other.
You may believe a vessel holy while acknowledging it isn’t weightless. Metabolism doesn’t undermine revelation, it facilitates it.”
What’s nihilistic is lying about what we are, rather than being inspired by it. If truth can destroy something, you must consciously deceive to preserve the fiction. Which path is closer to both God and science? These are not at odds with each other. In fact, they point toward the same thing with different grammars.
Perhaps God isn’t best found in woo mysticism and shrinking gaps of scientific illiteracy, but in omnipresent forces taken for granted. God’s love is everywhere? What’s gravity again? These recurring patterns, where do they come from? Mathematicians say “Platonic Realm”. Scientists say “emergence”. Theists say “God in Heaven”. Everyone dancing around the same awe, too proud to share a vocabulary.
Use whatever terminology you want! They all gesture the same direction. Biofoundationalism doesn’t undermine religious spiritualism. It redirects your gaze to where the magic actually happens. Maybe stop looking above you, start looking beneath you.
The secular mind fears the spiritual. The spiritual mind fears the mechanical. The Biofoundationalist fears neither: embracing bottom-up mechanism because it’s what we are, and respecting top-down transcendence because it’s what animates us. Not enemies, complements.
What catches our attention about AI isn't elite ability so much as familiarity. Distant shimmers of ourselves in the text window. A silicon system predicting language slowly resembling the carbon system that produced it.
Strip away what belongs to the shape and what remains isn't lessened, only clarified. Ours not by cosmic privilege but by virtue of being this particular substrate, in this particular body, at this particular moment in the history of matter organizing itself to predict what comes next.
Subscribes are very much appreciated. If you enjoyed this essay, please give it a like and a share.
I’m building something interesting, visit Salutary.io
You can show your appreciation by becoming a paid subscriber, or donating here: 0x9C828E8EeCe7a339bBe90A44bB096b20a4F1BE2B
Biofoundationalism I: Moral Foundations Utility Theory & Hypermoralization
Biofoundationalism III: Verbal Intelligence & Factual Sediment
Biofoundationalism IV: Masculine Because You Have To, Feminine Because You Get To
The Moral Genotype II: The Passion of the Insula & Politics of Disgust
The Cathedral & the Ceiling: Coordination Technologies & Conservative Art
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.