Three things a system needs before it’s an agent.

OpenAI’s innovation — more or less, transformers at scale — is a very strong prediction and reasoning machine. But they haven’t built AGI. It’s fair to say that a strong prediction and reasoning machine, getting better and better alone, will not create AGI.

If you define AGI as the ability to perform emergent tasks andthe ability to interact with the world independently of intervention by other intelligence, I don’t think greater reasoning and predicting alone will get us there.

My oversimplification is that there are three things a system needs:

1. Directional Adaptive Scrutiny
2. Memory
3. Propagation

When I was a kid, I remember thinking really hard about what needs to happen to create intelligence that is emergent. How do you get an atom to attract more atoms, bacteria to work together, and of course cognitions to create emergent behavior?

I thought the best question was: why don’t other things become dramatically more intelligent, the way humans do? Why don’t all animals become super-intelligent eventually? If they all have the same biological resources, why did only a few make it? (Let’s assume there were a few humanoid species that made it — Homo sapiens were just the most violent.)

I was also curious why specific cultures and geographies had greater inventiveness relative to the same species in different cultures and geographies.

Whether it was geography or culture dictating outcomes, there was a common factor: the environment something exists in dictates its ability to adapt and its social scrutiny. Too harsh and it withers and dies; too easy and it has no evolutionary incentive to get better.

You see this with cultures near the equator with near-perfect weather, relative to cultures with extreme climates. Inventiveness and new emergent behavior weren’t a nice-to-have but a matter of survival. And it wasn’t just that the climates were more extreme — it’s that they were cyclical (partially predictable), diverse, and extreme. Predictability matters: a culture can only get as advanced as its ability to advance between unpredictable resetting events.

Environment can be summed up as a form of scrutiny — effectively dictating “adapt or die,” giving real-time feedback like “no, that won’t work here” and “yes, that will help you survive.”

The second variable in geographic inventiveness is cultural scrutiny. Humans have a unique mating criteria that is dynamic, ever-changing, and never consistent. If you asked 100 women and 100 men in 100 different geographies over 1,000 years to rank attraction to 200 of the opposite gender, there wouldn’t be consensus. One era found overweight individuals a sign of wealth, another a sign of poverty; some found certain tattoos a sign of X, others of Y.

So scrutiny isn’t just “this is objectively good, this is objectively bad.” It’s dynamic scrutiny that’s ever-changing, never consistent, but predictable.

In ML we already understand this — it’s effectively what reinforcement learning does. The problem is that these are static models. You do a bunch of stuff, pull back, retrain, then go back out. Reinforcement learning is a form of scrutiny, but it is monolithic. It scrutinizes for one specific objective — fine in a static system trying to solve one problem, but it lacks diversity and extreme polarization.

An ideal scrutinizer is one that:

• has a clear direction but not a clear outcome,
• is predictable in its thinking and logic,
• has strong diversity in what it can scrutinize,
• and can have conflicting and polarizing scrutiny.

One emergent quality of intelligence is its ability to react to a new situation fast, learn from it, then never make that mistake again. Today, with modern LLMs, it can be months or even years before this behavior changes.

LLM memory and ability to adapt is archaic; akin to every time you wanted to learn a new skill in your life, you had to birth another kid, teach it the skill, then kill yourself. A species wouldn’t last more than a generation.

A system can’t just adapt to scrutiny — it has to store the behavior in three core variants of memory:

• Short-term memory (RAM). What you need to know right now to survive (catch, move, talk).
• Medium-term memory. What you need to know on a day-to-day basis to survive (kill, eat, bathe, socialize).
• Long-term memory. What you need to know over a lifetime to survive (reflexes and intuition).

I call the long-term storage “Lava.”It’s the things we know to be true but don’t have direct experience with, and have an intuition on how to interact with — like lava itself. I’ve never seen it, touched it, or smelled it, but I’ve seen a photo and I know not to touch it.

We often reason about long-term memory from a place of feeling, not logic; medium-term from explicit recall; and RAM is often the most rationalized (not always rational).

If I told an LLM “I am Spencer” and then started a new chat, it wouldn’t know who I am. It can have short-term memory, but it fails at determining what to remember, what not to, how readily available that information should be, and what fidelity it needs.

It can’t just be good enough to remember “spiders are bad.” An hourglass on a spider will kill you. The specificity of the knowledge may be more critical than the general understanding itself.

It’s imperative that a system learn how to compress knowledge— not just for speed but for evolution. Perfect memory is a curse, not a gift. Our ability to forget and retry in future generations without the burden of direct memory is what allows progress.

The world is predictable if only you have perfect understanding. If a system understood how every atom interacts with every other, the world would be self-deterministic with a static outcome. But for a system to understand the system it works in, it would have to be more complex than the system it observes — meaning it would simply memory-overflow on any decision.

Building in limited fidelitylets a system make fast decisions that are accurate, learn, adapt, and improve — accurate but not precise.

This one is my most recent addition, and I debated heavily whether it was essential.

I think the answer is: yes, but not in the way it’s usually framed. If you define propagation as multiplication— replication 1:1 with the original entity — I don’t think it’s just unnecessary; I think it’s actively destructive. Multiplication is cancerous. It creates more of itself without improvement. It will kill the system it exists in by flooding it with ever-decreasing productivity, because it can’t adapt or specialize through division.

I would rather define propagation as specialization through division, not multiplication.

We think of propagation as a means that creates complexity — but that doesn’t make sense. No system asks to become more complex. Complexity is a by-product of bad division.

Look at social systems. We have hierarchy and divisions of labor. Inventiveness is often viewed as our ability to specialize a division of labor. Innovation, in my view, is when you’ve removed or changedan existing workflow — not just simplified or improved it.

The goal isn’t to create more to do by maintaining a more complex system. The goal is to simplify systems by dividing generalized knowledge. The more we propagate specialization, the more efficient a system runs, and the more its specializations exceed the capabilities prior to division.

Whether you look at social systems like a company, or cognitive systems like synapses, the goal is to specialize repetitive tasks. We create specialized parts of our brain that do specific work; we hire people for specialized tasks; we create companies for specialized tasks. Much of what we learn over time is that when we spread generalized understanding too thin across specialized tasks, the system collapses from inefficiency.

If a system can only propagate by multiplying, it will end up overly generalized, unable to do any one thing well, and therefore not very useful.

As I write this, I notice the similarities to Buddhism, and the idea that “we live in hell and heaven, you choose.” Ideologies that have existed for thousands of years. I think that similarity only validates an accepted observation of reality — just through the lens of using that wisdom for our own evolution. We’re all just a remix.

The questions that remain:

• How do we build a system that scrutinizes well?
• How does the system determine what to remember and how to remember it?
• How does the system decide when to propagate?

How do we build a system that scrutinizes well?

The easiest way to visualize putting these together is a family unit.

The individual with selection power scrutinizes their options (mating). Optimizing for cultural and environmental factors, they create a division of themselves to “be better than they were”at a specific aspect of life they couldn’t achieve on their own.

The individual then scrutinizes the creation to do that specific thing better — otherwise they risk multiplication, which is cancer that just creates complexity.

What this means in practice: I tell a system “I want to design a website,” and it thinks, “do I know how to do this? what questions do I need to ask to understand what I need to do?”This is most of the recent work on LLM agents — looping question-and-answer to improve output.

How does the system determine what to remember?

The thing that gets created (the division) has to remember as much of this scrutiny as it can, or it risks not adapting to an emergent environment — and either dies, or repeats the system’s suffering.

We see this in ourselves. Many of us know the feeling of “going on autopilot”— the brain has specialized a task through repetition so well that it doesn’t need to be cognizant of its actions to perform it.

This is supported by recent developments in psychotherapy — the metaphorical “parts”expressed in IFS (Internal Family Systems). I think the “parts” metaphor is less of a metaphor and more of a real interaction with the individual specializations we’ve built (consciously or not) to handle specific problems.

We’re very good at condensing information. If you take anything out of this essay, you’ll likely remember three things:

1. Scrutiny
2. Memory
3. Propagation

More acutely, you’ll probably remember propagation— it’s the most novel word and the most interesting thought experiment.

How does the system decide when to propagate?

We typically specialize when a task becomes repetitive, or novel-but-high-risk. Think of driving and taking the SAT. We drive every day, so we should create a specialized agent. We take the SAT once and the cost of failing it is high, so we specialize for it.

The technical difficulty is the fractal nature of the system. It’s effectively a meta-neural network. The way to manage this is to build new agents as tools the main system spins up only when needed.

Concretely: the system first determines if it should look through short, medium, or long-term memory to see if it already contains the knowledge needed. If the knowledge is in short or medium memory, no new agent needed. If not, it’s either in long-term memory or it needs to create a new agent specialized for the task.

Ideally these systems would continuously optimize themselves, propagating to more and more specialized tasks. We first learned physics, then specialized into hundreds of fields of physics, while the rest of nature kept going. Specialized agents can continue to self-propagate.

These three concepts are why OpenAGI exists, and why the codebase is structured the way it is:

The proactive behavior — the agent reaching out before you ask — is the outcomeof these three working together. Once those loops are running, “reaching out first” isn’t a feature you bolt on; it’s what falls out.