When Memory Becomes Influence

Controlling what an AI agent can do is only half the problem. We also have to pay attention to what’s allowed to influence how it reasons.

We’re getting better at asking the right questions about AI agents.

What systems can they access? What tools can they invoke? What credentials do they hold? What still requires human approval?

Those are important boundaries.

But I keep coming back to another one:

What prior information is allowed to influence the inference being made now?

Persistent memory is part of what makes AI agents useful. An agent that can retain observations, context and previous work doesn’t have to start every transaction from zero.

That’s a good thing.

But persistence changes the trust problem.

A prompt disappears. A memory can survive.

And I don’t think the most interesting risk is simply that an agent might remember something false.

Memory can shape the inference

Consider an agent looking at a subtle rise in process temperature.

The current evidence is ambiguous. Flow is stable. Pressure is still within range. The temperature increase could point to developing fouling, a measurement issue or simply a normal consequence of the current operating condition.

Now add one retained memory: during an earlier investigation, a similar temperature rise was traced to sensor drift.

That memory may be completely accurate. It may have come from a trusted source. There may be nothing malicious about it at all.

But once it enters the current inference, it can start shaping how the deduction is built.

Sensor drift becomes a more “top of mind” available explanation. Evidence supporting that possibility may seem more important. Conflicting evidence may get a little less weight. The agent can begin constructing a coherent case around an explanation that the current evidence, on its own, might not have favored.

The memory doesn’t have to tell the agent what to conclude.

It only has to influence how the agent gets there.

That’s the part I think deserves more attention.

What if the earlier event involved a different instrument, maintenance state, process configuration or operating mode?

The memory can still be true.

But should it be allowed to influence this inference?

Trusted isn’t necessarily admissible

Most of the current attention around agent memory is understandably focused on poisoning.

OWASP recently described persistent memory as an agentic attack surface, making a straightforward point: once an agent can retain context and reuse it later, that context deserves the same kind of scrutiny we give other sensitive control surfaces. (OWASP Gen AI Security Project)

The research is getting more interesting as well.

A June study, From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents, examined the different ways information can enter agent memory and showed why persistent memory creates vulnerabilities that aren’t adequately addressed by thinking about a single prompt or transaction. (arXiv)

Then a July study, MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents, surfaced something particularly relevant to this discussion. The researchers found that apparently benign memory records can become harmful through combination with other retrieved information or activation in a particular context. Their findings point toward context-sensitive defenses rather than relying only on filtering when the memory is originally stored. (arXiv)

That caught my attention because it points beyond malicious memory.

An accurate memory can still be the wrong evidence for the inference being made now.

So maybe the question isn’t simply:

Can I trust this memory?

Maybe it’s:

Should this memory be allowed to influence this inference, here and now?

The evidence can help us answer that

This is where I think the problem becomes less about memory itself and more about context.

What asset are we looking at? What’s its current operating state? Which instrument produced the observation? Under what conditions was the retained information originally established? Has something changed since then? Does the earlier evidence actually apply to this situation?

Looked at together, the evidence can tell us more than whether something is known or trusted.

It can help us determine whether it should matter now.

That’s an important part of how we’re thinking about bounded AI at NexGenomics.

Using a contextual inference and evidence lens, we can examine the evidence around the current transaction and use it to help determine what belongs inside the inference. The architecture can then enforce that boundary outside the model rather than asking the model itself to decide what it should or shouldn’t consider.

And the evidence matters afterward too.

We should be able to understand which observations were considered, which historical context was admitted, what was excluded and what actually influenced the resulting deduction.

That gives us a better question than:

What did the AI say?

We can ask:

What was the AI allowed to consider when it got there?

A bounded agent needs a bounded inference context

Bounding tools matters. Bounding credentials matters. Bounding actions matters.

But an agent can stay inside all of those controls while prior context quietly changes the reasoning that comes before the action.

So I think the boundary has to reach a little farther upstream.

It has to include influence.

The model shouldn’t be the final authority on which influences are permitted into its own reasoning process.

A bounded agent needs a bounded inference context.

Because memory doesn’t have to be false to create a problem.

Sometimes it only has to be true, relevant-looking and wrong for the inference being made now.

Govern every influence. Prove every outcome.