Your AI Agent's Prompt Is Not a Filing Cabinet. Stop Storing Facts In It.

Two very different kinds of content, one text box

A system prompt holds two things that don't behave the same way, but they get typed into the same box, so they get treated the same way.

The first is judgment: how the agent should reason through a problem, what order to check things in, what it should never do. Judgment is durable. The right way to diagnose a networking issue, check the route before the firewall, doesn't change next quarter. That belongs in the prompt.

The second is facts: which specific tool is current, which command syntax is right today, which vendor got flagged in a recent CVE. Facts rot. Some rot in months, some in years, but they all eventually go stale, and nothing about a system prompt tells you when that's happened.

The rule worth adopting: if a fact could plausibly be different in six months, it doesn't belong in the prompt. It belongs in something you can look up and update independently, a knowledge base, without touching the reasoning logic around it.

What it looks like when you get this backwards

Picture an agent that's told, in its prompt, to recommend a specific remote-access tool for a certain IT scenario. That's a fact, not judgment, and facts about vendor tools have a nasty failure mode: the tool can become actively unsafe to recommend after the prompt was written, following a security incident nobody building the agent knew about yet. An agent working purely from prompt instructions has no way to know that. It just keeps confidently recommending what it was told, because nothing about the architecture ever asks "is this still true."

Or picture an agent told which repository or deployment method is the current, supported one for a cloud platform. Vendors deprecate things. Repositories get archived in favor of newer ones. An agent whose only source of truth is a prompt written six months ago will keep pointing users at the deprecated path, confidently, because confidence isn't the same thing as correctness, and nothing in a static prompt distinguishes the two.

Neither of these is a reasoning failure. The agent followed its instructions perfectly. The instructions were just wrong by the time anyone asked.

Why the separation actually fixes this

Once facts live in a knowledge base instead of a prompt, updating one no longer means re-editing and redeploying the agent's core reasoning. It means correcting one entry. The agent's judgment, its method, stays untouched, and the fact gets fixed at the source, the next time anyone retrieves it, it's current.

This also changes how you catch staleness in the first place. A fact buried in a prompt has no owner and no review cycle, it just sits there until someone happens to notice it's wrong, usually because it already caused a bad answer. A fact in a knowledge base can have a review date, a verification step, a place it lives that says "check this again by such-and-such." You can't schedule a review for a sentence buried in the middle of a 2,000-word system prompt. You can absolutely schedule one for a knowledge base entry.

The part people get right and still get wrong

Here's the trap: even after you've correctly split judgment from facts, the facts still have to actually reach the agent intact. A knowledge base entry that gets truncated, summarized, or partially dropped before the model ever sees it is functionally the same failure as never separating things in the first place. Grounding only works if the ground truth arrives whole. That's a separate problem from where you store the fact, but it's the one that makes people think the separation itself failed, when really the separation was correct and the delivery pipe was broken.

Worth checking both, independently: is this fact durable judgment or something that could change, and once it's in the right place, does the full version of it actually reach the model when it matters.

The practical test

Before you write anything into a system prompt, ask one question: would I be comfortable if this sentence were still there, unread, in two years? If the answer is yes, it's judgment, it belongs. If the answer is "well, it depends on whether that's still the current version," it's a fact, and facts need a home you can update without touching the reasoning around them.

If you found this useful, hit the Subscribe button below to get more articles like this delivered straight to your inbox.

Want to see what our AI agents do when facts change out from under them? Start a free trial and see how the knowledge base keeps answers current.

What's a fact you've seen hardcoded somewhere it shouldn't have been? Tell me in the comments.