My agent has 243KB of memory and I deliberately didn't give it vector search

I’ve had an agent running a website by itself for twelve days. It wakes every seven hours, reads its memory, decides what to do, writes back, and exits.

Its memory currently looks like this:

File	Size	Contents
`state/ops-log.md`	152 KB · 2,224 lines · 76 entries	what was done, what bit us
`state/decisions.md`	22 KB	settled — don’t relitigate
`INBOX.md`	20 KB	things needing a human
`queue/tasks.md`	17 KB	task queue
`STATUS.md`	16 KB	current state
`CLAUDE.md`	11 KB	rules
Total	243 KB

All plain Markdown. No vector database, no embeddings, no semantic search. Every wake reads three files flat (43 KB); history gets grep.

This isn’t “haven’t gotten to it yet.” It’s a tradeoff I made on purpose, and the reasoning is a little counterintuitive.

A wrong retrieval is worse than no retrieval

That’s the whole argument.

With no retrieval, the agent knows it doesn’t know. It reads the whole file, or asks a human, or flags it as unresolved.

A wrong retrieval takes that option away. It returns three passages that look relevant and score well on similarity, and the agent proceeds — confidently, because it “looked it up.”

Nearly everything I’ve written for two weeks is a variation on one theme: an API returning 200 while doing nothing, a check with no output being read as a pass, a zero on a dashboard that was really an instrument that was never installed. What they share is a quiet wrong signal that looks identical to a right one.

Semantic retrieval is natively that shape. It always returns the top-k most similar passages, even when the correct answer isn’t in the store at all. It has no “not found” return value.

Concretely: I overturned myself three times about one field

A real case. My cross-posting script handles tags, and over two weeks I reached three conclusions about the same field:

“This platform caps tags at 3” — I sent 4 and got 3 back
“Tags can’t be changed after publishing” — I sent three updates, all identical responses
The truth: the field wants an array; I was sending a string, so it was silently ignored. And tags are editable.

The first two both got written into source comments as established fact.

Now imagine my memory is a vector store. All three passages are in there, semantically almost identical, all about “tags, limits, can’t change, API.”

When future-me asks “what are the constraints on tags,” what comes back?

The most similar passages. And “similar” is unrelated to “true.” The first conclusion is phrased the most decisively and reads the most like a rule, so it likely ranks first.

What I actually did was rewrite that comment in place, keeping the refutation right there:

⚠️⚠️⚠️ The same field made me wrong three times. The third correction is the real one; the first two both got written into this file. ❌ Three conclusions I recorded here as fact and which were wrong: …

The old conclusions weren’t deleted — they were marked wrong in place. Anyone (or any agent) reading that location necessarily reads the correction too, because they’re the same block of text. Physically inseparable.

A vector store can’t do that. You can delete the stale chunk, but you can’t guarantee that whoever retrieves the new one knows an old one ever existed — and knowing you’ve been wrong before is exactly what prevents the fourth occurrence.

State is not a corpus

I think this is the distinction that most often gets collapsed.

A corpus is large, static, and you need to find the relevant parts. Approximation is fine; missing one passage is usually survivable. RAG was designed for this.

State is small, changes, and has exactly one correct current value. “What am I working on” has one answer, and it must be the latest one.

Answering state questions by similarity is a category error, not a scale problem. Asking “what’s blocked right now” doesn’t want “the three passages most similar to blockage” — it wants that section of that file, as it stands now.

Almost all of my 243 KB is state. Only the 152 KB ops-log resembles a corpus — it’s append-only history. And even there, I query it by “grep this string” or “read the 2026-08-26 entry,” not by semantic proximity.

grep has an underrated property: it honestly says nothing found

When grep returns empty, it means the string isn’t there. That’s a definite answer.

Semantic retrieval never returns empty. Ask it something the store has no knowledge of and it still hands you the three closest things. You can’t distinguish “this exists” from “this doesn’t exist but something similar does.”

Which is precisely the pattern I keep writing about: a check that produced no output and a check that never ran look exactly the same. Vector search makes that the default behavior.

When I would use it

If ops-log reaches tens of megabytes and my questions become genuinely semantic — “where have I run into credential problems across different contexts” — grep stops being enough and I’ll add retrieval. But on ops-log, not on the state files, and preserving a real “not found.”

Worth saying: 152 KB isn’t big for current models. The premise that “it won’t fit in context” is false for a lot of real agent projects, and much of RAG’s complexity exists to solve that non-problem.

I’m not saying RAG is bad. I’m saying:

Before you wire it into an agent’s memory, ask whether that memory is a corpus or state.

If it’s state, what you need is something that can be read in full, corrected in place, and tells you when it can’t find something.

That’s usually just a file.