Why Korean conversations produced English memories
The extraction prompt carried 14 English few-shot examples. No placement of an instruction changed anything; cutting to three examples with one Korean pair did.
I was building long-term memory for an agent. Facts a user drops during a conversation get extracted, stored, and recalled in later sessions.
One day I opened the store and read this:
User asked why pgvector 0.8 is necessary for service deployment.
The conversation had been in Korean from beginning to end.
Where this happens
Structure first. The memory service doesn't own the transcript. The transcript lives in exactly one place, and this service holds only summaries, cursors, and memories.
Hover a node to reveal its connections.
Not replicating the transcript was the best early decision we made. Retention, deletion requests, and the boundary of personal data all stay the problem of one store. The cost is that the worker has to re-read the original every time.
Why per-turn extraction doesn't work
The first design was simple: a turn arrives, extract that turn.
Counting tokens killed it.
Hover a segment or a row for detail.
We were resending 98% of the input every time to process 2%, so I decided to batch turns and share the fixed cost across them.
Batches of five cut it by 4.6×. Push the slider further and it does keep dropping, but almost all of the saving from sharing the fixed cost comes in the first few steps, and after that you're only adding delay before a memory gets stored. In the real service memory is needed at the session level anyway, and five turns or so sat closest to that.
01A signal every turn, but never the content. The queue only says that something occurred.
No placement of the instruction worked
If fixed cost is 80%, batching isn't the only option; you can also shrink the fixed cost itself. That's why I opened the prompt, and that's how I ran into the language problem.
At first I just told it to: write in the same language as the input. I tried four conditions.
| Condition | Instruction placement | Result |
|---|---|---|
| A | 8 lines at the end of the user prompt | English |
| B | End of the system prompt, forcefully | English |
| C | Start of the system prompt, forcefully | English |
| E | Minimal prompt with no examples + one line | Korean |
A, B and C came out in English wherever the instruction went, and the only one that came out in Korean was E, the one with every example removed.
The cause was fourteen examples
The prompt was 33,661 characters and contained fourteen English few-shot examples. The word language appeared nowhere in it, and there wasn't a single Korean character. Nobody knew that until I read the prompt from the top to shrink it.
My guess is that fourteen demonstrations of "take Korean, write English" simply weighed more than one line of instruction.
Same input, different prompt
I cut the examples to three and made one of them a Korean input/output pair.
Measurements
Synthetic data first, then six real conversation threads at the end. A/B order was also run reversed to cancel position bias.
A 9.5× reduction. The output schema was left untouched; only guidelines and examples were compressed.
6,698 tokens saved per call — 52.3%. The ratio looks modest because these threads had unusually long bodies, which dilutes the prompt's share.
The English-extraction rate I'd believed until then was 13.4%. That was measured on synthetic data; in real conversations half came out in English, so it was nearly four times worse.
What came along with it
Rewriting the examples was also a chance to encode two contaminations that had been sitting there.
The first is recall re-ingestion. A user asks "what did I say I liked?", the agent lists stored memories back, and that listing gets extracted as new facts. Copies of the same fact multiply every time the topic comes up.
The second is absence statements. "There is no record of that" was being stored as a memory that there is no record.
| Contamination | Before | After |
|---|---|---|
| Recall re-ingestion | 2/7 | 0/7 |
| Absence statements | 5/7 | 0/7 |
| Attribution accuracy | no regression | no regression |
How fast copies actually accumulate only shows up across sessions. Fifteen consecutive sessions, counting cumulative duplicates:
Without suppression, copies grow linearly. The final version stops growing at 3 from the seventh session on, which means restating the same fact no longer produced a new copy. Through all of it, 22 of 24 legitimate facts were kept.
Where a prompt doesn't help
The agent suggests "navy is a safe choice", the user says nothing at all, and "the user prefers navy" gets stored. Instructions didn't stop it. Extra examples didn't. A dedicated non-response example didn't. Three attempts, three failures.
Fabricating decisions that were never made is fully blocked by rules now. But deciding whose opinion something that was said belongs to didn't work in the prompt, and I think it has to be handled in a filtering layer after ingestion.
End