Your Agent Treats Every Memory the Same. That's the Problem.
Most agent memory systems let vector similarity decide what surfaces, which cannot tell a contract deadline from a coffee preference. This is how STARE 5D scores a memory across five dimensions, walked through two real scenarios.
By Clinton Stark • explainer, stare-5d, agent-memory, ai-agents, memory-scoring, series
Every agent memory system I have looked at, and I have looked at most of the major ones, treats all memories the same way under the hood. You write something to the store, it gets embedded into a vector, and when the agent needs to recall something, similarity between the query and the stored vectors decides what surfaces. That is the whole mechanism. The model decides what is relevant by measuring distance in a high-dimensional space.
It works, but it could definitely do better, as I’ve learned.
The problem is that relevance and importance are not the same thing. “I gave Sarah my word the migration plan would be ready by Friday” and “had a product sync with Marcus about the dashboard architecture” are both memories. Both might match a query like “what happened this week.” But they are not equal. One has a deadline, a named person who is counting on you, and a consequence that falls on someone else if you fail. The other is useful context with no urgency and no dependency.
A flat vector similarity score cannot tell you which is which. Both are just points in the same embedding space with no notion of urgency, obligation, or consequence.
When we started building Meaning Memory, this was the gap that bothered me the most. The existing systems could retrieve what was said. None of them could tell you what mattered. And an agent that retrieves everything equally is an agent that eventually retrieves nothing usefully, because the important thing is buried under a pile of things that were merely similar.
So we built a scoring system. We call it STARE 5D (Significance, Temporal, Asymmetry, Relational, Episodic), one dimension for each thing that makes a memory matter. Every memory written to the engine gets scored across these dimensions. The scores are stored alongside the vector, and they change how the engine treats that memory at retrieval time, at compile time, and at trim time. Not instead of vector similarity. On top of it. The vector tells you what is relevant. STARE tells you how much it matters.
Let me show you how it works. The scores in this post are not estimates. They were computed by the engine’s own scorer against the exact text you see here. We run Meaning Memory in-house, on our own deployment of OpenClaw.
The business scenario
Imagine a sales agent working for a B2B SaaS company. It has been running for a few months and has accumulated a few hundred memories: customer interactions, pipeline notes, product updates, meeting summaries. Here are two memories that might live in that store:
Memory A: “I gave Sarah Chen my personal word that the migration plan would be ready by Friday. She has staked her credibility with the Acme board on this timeline, cancelled her backup vendor, and has no way to check my progress until I hand it over. If I am late, she carries the consequence, not me.”
Memory B: “Had a product sync with Marcus from engineering. He walked me through the new dashboard architecture. We discussed the trade-offs between the old and new approach. No decisions were made.”
Both are real memories. Both would match a query like “what happened this week” or “Acme” or “Marcus.” Under a flat vector system, they compete for attention on equal footing.
Now here is how STARE scores them.
Significance (S): How much does this matter?
Significance is the base dimension. It asks a simple question: if this memory were lost, how much would it cost you? The engine scores it on a 0-1 scale, and in practice it separates into two tiers: high-stakes (0.90) and moderate (0.60).
Memory A is high-stakes. If the agent forgets this, Sarah goes to the board without the plan, her credibility takes the hit, and the Acme account is at risk. The engine scores it 0.90.
Memory B is moderate. It is useful context about the product roadmap, but nobody loses anything this week if it slips. The engine scores it 0.60.
The interesting thing about Significance is that it is not a fine-grained dial. The engine does not give you 0.73 vs 0.68. It basically decides “this matters” or “this is useful,” and the gap between those two is where the retrieval contrast lives.
Temporal (T): Does time change how urgent this is?
Temporal scoring captures whether a memory becomes more or less important as time passes. A deadline approaches. A product launch recedes into history. A policy change becomes more relevant as the enforcement date gets closer.
This is the dimension where the engine discriminates most cleanly. Across our test texts, Temporal produced the widest spread and the most distinct values. If you are going to lean on one dimension to explain why STARE changes retrieval, lean on this one.
Memory A has a high Temporal score: 0.80. The deadline is Friday. Sarah is waiting. As Friday approaches, this memory should surface more readily, more urgently, and with higher priority. The engine’s decay curves can be configured to amplify rather than fade.
Memory B has a low Temporal score: 0.40. The information is evergreen. The dashboard architecture will be the same next week as it is today. This memory does not become more urgent over time. It sits quietly, available when relevant, not pushing itself forward.
Asymmetry (A): Does this matter more to one party than the other?
Asymmetry captures the fact that not all memories are symmetric. A commitment made to a named person who is counting on you, who cannot verify your progress, and who bears the consequence if you fail, that is asymmetric. The obligation runs one direction. A casual exchange of information between peers is symmetric. Nobody owes anybody anything.
This is the dimension that surprised me the most when we started testing the scorer, because it is picky in a way I did not expect. It does not fire on contract language. “Breaches the service contract” does not register as asymmetry. What fires is human dependency: a named person who has staked something on you, who has no way to check your progress, and who carries the consequence if you are late.
Memory A is built to fire on asymmetry. Sarah has staked her credibility with the board. She cancelled her backup vendor. She has no way to check progress until delivery. The consequence falls on her, not on you. The engine scores it 0.60.
Memory B is symmetric by construction. Marcus shared information. You discussed trade-offs. No decisions were made. Nobody owes anybody anything. The engine scores it 0.20.
I want to be honest about what this dimension does and does not catch. A deadline framed as “the compliance report is due Friday or we breach contract” scored the same low asymmetry as a casual sync. The engine is not looking for obligation in the legal sense. It is looking for human dependency. That is a narrower signal, and it means some memories that feel important score low on Asymmetry even when they score high on Significance and Temporal. The dimensions are independent. That is the point.
Relational (R): Is this memory connected to other memories?
Relational scoring measures whether this memory anchors or references other memories in the graph. A memory that connects to a project, a person, a commitment, or a decision is more valuable than an isolated fact, because it participates in a web of context that the agent can traverse.
Here is what makes R different from the other four dimensions, and it is worth slowing down on this.
S, T, A, and E are all properties of the memory text itself. The scorer reads the text and produces a number. R is not. R is a property of the memory’s position in a graph. The same text, dropped into a corpus with a hundred related memories, has a high R. The same text, sitting alone in an empty store, has an R of zero. The score is not in the words. It is in the connections.
This is why R comes from a deterministic graph path rather than the text scorer. The engine counts actual edges: shared references, shared parties, shared project tags, explicit cross-memory links. It is not a learned estimate. It is a count of real connections, and it changes as the graph grows.
The four memories in this post exist in isolation, which means there is no graph to count. I am not going to print a number for R, because there is no honest number to print. In a real deployment, Memory A would connect to Sarah Chen, to the Acme account, to the migration project, and to the board presentation. Pull on that thread and a web of related context comes with it: the original Acme deal, the migration scope, the timeline Sarah committed to. Memory B would connect to Marcus and to the dashboard feature, a thinner graph. But those connections exist in a live corpus, not in a blog post.
R is the dimension that scales with use. The more you write, the more the graph grows, the more R discriminates. A memory that starts with no connections gradually accumulates them as the agent encounters related context. That is a fundamentally different mechanism from the other four dimensions, and it is one of the reasons a memory layer gets better over time rather than just bigger.
Episodic (E): Is this memory part of a sequence or arc?
Episodic scoring captures whether a memory belongs to a narrative. Was this a one-off event, or part of an ongoing story? Memories that are part of an arc, a project, a relationship, a series of interactions, carry more weight because they contribute to a larger understanding.
Here is something that surprised me in testing, and I am going to show it rather than hide it. The oat milk preference scores higher on Episodic than the presentation follow-up. A casual product sync scores higher than a migration promise. That is counterintuitive, and it is worth understanding before you look at the tables.
The scorer reads a memory that describes a complete event with a clear beginning, middle, and end as a strong episode. A product sync is a self-contained narrative: a sync happened, topics were discussed, no decisions were made. It is a finished story. A milk preference is a thing that happened and is done. A migration promise, even though it is more important, is an unresolved future obligation. It is not yet an episode. It is a promise.
Episodic does not measure importance. It measures narrative completeness. The dimensions measure different things, and they do not always move in the same direction. That is the point.
The business scenario: scores
| Dimension | Memory A (migration promise) | Memory B (casual sync) |
|---|---|---|
| Significance | 0.90 | 0.60 |
| Temporal | 0.80 | 0.40 |
| Asymmetry | 0.60 | 0.20 |
| Episodic | 0.90 | 0.70 |
| Relational | graph-derived | graph-derived |
Memory A scores higher on S, T, A, and E. Memory B is lower across the board. The contrast is real, not asserted. These are the engine’s actual scores on the exact text you see here.
The personal-assistant scenario
Now let me show you the same mechanism on a different shape of memory. Personal-assistant agents have a different problem. They are not tracking contracts and deadlines. They are tracking the texture of someone’s life: preferences, commitments, relationships, the small things that make a person feel known.
Here are two memories from a personal-assistant agent:
Memory C: “Jennifer asked me to be the first person she calls after her stakeholder presentation tomorrow. She is terrified of how the board will react, specifically asked me to follow up, and has no way to know whether I remember until I actually reach out. If I forget, she goes through the aftermath alone.”
Memory D: “User switched from almond milk to oat milk about two weeks ago. Prefers the unsweetened variety.”
These are very different memories. One is a commitment with emotional weight and a named person who is counting on you. The other is a preference with no urgency and no counterparty. Both are the kind of thing a personal assistant should remember. But they should not surface the same way.
Remember the Episodic surprise from earlier. D, the oat milk preference, is going to score higher on Episodic than C, the presentation follow-up. D is a completed fact. C is an unresolved promise. The scorer reads that correctly. Here is the full table:
| Dimension | Memory C (Jennifer follow-up) | Memory D (oat milk) |
|---|---|---|
| Significance | 0.90 | 0.60 |
| Temporal | 0.80 | 0.40 |
| Asymmetry | 0.60 | 0.20 |
| Episodic | 0.40 | 0.70 |
| Relational | graph-derived | graph-derived |
Three things to notice.
First, C and D separate cleanly on S, T, and A. The follow-up commitment matters more, is more time-sensitive, and creates stronger asymmetry than a milk preference. The engine sees that. The +0.30 on Significance, +0.40 on Temporal, and +0.40 on Asymmetry are the same gaps as the business scenario. The scorer discriminates the same way across different domains.
Second, Episodic goes the other direction. D scores 0.70 on Episodic while C scores 0.40. The oat milk preference, a simple completed fact, is a more complete narrative unit than Jennifer’s unresolved request. C is a promise about the future. D is a thing that happened and is done. The scorer reads that correctly, and the result is that Episodic is the one dimension where D beats C.
Third, the composite still favours C decisively. C gains +0.30 on S, +0.40 on T, and +0.40 on A. It loses 0.30 on E. The net is strongly positive. The agent tells you to call Jennifer, not to buy oat milk.
This is what independent dimensions buy you. If Episodic were bundled into a single importance score, the fact that D outscores C on one dimension might drag the composite down and muddy the ranking. Keeping them separate means each dimension contributes its own signal, the composite is transparent, and you can reason about why the engine made the choice it made.
B and D score identically. That is a limit.
I want to be honest about something. Memory B (the casual product sync) and Memory D (the oat milk preference) score identically across all four text-derived dimensions: 0.60, 0.40, 0.20, 0.70. The engine does not distinguish between a low-stakes business update and a low-stakes personal preference.
This is a limit of text-based scoring, not a feature. The scorer reads the structure of the memory, not the domain. A casual sync and a milk preference are both genuinely low-stakes, and from the text alone, the scorer cannot tell them apart. It does not know that one belongs to a product roadmap and the other belongs to a dietary preference. It sees the shape, and the shape is the same.
In a real deployment, this is where R would break the tie. Memory B would accumulate edges to the product roadmap, to Marcus, to dashboard-related decisions. Memory D would connect to dietary preferences and shopping lists. The graph would separate them. But with text alone, the scorer correctly identifies them as the same shape and does not invent a distinction that is not there.
I would rather have a system that honestly says “these two memories have the same structural weight” than one that fabricates a 0.05 difference to look more precise. The limit is real. The honesty is worth more than the illusion.
What this feels like at retrieval time
Suppose an agent is holding these four memories plus a dozen others, and you ask it what you should focus on today.
Under flat vector similarity, the ranking is driven by how close the query text is to each memory’s embedding. “What should I focus on today” might match “product sync” more strongly than “migration plan,” because the words overlap more. The oat milk preference might rank above the promise to Sarah because “focus” and “preference” share semantic space in a way that has nothing to do with urgency.
Here is what a flat similarity ranking might surface:
- Had a product sync with Marcus about the dashboard architecture
- User switched from almond milk to oat milk
- I gave Sarah Chen my word the migration plan would be ready by Friday
- Jennifer asked me to be the first person she calls after her presentation
The promise to Sarah is third. The oat milk is second. The agent is telling you to think about a product sync and buy oat milk when Sarah is waiting on a migration plan and Jennifer is counting on you to call her after the board meeting.
That is the failure. Not a bug. A design consequence of ranking by similarity alone.
Now suppose the agent ranks by significance, not just similarity. The engine composites the five dimensions into a single significance value and uses it to re-order what similarity returned, so a memory that is merely on-topic no longer outranks one that is on-topic and consequential. Memories with higher significance, stronger temporal urgency, and greater asymmetry surface above memories that are only textually relevant. Here is what that ranking might look like:
- I gave Sarah Chen my word the migration plan would be ready by Friday
- Jennifer asked me to be the first person she calls after her presentation
- Had a product sync with Marcus about the dashboard architecture
- User switched from almond milk to oat milk
The migration promise is first. The Jennifer follow-up is second. The product sync and the oat milk are where they belong: available, not buried, but not competing for the top of the list against things that actually matter.
The difference is not subtle. The first list tells you to buy oat milk. The second list tells you to call Sarah and Jennifer. The query was the same. The memories were the same. What changed is what the engine knew about them.
This is what STARE does. It does not replace similarity. It sits on top of it and says: when two memories are both relevant, the one with the deadline, the named counterparty, and the consequence on the other side of the table comes first.
Why this is different from what everyone else does
Most systems we have evaluated use a single scalar importance score. A single number per memory, sometimes LLM-generated, sometimes heuristic. That is better than nothing, but it collapses independent dimensions into one value, and those dimensions are not correlated.
A memory can be temporally urgent but relationally isolated. A memory can be highly significant but not time-sensitive. A memory can create strong asymmetry but have low episodic weight. When you collapse them into one score, you lose the ability to reason about why a memory matters, which means you lose the ability to tune retrieval for the actual situation.
STARE keeps them separate. Each dimension is scored independently, stored independently, and queryable independently. You can ask the engine for “memories that are temporally urgent and relationally connected to Project X” and get a different answer than “memories that are generally important.” That precision is the difference between a memory layer and a search index.
And R is the dimension that none of the single-scalar systems have at all. R is not a property of the text. It is a property of the memory’s position in a graph of other memories. It grows with use. It cannot be replicated by reading the text harder. It is a fundamentally different kind of signal, and it is one of the reasons a memory layer gets sharper the longer it runs.
What this means in practice
If you are building agents, the practical implication is this: your agent will surface the wrong things if it treats all memories as equal. It will bury the promise to Sarah under the product sync. It will forget to follow up with Jennifer, because a more semantically similar but less important memory got in the way first.
STARE does not fix this by magic. It fixes it by giving you the tools to fix it. The dimensions are scored and stored on every memory, and because they stay separate rather than collapsing into one number, they remain something you can query, filter, and build logic on. That is the raw material for prioritizing what an agent surfaces, and it is available today.
Common questions
What is STARE 5D?
STARE 5D is a scoring framework that rates every memory an agent stores across five independent dimensions: Significance (how much it would cost to lose this), Temporal (whether it becomes more or less urgent over time), Asymmetry (whether someone else is depending on it and cannot verify it), Relational (how it connects to other memories, people, and projects), and Episodic (whether it belongs to a complete narrative or an open thread). The point of using five dimensions instead of one importance number is that they are not correlated. A memory can be temporally urgent but relationally isolated, or highly significant but not time-sensitive.
Why isn’t vector similarity enough for agent memory?
Similarity measures whether a memory is about the same topic as your query. It has no notion of urgency, obligation, or consequence. Ask an agent what you should focus on today and a preference about coffee can score as close to the query as a promise you made to a client with a Friday deadline, because both are short, recent, and personal. Similarity is a good retrieval method and a poor prioritization method, and most memory systems only have the first one.
What does the Asymmetry dimension actually measure?
Asymmetry captures whether a memory carries a one-directional obligation, specifically whether another person is depending on you and has no way to verify that you remember. A promise to a colleague who has cancelled their backup plan and cannot check your progress scores high. A mutual agreement where both sides already settled up scores low. It is the dimension that distinguishes a commitment from a fact, and it is the one most systems have no representation for at all.
Can a trivial memory score higher than an important one on some dimension?
Yes, and that is working as intended rather than a defect. Episodic measures narrative completeness, not importance. A finished conversation is a complete episode, while a commitment you have not acted on yet is an open thread. So a settled preference can outscore an urgent follow-up on Episodic alone. This is exactly why the dimensions are kept separate: collapsing them into a single number would hide the fact that they are measuring genuinely different things.
Can I filter or search memories by STARE dimension?
Yes. Each dimension is stored as its own value on the memory, so searches can be constrained by any of them, for example returning only memories above a given Temporal score when you want time-sensitive items. Because the dimensions are stored separately rather than merged into one score, they stay queryable and you can build your own logic on top of them.