The Person Who Built Cursor's "Memory" Feature: Full Video Summary

1. Video Introduction and Speaker Bio

This video features Yash, the developer of one of Cursor IDE's core features -- the "memory" feature -- personally explaining the development process, challenges, and future direction. Early in the talk, Yash introduces himself as working on the Cursor engineering team, having spent the past few months prototyping the memory feature solo.

"I'm an engineer at Cursor, and I've been prototyping the memory feature by myself for the past few months."

The host emphasizes that Cursor is a $9 billion IDE, expressing amazement that Yash handles such an important feature alone.

"The memory feature plays a hugely important role in Cursor, and it's really impressive that Yash is developing it by himself."

2. How Cursor Classifies "Context" and the Role of Memory

Yash explains that because the concept of context can be too broad and confusing, Cursor approaches it in three categories:

• 1. Directional context:

  • Information that helps the agent start with a broad search scope and gradually narrow down to relevant files.
  • For example, when a user gives a large task, this context quickly narrows the search scope.

• 2. Operational context:

  • Information about work procedures and rules, like a "runbook" -- deployment methods, file modification rules, etc.
  • In Cursor, hand-written "Cursor rules" fall into this category, which the model retrieves and references when needed.

• 3. Behavioral/User context:

  • Information about user preferences and communication styles that guides the agent to behave in specific ways.
  • For example, user rules like "Only respond to me in Spanish."

"The word 'context' is too broad and confusing, so we divide it into three categories: directional, operational, and behavioral context."

Yash notes that until now, the focus has been mainly on code-based search (i.e., directional context), but recently he's been developing the memory feature to complement all three.

"The goal of memory is to complement all three types of context. Ultimately, to reduce dependence on rules and search."

3. Prototyping the Memory Feature: Two Approaches

Yash explains two main approaches he tried while developing the memory feature.

3-1. Sidecar Approach

• How it works:
  • Separately from the main agent, a small model listens to the conversation and, at appropriate moments, independently decides what information to store.
  • This small model operates completely independently from the main process.

"In the sidecar approach, a small model listens to the conversation and independently decides what information to store."

3-2. Tool Call Approach

• How it works:
  • The main model is given a "memory update" tool, and it directly creates, modifies, or deletes memories when it deems necessary.
  • Recent models (e.g., Sonnet 4, Opus 4) follow instructions well, so this approach actually works well in practice.

"In the tool call approach, the model directly creates a memory when it judges 'this is worth remembering.'"

3-3. Differences and Experiments

• Sidecar:
  • Separated from the main process, allowing more independent judgment.
• Tool Call:
  • Everything is handled within the main process, making implementation simpler.

Yash experimented with both, and especially as recent model performance improved, the tool call approach became quite viable.

"Recent models are really smart -- just give them the tool and they handle things well on their own."

4. Defining Memory and Real-World Application Challenges

Yash highlights the difference between "memory" as defined by Anthropic and other external models versus what Cursor needs.

• Anthropic's memory:
  • Primarily like a "work log," recording everything an agent has attempted.
• Cursor's memory:
  • What matters is remembering generalized information usable across different contexts, not just information limited to a specific conversation or task.

"Anthropic's memory is closer to a work log, but we want to remember more generalized information."

In practice, if the model remembers incorrect memories, even when the user tries to correct it, the model can stubbornly insist its memory is right.

"If the model remembers something wrong, the user says 'That's not right' but the model might insist, 'No, my memory is correct.'"

5. Prompt Design, Evaluation, and UX Challenges

5-1. Evolution of Prompt Design

• Previously, models were less capable, requiring many examples and complex prompt design.
• Recently, models have improved enough that short, clear explanations are sufficient to elicit desired behavior.

"Now you don't need long prompts -- just short, precise descriptions and the model follows well."

5-2. The Difficulty of Evaluating Memory

• The memory feature has the property of feeling inconvenient when absent but barely noticed when present.
• Deliberately creating examples where memory helps makes evaluation inherently unfair.
• So evaluation focused on how well the system filters out unnecessary (task-specific) memories.

"Memory is something you miss when it's gone but barely notice when it's there. That made evaluation really difficult."

5-3. User Experience (UX) and Memory Visibility

• Users tend to worry excessively about what the model should remember.
• In reality, even if half the memories are useless or irrelevant, the model can filter the noise.
• So the memory creation process was designed to be hidden as much as possible, only letting users manually edit when needed.

"Users think the memory bank needs to be perfect, but actually the model handles things fine even when it's not."

6. Future Direction and Challenges

Yash says he's researching how to better understand and retain as memory information beyond code.

• Embedding the entire codebase alone isn't sufficient; information that doesn't exist in code, like sidebar conversations, should also be preservable as memory.
• Various experiments are underway on including new types of memory in context and improving the search pipeline.

"I'm looking for ways to preserve as memory even information that doesn't live in code. We're still in the experimental phase, but I believe we'll get there soon."

The ultimate goal is for the entire team to share knowledge. However, if incorrect memories spread across the entire team, quality could severely degrade, so the approach is cautious.

"The goal is to prevent teammates from repeating mistakes I had with the agent. But if wrong memories spread, it's a big problem, so we're being really careful."

7. Closing and Q&A

Yash wraps up the talk by stating he'll continue prototyping and experimenting. He mentions a separate Q&A session will be arranged and thanks the audience.

"Thank you. I'll keep experimenting and improving."

Key Concepts Summary

• Cursor IDE
• Memory feature
• Context (directional, operational, behavioral/user)
• Sidecar vs Tool Call approaches
• Prompt design evolution
• Difficulty of memory evaluation
• User experience (UX)
• Team knowledge sharing
• Experimentation and prototyping

This video is a valuable resource for hearing the firsthand developer experience of how Cursor's memory feature was designed and evolved, and what direction it's heading. Yash's honest reflections on challenges, experimentation, and careful approach toward users and the team are truly impressive.