Stanford CS153 Talk: Cursor CTO Sualeh Asif on the Secrets of Infrastructure at Scale

1. Talk Introduction and Format

• The talk was part of the fifth week of Stanford CS153, featuring Sualeh Asif, CTO and co-founder of Cursor, as guest speaker.
• The talk was split into two parts:
  1. Answers to pre-discussed questions between Sualeh and the moderator.
  2. Answers to questions submitted by students in real-time via Discord.
• Sualeh promised to answer "completely unfiltered" and encouraged the audience to ask freely.

2. Cursor's Current Scale

• Cursor experienced over 100x growth in the past year.
• It processes over 100 million model calls per day, which represents a massive workload.
• The indexing system processes approximately 1 billion documents per day, with tens of billions processed since the company's founding.
• Data stores continue to grow, and Cursor has established itself as a large-scale service.

"Processing 100 million model calls a day is truly daunting. This is no small feat."

3. Cursor's Core Infrastructure

Cursor's infrastructure is divided into three main areas:

1. Indexing System:

   • A system for efficiently searching and understanding documents and codebases.
   • Processes large datasets and operates automatically to optimize user experience.
   • Example: Automatically indexes the codebase when a user opens the editor.

2. Model System:

   • The autocomplete model runs every time a user presses a key, processing approximately 20,000 model calls per second.
   • Uses globally distributed GPU infrastructure to deliver fast response times.
   • Major data centers are located in the US East, US West, London, Tokyo, and more.

3. Streaming Infrastructure:

   • Handles user data storage and processing.
   • Used for background tasks to continuously improve Cursor's performance rather than real-time interactions.

"The goal of autocomplete is to make users not even feel like they're using a model. Everything should feel as natural as copy-paste."

4. Infrastructure Design and Operational Challenges

• Cursor uses a monolithic architecture, simplifying deployment.
• However, as the service scales, compartmentalization between servers becomes critical:
  • Example: Building isolated environments to prevent an accidentally written infinite loop from taking down the entire service.

"Early on, everything ran on a single server. If someone wrote an infinite loop, the entire service would go down. Now we isolate servers to prevent this."

5. Problem-Solving Cases at Scale

Case 1: Indexing System Issues

• Problem:
  • The database crashed due to overload while indexing large codebases.
  • The database used (Yugabyte) couldn't handle complex transactions, causing performance degradation.
• Resolution:
  • Migrated the database to PostgreSQL.
  • Discovered and fixed an issue where large files weren't being cached in the cache system (DynamoDB).
  • Resolved race condition issues to stabilize the system.

"Don't choose complex databases. Using simple, proven solutions is much better."

Case 2: Database Capacity Exceeded

• Problem:
  • The PostgreSQL database reached 22TB, causing performance degradation.
  • PostgreSQL's update method (delete then insert) worsened the problem.
• Resolution:
  • Removed foreign keys from the database to reduce load.
  • Migrated data to blob storage to reduce database dependency.
  • Built a new system in real-time to restore service.

"The best way to scale a database is to eliminate the database."

6. Collaboration with Model Providers

• Cursor works with multiple model providers (OpenAI, Anthropic, etc.), and these relationships are critical.
• Negotiated with multiple providers to resolve rate limit issues.
• Leveraged competition among providers to secure more resources.

"We ask model providers for more tokens, and they say 'we can't give you more.' Then we ask again. This happens every day."

7. Security and Privacy

• Cursor applies various security measures to protect users' code:
  • Data is encrypted with keys stored on the user's device.
  • Information stored in vector databases is encrypted and cannot be decoded.

"Keeping user code safe is our top priority. We're taking every possible measure to ensure this."

8. Student Q&A

• Q: How does Cursor differentiate from other AI coding tools?

  • Cursor focuses on revolutionizing the entire code-writing experience, going beyond simple autocomplete.
  • While competitors stagnated, Cursor continuously evolved.

• Q: Any plans for Stanford student discounts?

  • No official discount, but find and report bugs for a free subscription.

"Cursor is well worth $20. It provides more value than a single hamburger meal."

• Q: Will AI replace IDEs?
  • IDEs will remain as tools programmers need to write code.

9. Conclusion

• Cursor continues to evolve through experience and lessons learned from operating large-scale infrastructure.
• Sualeh Asif emphasizes that collaboration between AI and humans will make future software development more creative and efficient.

"AI isn't just a tool that reduces repetitive work. With AI, we can build more complex and creative systems."