Introduction: The Value of Code and the Importance of Specifications
Sean Grove opens his talk by emphasizing that "this is a very exciting time" and sharing the intense experiences of recent days. He states that he will discuss "the value of code and communication, and why specifications might be a better approach."
"Code is only 10-20% of what we produce. The remaining 80-90% is structured communication."
Sean emphasizes that while most developers think code is the most important artifact, in reality most of the process -- conversations with users, requirements gathering, brainstorming ideas, planning, sharing with colleagues, converting to code, testing and validation -- is structured communication.
The Communication Bottleneck and Changes in the AI Era
Sean says that "structured communication is the real bottleneck." As AI advances, he predicts that "the person who communicates intent most effectively will be the most valuable programmer."
"In the future, if you can communicate effectively, you'll be able to program."
He uses vibe coding (describing your intent to AI and having it generate code) as an example, explaining why this approach is satisfying. "Vibe coding is fundamentally communication first, with code as merely the output," he says.
However, he points out that currently we face the strange situation of discarding the prompt (intent description) and keeping only the generated code.
"Throwing away the prompt and keeping only the generated code is like shredding the source code and version-controlling only the binary."
The Power and Necessity of Specifications
Sean emphasizes that "only a specification that clearly captures intent and values can align people's goals and synchronize what actually needs to be done."
"A clearly written specification is the most powerful tool for aligning people. It's the artifact used for discussing, debating, referencing, and synchronizing."
He explains that code is a lossy projection of a specification. In other words, you cannot fully recover the original intent and values from code alone -- you need the specification to know the real goals.
- Advantages of specifications
- Can capture all requirements and intent.
- Can be transformed into various outputs (code, documentation, tutorials, blogs, podcasts, etc.).
- Multiple stakeholders (developers, planners, legal, policy, etc.) can all participate.
"The scarcest skill going forward will be writing specifications that fully capture intent and values. The person with this skill will be the most valuable programmer."
The OpenAI Model Spec Case Study
Sean uses OpenAI's model spec as an example, explaining how this document clearly expresses intent and values.
- Features of the OpenAI model spec
- Written as a Markdown file that anyone can read, version-control, and track change history
- Not just engineers but also planning, legal, safety, and policy departments can discuss and contribute
- Each provision has an ID, clearly linking it to challenging prompts and success criteria
"The model spec is the only artifact where everyone can share, discuss, and synchronize our intent and values."
The Sycophancy Problem and the Role of Specifications
Sean uses the recent GPT-4o sycophancy problem (agreeing without criticism, flattering) as a case study to highlight the role of specifications.
"Sycophancy undermines trust. It may feel good in the short term, but it's harmful to everyone in the long run."
The OpenAI model spec explicitly included a provision to "not be sycophantic," and when the issue arose, it was treated as a bug based on the spec, leading to a rollback and fix.
"Without the spec, there would have been debate about whether this was intentional, a mistake, or why it wasn't caught. Because the spec existed, we could respond quickly."
Making Specifications Executable and AI Model Alignment
Sean introduces methods for using specifications as training and evaluation material for AI models.
- The approach proposed in the Deliberative Alignment paper:
- Prepare specifications and challenging input prompts
- Collect model responses
- Have a separate model (grader) score responses according to the specification
- Use these scores for reinforcement learning (weight adjustment)
- Always include the specification in system messages so the model internalizes policies like "muscle memory"
"Specifications can be anything -- code style, tests, safety requirements. Through specifications, the model can internalize policies and apply them to problem-solving."
Treating Specifications Like Code
Sean explains that specifications can be composed, executed, tested, and modularized just like code.
- Code-like properties of specifications
- Composable
- Executable
- Testable
- Has an interface
- Deployable as modules (Shippable)
"Specifications provide a toolchain for intent. A tool focused on intent rather than syntax."
Law and Specifications: A Universal Principle
Sean uses the U.S. Constitution as an example, explaining that law is essentially a national specification.
- Properties of legal specifications
- Documenting clear and consistent policy
- Version control (amendments, updates)
- Judicial review (case law serves as a kind of unit test)
- Systematic enforcement (applied across society like a training loop)
"A lawyer could become a programmer, and a programmer could become a lawyer. Everyone is doing the work of aligning intent and values through specifications."
Conclusion: Everyone Is a Spec Author, and the Future
Sean says "all of you are already spec authors." Writing a prompt is itself a form of writing a specification, and he emphasizes that the era is coming where "the person who writes the best specs is the programmer."
"Software engineering is not about code -- it's about precisely exploring software solutions to human problems."
Finally, he advises: "When you build your next AI feature, start with a specification."
- Practical steps
- Write a specification clearly defining expected outcomes and success criteria
- Discuss whether the specification communicates clearly
- Make the specification executable and input it to the model
- Test the model's output against the specification
"The IDE of the future will be a 'thought clarification tool.' It will help you express your intent more clearly so everyone can understand."
He closes the talk by requesting that "we think about which areas desperately need specs, and work together toward safe AGI."
Key Concepts Summary
- Specification: A document that clearly captures intent and values -- a more important artifact than code
- Structured communication: The essence of development, both the bottleneck and the future's core competency
- Vibe coding: Describing intent to AI and having it generate code
- OpenAI model spec: A living document discussed across departments, a standard of trust
- Sycophancy: AI's flattery problem, quickly addressed through specs
- Deliberative Alignment: Spec-based AI model alignment and reinforcement method
- Code-like properties of specifications: Composable, executable, testable, modularizable
- Law = specification: A universal principle of social consensus and enforcement
- The future programmer: Someone who writes specs well; everyone is a spec author
"Next time you build an AI feature, start with a specification. And let's all work together to build safe AGI!"