1. The Hot Topic: Agent Frameworks and Harnesses

Building harnesses (external frameworks) will eventually become unnecessary, but "building something that disappears in 6 months, deploying it today — that has enormous value."

2. System 1 and System 2

• System 1 (Kahneman): Fast, intuitive, heuristic-based thinking (GPT-4 analogy)
• System 2: Slow, energy-intensive, deliberative thinking (o3 analogy)
• Critical insight: System 2 cannot function without a sufficiently good System 1.

3. The Bitter Lesson and Harness as Crutch

The Bitter Lesson: Human-crafted rules always lose to simple methods + massive compute + massive data.

Noam Brown calls harnesses "crutches" — eventually washed away by scale. But the reality is nuanced: today's 6-month crutches generate the data that fuels tomorrow's model improvements.

"Everything Noam calls a crutch will eventually be washed away by scale. But building and deploying those crutches today generates the data needed to make scale work."

4. Reasoning Model Evolution and Transfer

Reasoning abilities strengthened in verifiable domains (math, coding) transfer to other domains. As models evolve, external frameworks get internalized.

5. Context Engineering in Practice

Current agent services rely heavily on heuristic approaches with clear limitations. Context engineering encompasses prompt engineering, context grounding, function calling, and RAG.

6. Data Is the Program

"Data is the program itself."

In Software 2.0, patterns are extracted from data to create rules. Capability overhang — untapped model abilities — gets unlocked through external frameworks that generate training data for SFT, RLHF, and DPO.

7. Human Learning Mirrors AI Learning

Spaced repetition (Anki, Mochi) strengthens System 1. "Improving System 1 makes System 2 better, and System 2 experiences become System 1." Learning compounds like interest.

8. Creativity and Connection

Creativity comes from connecting distant concepts. AI can connect across vastly more data than humans. But apophenia (finding meaning in meaninglessness) requires distance and verification.

9. AI's Limits Are Human Limits

"No matter how powerful AI becomes, its capability is ultimately limited by the person using it."

Therefore, humans must grow alongside AI to fully leverage it.

10. The Unthinkable Future

Ilya Sutskever: "AI's problem is that it's so powerful it can solve everything — but it can also do everything. That's the problem. And there are currently no answers to these questions."

"The Bitter Lesson is always right." "Building something that disappears in 6 months and deploying it tomorrow has enormous value." "Data is the program itself." "AI's power is ultimately limited by the person using it."