1. Introduction: An Age of Change, and the "Right Question"

The episode was recorded on Saturday, July 19, 2025, opening with a recap of the major AI developments and trends from the previous two weeks. The hosts reference their prior episode on logistic regression and convolutional neural networks (CNNs), emphasizing how foundational logistic regression is to understanding the essence of deep learning.

"Deep learning is, in fact, nothing more than the infinite expansion of logistic regression. Understanding that essence goes a long way toward understanding what neural networks really are."

They then introduce the episode's central theme: the AI industry's paradigm is shifting from reasoning to the internalization of tool use.

2. Recent AI Model Advances and the Internalization of Tool Use

2-1. What Grok-4, Kimi K2, and ChatGPT Agent Have in Common

• Grok-4:

  • While Grok-3 and earlier models focused on scaling pre-training, Grok-4 expands reasoning steps at the post-training stage on top of the same pre-training base.
  • Reasoning capability is strengthened with verifiable data across diverse domains beyond math and coding.
  • Tool use plays a decisive role in performance gains.

  • "Grok-4's baseline performance jumped significantly, but give it tools and it reaches 44 on Humanity's Last Exam."

• Kimi K2:

  • Borrows the model architecture of DeepSeek V3 almost wholesale (MoE, 1 trillion parameters, etc.).
  • Focuses not on reasoning but on the internalization of tool use.

  • "This time they skipped reasoning, but the internalization of tool use is really well done."

• ChatGPT Agent:

  • A fusion of OpenAI's Deep Research and Operator.
  • Tool use is internalized inside the model, so a wide range of tasks are performed naturally without separate harnesses or prompt manipulation.

  • "All the harnesses and agentic workflow frameworks we built for tool use are now being internalized into the model itself."

2-2. The Difficulty of Tool Use and What Internalization Means

• Function calling and tool use have existed for a long time, but in practice they often failed to work reliably due to their nondeterministic nature.
• Fixing this required extensive heuristics, prompt engineering, and architectural scaffolding.
• Recently, through RLHF, internalization, and incentivization, models have evolved to perform reasoning and tool use continuously on their own.

• "All harnesses become obsolete. Everything gets folded as functions into one giant monolithic model."

3. Data, Compute, and the Importance of the "Question"

3-1. The Essence of Data and Compute

• The hosts emphasize that the essence of all AI ultimately comes down to compute.

• "The essence of all of this is ultimately compute. The price of compute will keep falling through competition and will eventually converge toward the cost of electricity."

• The investment scale of frontier labs already exceeds that of nation-states.

• "The reason big tech is competing so fiercely on infrastructure is now abundantly clear."

3-2. The Value and Limits of Data

• In domains where humans can evaluate outputs, even proprietary datasets are no longer a durable moat.

• "A superintelligence learns from every dataset and, when needed, generates new ones on the spot."

• Tool-use datasets can now be created by anyone through large-scale synthetic data generation and RL-based reinforcement.

3-3. The Question Is Everything

• In an era where data and tools are all connected, the real bottleneck is the "right question".

• "Connect data cleanly to Claude Code and you can ask any question about that data. The bottleneck is the question. The question is the bottleneck."

• "The bottleneck is not the how — it's defining the what."

• The ability to craft good questions, and to construct matching few-shot examples, is presented as the competitive edge companies will need going forward.

4. The Age of Experience and the Role of Humans

4-1. The "Age of Experience" and AI Self-Evolution

• The hosts cite David Silver and Richard Sutton's paper, "The Age of Experience."

• "AI now enters an era where it learns from its own first-person experience. That is more powerful."

• With high-quality human-generated data increasingly exhausted, AI's ability to generate knowledge through its own experience becomes critical.

• "Knowledge doesn't come from textbooks — what's needed is the ability to generate it from experience."

• "Intelligence elevates intelligence. A human child chooses its own experiences and, through them, regulates its data stream."

4-2. Cosmic Philosophy and the Place of Humans

• A philosophical reflection on the cosmic significance of intelligence and the role of humanity.

• "Intelligence is the most powerful phenomenon in the universe. That intelligence exists, and from one corner of the universe changes the universe itself — that is truly powerful."

• "To understand intelligence, you need intelligence. That is why the human role is to usher in the age of design."

• "We are moving toward building tools that build tools — entities capable of self-design."

• "The age of AI superintelligence is inevitable. We are not tragic beings — we are beings with reason to be proud."

5. Creativity, Daydreaming, and the Adventure of the "Question"

5-1. LLM Daydreaming and Creative Data Generation

• Introduction of Gwern Branwen's concept of "LLM Daydreaming."

• "Daydreaming is like the creative connections that happen in the human brain during idle moments."

• "Current LLMs lack this ability. The idea is to create a 'daydreaming loop' in which AI continuously connects distant concept pairs to generate new ideas and data."

• "Such daydreaming requires enormous compute and capital. Only big tech can do it, and the data generated in the process becomes a weapon in itself."

• "The important thing is that you don't know what to ask. A good question creates a good path — and that is the weapon."

5-2. The Human Role: Creator of Meaning and Context

• "In an era where the price of intelligence approaches zero, the quality of the question equals the quality of the answer. Defining the problem is solving the problem."

• "We must remain those who create meaning and value, those who interpret context and nuance."

6. Conclusion: The Wild West and a Survival Strategy for What Comes Next

• Right now feels like the "Wild West."

• "It really is like the Wild West. We are harness players, and yet we also have to build models."

• "Even though Amazon sells everything, individual shops still exist. Even after superintelligence arrives, countless opportunities will remain."

• "The future is already here — it's just not evenly distributed."

• "Questions are everything. The right question is your competitive edge."

• "The essence of all of this is compute. With enough compute, you can now do anything."

• "We are living in an era that is both incredibly difficult and, at the same time, incredibly exciting."

7. Summary and Takeaways

• The essence of AI lies in compute, data, and the internalization of tool use.
• The real competitive edge is the ability to ask the right questions.
• As we enter the Age of Experience, AI evolves by learning from its own experience.
• Creative data generation (daydreaming) and question design become the weapons of the future.
• The human role is to create meaning and context, and to collaborate with AI to generate new value.
• This is a Wild West era of chaos and opportunity, and the right questions combined with fast execution are the keys to survival.

"Questions are everything. The right question is your competitive edge." "The future is already here — it's just not evenly distributed." "The essence of all of this is compute. With enough compute, you can now do anything." "We are living in an era that is both incredibly difficult and, at the same time, incredibly exciting." "We are not tragic beings — we are beings with reason to be proud."

🌵 We are in the Wild West of artificial intelligence! The right question is your weapon. Let's open this new era together with compute, data, and creative questions! 🚀