Andrej Karpathy identifies a missing learning paradigm for LLMs. Pretraining provides knowledge, finetuning shapes habitual behavior, but much human learning happens through explicit instruction -- similar to system prompts. This "system prompt learning" paradigm may be key to more effective AI systems.

Related writing

Related writing