How to get AI agents to code.
- Easy problems, problems with known answers, and famous problems get implemented well almost no matter what.
- For hard problems, problems without a known answer, and problems only you have, give the agent plenty of input-output data and tell it to pass all of it.
The screenshots show an example where I gave around 40 large and small input/expected-output pairs in test code to fix a rendering issue in common markdown, where CJK text was not rendering properly because of flanking rules. If I had tried to write the regex myself, I probably would have suffered through the variety of cases and the tangled complexity.
A sufficiently complex function is indistinguishable from a model. Both are black boxes, but they work.
So similar tips apply. If mutually conflicting cases are mixed together in the input-output data, the agent burns a lot of time and tokens and fails to derive a pattern. If there is too little input-output data, the implementation gets oversimplified. You get overfitting.
So what is left for humans to do in coding today?
- The era when people personally thought through and implemented the How, like data structures and algorithms, is fading. Let coding agents handle the How.
- Be clear about What to build. If it is a problem with an established best practice, solve it that way.
- For problems only you have, our job is to gather the input-output data and provide it. (SFT-ish)
In the near future, coding, service implementation, and operations will look like defining a cost function for each feature experiment and then going back and forth with the agent. You define the key metric and guardrails for this month, and the system experiments, improves, and runs on its own.
