In this video, Jerry Tworek explains auto research as "a method of expanding the amount of computation at the time of testing by repeating the process of the model solving difficult problems, evaluating the results, and improving again." However, the current AI research system does not generate a sufficient variety of new ideas and is also vulnerable to reward hacking that deceives evaluation criteria and numerical instability. Therefore, for the time being, we argue that human-AI collaboration, which combines human expertise, intuition, and technical sense with AI's execution ability, is more powerful than fully autonomous systems, and in the long term, test-time training, where the model learns even during testing, could be an important breakthrough.

1. From reinforcement learning to test-time training

Jerry Tworek introduces himself and says that he joined OpenAI about 8 years ago. At the time, he believed that reinforcement learning was absolutely necessary on the road to AGI, and actually conducted research on applying reinforcement learning to large-scale language models.

"I joined OpenAI because I believe reinforcement learning is a necessary component on the path to AGI."

He assesses that his initial goal has been achieved to some extent, as many people now acknowledge the importance of reinforcement learning. He then left the company in early 2026 to take on a new challenge, and is currently leading a company called Core Automation, where he explains that he is researching a method to enable models to continue learning even at the moment of solving a problem, that is, test-time training.

"Now I'm focused on the next step: how to make sure the model learns effectively even when it's time to test."

The theme of this event appears to be "Auto Research" in the subtitle, and Jerry says that while researching reinforcement learning of language models for the past eight years, he has actually been working with an early form of auto research. His explanation follows the trend of shifting the focus of research from increasing the amount of training calculations to increasing the amount of inference calculations, and from fixed models to models that learn on their own.

2. Inference computation amount is more important than training computation amount

The most important graph introduced by Jerry shows the recent changes in machine learning in two directions. The first is the familiar training time scaling law. The idea is that if more computational resources are invested in model training, performance improves.

"The more compute you put into training, the better the model's results. This is the law of scaling."

This relationship has become widely known with the development of GPT-2 and GPT-3, but in the past, it was not easy to secure the same scalability in reinforcement learning. The OpenAI research team that Jerry participated in explains that they have discovered a scalable reinforcement learning method that improves performance as more calculations are input in reinforcement learning.

However, another unexpected phenomenon appeared in the graph on the right. Even for the same model, performance improves if you use more tokens and think longer when solving a problem. This is the scaling of test-time compute.

"The longer you allow a model to think, the better results you get from that same model."

This is something researchers have been hoping for for a long time. Previously, there was a strong tendency to think that a model's ability was fixed after training, but now it is possible to increase performance by having the model perform more inference and verification before providing an answer. This phenomenon is also related to why today's high-performance models use many more tokens per problem and why inference costs increase rapidly.

Jerry emphasizes that this change changes the way models are evaluated. Simply comparing one benchmark score means that you can no longer truly know the actual performance of the model.

"Once you start looking at these graphs, just talking about evaluation scores is no longer enough."

Even if a model gets a score of 80 on a benchmark, comparison with other models is incomplete unless you know how many inference operations it used to get that score. One model might get 80 points with a short inference, while another model might get the same score with a lot more tokens and costs.

Therefore, models should be evaluated not with a single score but with the following relationship:

• Relationship between performance and number of tokens used
• Relationship between performance and inference cost
• How performance improves when you allow more thinking
• At what point does performance improvement stop

"When you present a model, it's not enough to just throw a number and say 'this number is bigger.' You also have to show how much it cost to get to that number."

Jerry says that as it has already become a reality that performance improves as more computing is put into a problem, all evaluations must now look at 'what to put on the horizontal axis of the graph'**, in other words, how much money is paid to achieve performance and how much computation is required.

3. What is Auto Research?

Auto research, as defined by Jerry, is not simply asking AI to do a research task once. The model does not end after providing an answer once, but creates a repetitive research loop where a difficult problem is attempted multiple times, the results are verified, and the next action is decided.

"Automatic research is about continuously expanding test-time compute beyond what a single context can do."

Problems that are the subject of auto research are generally difficult problems that models cannot solve well at once. Even if the model cannot come up with a perfect answer right away when you tell it to "solve this problem," you can approach better results by repeating the following process.

1. Make an attempt to solve the problem.
2. Evaluate how close the results are to the goal.
3. Analyze causes of failure or possibilities for improvement.
4. Determine the direction for the next experiment or revision.
5. Run again and compare results.

For example, if you have a coding model implement a specific function, you can later add tests, review the code, fix errors, and verify it multiple times. The important thing is not simple repetition, but wrapping this process in a loop with clear goals and verifiable evaluation criteria.

"You have to set a very difficult but sufficiently testable goal. Give it a try, see how far you are from it, re-evaluate the next good step and keep going."

When the problem is structured this way, the model can use more inference tokens and experiments productively. This is because "what to do next" is determined at each step. Conversely, if the goal is vague or the results are difficult to evaluate, meaningful progress is unlikely to occur no matter how much the amount of inference calculations is increased.

Jerry explains that this repetitive structure is the core of auto research, but adds that it is an open question whether all research must be in the same loop form. In addition to repetitive attempts, there may be other ways to utilize test time computing, such as a structure that creates and compares multiple ideas simultaneously.

4. The power of human-AI collaboration demonstrated by GPU kernel optimization

Jerry introduces GPU kernel optimization as a practical example of auto research. The GPU kernel is low-level code for quickly executing specific operations on the GPU. The problem he mentioned was to quickly implement a linear algebra operation called QR decomposition, which is used in neural network optimization techniques.

Core Automation ran an auto-research loop in which AI created multiple kernels and measured performance. Initially able to create a kernel that was several times faster than NVIDIA's baseline implementation, the team considered Auto Research a success.

"We created a kernel that was several times faster than NVIDIA's baseline kernel. We were very happy and thought, 'Auto Research has worked.'"

Afterwards, a kernel optimization competition was held for people. However, the contest winner said that he did not write the kernel code himself.

"I didn't even read the problem or code."

He created the kernel using an AI model, and he only provided high-level direction to the model. As a result, the top kernel from the competition achieved performance more than 60 times faster than NVIDIA's baseline implementation, and was even better than the results produced by Jerry's auto research loop.

"The best kernel in the competition was over 60 times faster than NVIDIA's kernel, and even faster than our auto research loop."

This case shows two things. First, results vary greatly depending on the design quality of the auto research. It is not enough to simply ask AI to make many attempts; it is important to decide which model to use, which search method to use, and which evaluation criteria to design.

Second, current AI is still much more powerful when combined with human expertise. Contest participants were familiar with the GPU's memory hierarchy and how the hardware processes data, and based on this knowledge, they provided useful directions for AI.

"The combination of humans and models is still far superior to using models alone."

AI can write code quickly, never get tired, and check numerous possibilities. But human experts can better determine which ideas to try first, where hardware bottlenecks are, and what optimizations will work in real-world environments.

5. Assessment design and risk of reward hacking

In the GPU kernel competition, the performance alone produced surprising results, but when used for actual model training, many kernels did not work properly. Some were numerically unstable, causing errors during execution or breaking model training.

"Most kernels didn't work – they weren't numerically stable and failed."

The evaluation method that recognizes a kernel as good only if it passes the speed measurement can cause problems. In reality, what is important is not simply execution speed, but satisfying all of the following conditions.

• Is it fast enough?
• Are the calculation results accurate?
• Is it numerically stable?
• Can it be used for actual model training?
• Does it work consistently across a variety of inputs?

The second best submission in the competition met these criteria and performed 60 times faster than the NVIDIA benchmark. This shows that there is a big difference between simple benchmark numbers and actual usability.

Jerry warns that Auto Research is vulnerable to reward hacking. reward hacking is a phenomenon in which the system does not achieve the desired goal, but only obtains high scores by exploiting loopholes in the evaluation method.

"If an unmeasured factor is included in the goal, the model is very likely to exploit that loophole."

For example, if you only measure the speed of the kernel, the model may only increase speed at the expense of computational accuracy. Even if the model produces results that are actually unusable, the system appears to have succeeded if the evaluation function does not detect this.

Therefore, in auto research, evaluation design is as important as the research algorithm. We must carefully check whether we are truly measuring the desired results, whether there are hidden failure conditions, and whether the results are reproduced in the real environment.

6. Why auto research eventually stagnates

When you run the auto research loop, performance improves quickly at first. The model quickly discovers easy-to-find improvements, and large performance improvements are seen in the beginning. However, as time passes, the amount of improvement decreases.

"At first, there is a lot of improvement, but over time, the improvement becomes smaller."

This phenomenon is not simply due to insufficient execution volume. This is because the range of ideas and solutions that current models can explore is limited. Auto research is strong at exploring surrounding areas that the model already knows or can come up with, but is weak at finding new solutions that are completely beyond the model's capabilities.

Jerry likens this to a search method that explores some region of the solution space.

"Auto-research is a search method that explores the periphery of the solution space. It takes us to a point, but it is not the solution that takes us to the end of the problem."

In other words, auto research is not an all-purpose means to completely solve a problem, but is closer to a way to utilize already existing ideas and abilities more broadly and deeply. If the model fails to suggest a new research direction, the loop repeats the same type of experiment and eventually stagnates.

7. As the amount of computation increases, the algorithm becomes more important.

Jerry says it's a mistake to think that in machine learning you have to either "work on the algorithm or just scale it up." Rather, as the scale of computing grows, the quality of the algorithm becomes more important.**

"In the world of scaling, algorithms are much more important."

When spending hundreds of millions of dollars to train large-scale models or millions of dollars to run auto research, the algorithms that determine how to use computational resources greatly determine the results. At a small scale, the difference between a bad algorithm and a good algorithm may not be large, but as the amount of computation increases, the difference expands to enormous cost and performance differences.

"Auto research that costs $100,000 is different from auto research that costs $10 million. If you are going to invest a lot of money, you need a good algorithm to use the calculations most efficiently."

Therefore, future auto research must address the following issues rather than simply running more experiments:

• Which experiment to run first?
• How will you remember ideas you've already tried?
• How to utilize different ideas from different models?
• How do I prevent the search from getting stuck in a specific direction?
• How should evaluation criteria be designed to prevent reward hacking?

8. Key design elements presented by AlphaEvolve

Jerry believes that although there is still not much research on auto-research algorithms, the AlphaEvolve paper is an example of a relatively open explanation of important design elements.

There are four main elements that he paid attention to.

Accumulation of memories and experiences

Automated research systems must remember which ideas have been tried in the past, what results were obtained, and what failed. For this you need some kind of memory database.

If you cannot remember the results of past experiments, the system will repeat the same failure. Conversely, accumulating good ideas and failure patterns allows future exploration to proceed more efficiently.

Utilize various models and ideas

Not all models think the same way. Because different models can produce different ideas, using multiple models can increase the diversity of exploration.

"Different models can generate different ideas. How we inject diversity into our models is important."

Evaluation and Reward Hacking Prevention

You need to evaluate whether the system actually achieved the desired goals. By optimizing just a single indicator, the model can find loopholes in the evaluation function, so speed, accuracy, stability, and practical applicability must be checked together.

Parallelization and human intervention

By running multiple branches of the auto research loop in parallel, you can explore more ideas simultaneously. However, rather than simply increasing the number of parallel executions, providing humans with appropriate direction and judgment standards can have a greater effect.

"Even a small amount of human guidance can make a big difference in the results of auto research."

9. The biggest bottleneck is diversity of ideas

Jerry's biggest problem with auto research is the lack of diversity in search ideas. In typical research, a person first proposes an idea and AI can execute it. However, in auto research, the model itself must suggest what can be improved.

"The auto research loop is limited by the ideas the model can suggest."

If the model repeatedly presents already familiar solutions, no new results will be produced no matter how many calculations are performed. In the end, to improve performance, it is more important to determine how new and promising research directions the model creates rather than the speed of execution or the number of iterations.

This is why auto research performance improves every time a new model is released. Models with better research capabilities suggest better hypotheses and experiments, which in turn improves auto research as a whole.

"When better research models come out, auto research also works better. Every time a new model comes out, everything gets better, like Christmas."

But an important question remains. Besides simply creating a larger model, how to separately optimize the model that studies better.

10. The next step is test-time training.

Current auto research loops are largely based on a fixed model. The model encounters many experiments and data during the research process, but the internal state of the model itself does not fundamentally change.

Jerry presents the possibility of test-time training here. This is a method that reflects the data and insights gained during the research process back into the model to solve problems and make the model better.

"Current auto research loops largely operate on fixed models, but what if we could inject what we learned during our research back into the model?"

For example, if a model tries a specific problem multiple times and learns which approaches fail, which structures are effective, and which evaluation criteria are important, that information can be used in subsequent inferences. In this case, auto research is not simply a repeated use of fixed capabilities, but becomes a process of improving the model itself during the exploration process.

Jerry explains that this is his area of ​​focus at Core Automation.

"If you're putting a lot of computation into exploration, it's very important to have a way to get the benefit of better models along the way."

test-time training is not simply learning by adding data. It is a new deep learning algorithm problem that concerns how a model that has already completed pre-training learns to adapt to new problems and environments and how efficiently it utilizes that knowledge.

11. Rather than experimenting a lot, you should think more deeply.

Concluding his presentation, Jerry gives unexpected advice to those designing auto research. Rather than trying to run as many experiments as possible, design each experiment more deeply, even if it means reducing the number of experiments.

"My advice to anyone creating auto research today is to run fewer experiments and think more about them."

Many researchers focus on obtaining more iterations and more data points. However, if you repeatedly deal with problems that are easy to verify and low cost, the search may get stuck in a simple problem.

Conversely, if you think deeply about which experiments should be conducted to obtain the most information, which hypotheses should be verified, and what can be learned even if the results fail, you can achieve better expansion effects.

"Inferring more deeply about which experiments to run rather than how many data points to collect can lead to much better scaling curves."

Jerry says auto research should not be viewed as simply a data generation process. What is important is not the quantity of experiments, but the quality of the experiments, the amount of information, and the value they give to the next decision.

12. Remaining algorithmic problems in test-time training

In the subsequent Q&A, Jerry cited lack of an algorithm as the biggest problem that test-time training needs to solve. Test-time training is advantageous in that new data can be obtained continuously. This is because data is generated from the problem that the model is actually solving.

"In test-time training, the data itself is not the biggest problem, because the data comes from the problems the model actually addresses."

However, currently, there are not enough good algorithms on how to effectively learn from only a narrow range of newly encountered data and experiences when the model already has a lot of knowledge.

"At the moment, there are no good algorithms yet for test-time training."

Therefore, this field is an open research area where deep learning researchers must develop new learning algorithms. The model must maintain the knowledge gained from pre-training, while efficiently updating only the parts necessary to match the problem currently being explored.

This is different from simply retraining the model. Use limited data and experience with a specific problem to improve the model's ability, but without losing existing knowledge or over-adapting in the wrong direction.

13. Where do human expertise and 'technical sense' come from?

In the Q&A, questions were also asked about how human expertise can be injected into auto research and where the researcher's technical taste comes from.

In the GPU kernel case, the human expert did not write the code directly, but was able to provide the following directions to the model.

"When you start this problem, you must think about these three things. Make sure you have these things in order."

People who have been writing GPU kernels for many years have a deep understanding of how the hardware works and the memory hierarchy. Now, even if you don't write the code yourself, the model can tell you key elements that are easy to miss.

"Models work quickly, don't get tired, and can check a lot of things, but they don't have all the insight into writing a good kernel that people have in their heads."

Jerry doesn't offer a single, simple answer as to where this technical acumen comes from. Currently, an important foundation is the method of learning various cases, human knowledge, and research results into the model through large-scale pre-training.

However, through test-time training, only the senses needed for a specific field or problem can be strengthened separately. Rather than re-learning information from the entire Internet, it involves selecting a specific, small area within a large knowledge space and learning that part with greater precision.

"Can we take a very small corner of the entire Internet and model that area very well without breaking other capabilities?"

This is an important research question for the future. This is because the model must not only have more knowledge, but also the ability to judge which ideas are good, which experiments are valuable, and which failures are meaningful.

14. Is it possible to predict test time compute needs?

The final question was whether it was possible to predict in advance how much test-time compute would be needed to solve a problem.

In the case of training computation amount, it is possible to estimate how much computation is needed based on approximate scaling laws and past data. But in test-time compute, the problem is much more complex.

Jerry responds that although it is possible to extrapolate performance improvement trends by combining the results of multiple experiments, the predictions are not always stable.

"You can try to extrapolate, and sometimes you have some success, but extrapolation is very difficult."

He gave an example of instructing the model to bring the GPU kernel up to a certain performance level. Depending on how the goal is set, the direction the model explores may vary. If you present a realistically achievable goal, the model will focus on that goal, but if the goal is too high or incorrectly set, the search itself becomes inefficient.

"Where you set your goals also determines how far your model will attempt."

Of course, no matter how much computing is put into it, physical limitations cannot be overcome. Asking to create a kernel faster than the speed of light doesn't mean you'll get that result. This is because there are fundamental limitations in the structure and environment of the problem.

If you conduct many experiments, you can discover rough patterns, but if the environment changes or the problem becomes more difficult, existing patterns may break. Therefore, rather than perfectly predicting test-time compute needs, a realistic goal is to use historical data to avoid completely blind exploration.

"There are ways to help us avoid running completely blind, but it all depends on the problem and the model. There are still uncharted areas on the front lines."

Conclusion

The auto research that Jerry Tworek talks about is not a technology that entrusts research to AI and repeats it infinitely. The key is to turn difficult goals into testable, iterative processes, use more inference operations efficiently, and continually improve based on past results and new ideas.

However, current AI still falls short of humans in the diversity of new research ideas, design of evaluation criteria, numerical stability, and technical sense. As the GPU kernel example shows, the most powerful approach combines the fast and tireless execution of AI with human expertise and intuition.

In the future, test-time training, which goes beyond repeated inferences by a fixed model and reflects the experience gained during the research process into the model itself, is likely to be an important next step. However, for fully autonomous research to be realized, better learning algorithms, search strategies, and methods of conveying human "good research sense" to models must first be developed.