This video is a conversation held at Sequoia Capital's "Own Your Intelligence" event, in which Mercor CEO Brendan Foody explains what reinforcement learning environments are, why they are the hottest topic in AI training data right now, and how they are changing companies' AI strategies. He introduces the three components of an RL environment (worlds, apps, tasks) and shares a real case of building a legal RL environment along with post-training results. He also offers insight into how data is priced, what data quality means, misconceptions about synthetic data, and where things are heading (ultra-long-horizon tasks and virtual co-workers).

1. The evolution of the data market: from crowdsourcing to agentic data

Brendan begins with Mercor's story and the history of the data market. It started in 2020 with the era of crowdsourced data for behavior cloning. Back then the main inputs were supervised fine-tuning data and RLHF data, where annotators picked their preferred response among several model outputs.

"With agentic data from the crowdsourcing era we were able to achieve GPT-3 fine-tuning and all the progress toward ChatGPT and GPT-4."

But as 2024 arrived, an enormous shift hit the market. It moved away from low-skill crowdsourcing into an era of agentic data, in which the world's top experts collaborate to build frontier evals and RL environments.

"Software engineers, lawyers, doctors, bankers — every expert who can measure the frontier of intelligence is contributing to improving model capability."

Mercor's first large project was deep research, and it went on to become the core agentic data supplier for every major frontier lab as well as leading application-layer companies such as Harvey, Cera, Cognition, and Ramp. The most interesting change over the past twelve months in particular is that RLVR (reinforcement learning with verifiable rewards) has evolved, within the agentic data paradigm, into RL environments that include rich apps and worlds.

2. What is an RL environment: worlds, apps, and tasks

An RL environment consists of three core components.

The first is worlds. These include every message, slide, document, and spreadsheet you might have in a real project or company.

The second is apps. These are high-fidelity clones of popular applications such as Salesforce, ServiceNow, and Microsoft 365, which agents can interact with through MCP, a CLI, or a GUI.

The third is tasks. These consist of prompts and verifiers, where the verifiers take the form of rubrics or unit tests and are used for evaluation or training.

The core challenge frontier labs face is covering the full distribution of every world, app, and task that exists across the economy.

"You have to expand the environment distribution across every category of the economy. Some of you may know GDP Val — the 205 occupational domains from the U.S. Bureau of Labor Statistics — but you have to think about how to build the apps corresponding to all those occupations, the varied scenarios, and every task. It's a construction job at an enormous scale."

Expert hours contributed through Mercor's expert network have exploded over the past 24 months. In the second quarter alone they reached 2.5 million hours, and the growth rate keeps accelerating.

3. Why only humans can measure the frontier

In most domains, only humans can measure the frontier. Rare exceptions like mathematics have clean simulation environments where the model can learn on its own whether an answer is correct, but most domains are different.

"In a domain like building a slide deck, it's enormously difficult for the model to reliably identify where it went wrong. It's like asking a human to grade their own homework."

That is why it matters that humans build the rubrics — the way a professor creates a rubric to grade essays, or a teaching assistant evaluates a slide deck. It resembles the fact that the way we learn also depends heavily on feedback from the people around us rather than on calculators or clean simulations.

Building verifiers is hard. If there are ten good paths to consider when building a slide deck, the core challenge is how to build a comprehensive verifier that catches all of the dozens of mistakes that could arise along them.

4. A real case: building a legal RL environment

Brendan uses a publicly released legal RL environment as an example. It was built on project scenarios that lawyers from top-tier firms such as Latham & Watkins had actually worked on.

The lawyers write real project scenarios from their Big Law experience and create the full outline corresponding to a data room. This includes various messages, emails, files, and file sizes. A model in the loop is used heavily in this process too.

"It's similar to how a software engineer now shouldn't hand-write all the code but should work very productively by orchestrating agents."

The data room produced this way is rendered into an app clone such as Google Workspace, and prompts are attached to evaluate model trajectories. In the example, the task was "assessing the maximum aggregate liability between Star Tanker Tankers International Limited and Cooper Jefferies Energy Corporation under the Oil and Petroleum Act."

Just as a professor builds a rubric for grading essays, this environment has core rubric criteria corresponding to the characteristics of a correct model response. And verifying that every score is accurate when you roll out 100 trajectories is enormously difficult technical work.

5. Leaderboards and the shift in open weights

Leaderboards provide a composite score for how well various models perform in a given domain. One of the big changes in recent months is that open-weight models such as GLM 5.2 and Chimera K3 have appeared on the leaderboards.

"This is an enormous opportunity for all of you, because it provides the foundation to actually achieve frontier intelligence in a specific application and vertical."

Post-training results: Apex Agents

Post-training GLM 4.7 using Mercor's Apex Agents dataset (1,800 tasks) produced striking results. With only about $500,000 worth of compute, the improvement was dramatic.

• Corporate law: 4.7% → 26.6%
• It also generalized excellently to GDP Val and Apex V1 (a benchmark without data rooms).

This shows that when a company builds an RL environment specialized to its own domain, it can achieve frontier intelligence on its own. Brendan notes that Cursor is an excellent first case of an application-layer company building a best-in-industry model, and predicts that dozens of such companies will emerge within the next twelve months.

6. Three ways to buy data

There are broadly three ways companies build high-quality datasets.

1. Custom data per task. The most common approach. A company that wants a particular data shape pays $2,000 per task and scales from there. Some frontier labs buy 50,000 tasks a month. Tasks vary enormously in complexity — some take a month to complete, others a few hours.

2. Off-the-shelf data. Mercor sells its own datasets, built with hundreds of millions of dollars of investment, to multiple customers. Neo labs prefer this approach, because it is inefficient for ten different labs to each build the same dataset.

3. Providing experts. Supplying experts on an hourly model. This is how Harvey first started with Mercor, though over time there is a tendency to move toward broader data provision.

7. Q&A: How is data priced?

Brendan explains that data pricing works backward from the model improvement the customer is trying to achieve.

"The customer has a goal of reaching the frontier on a particular leaderboard. We can work backward with the customer on what that is worth and how many tasks are required to reach that goal. A company like Nvidia would gladly pay a billion dollars to have a frontier open-source model."

Prices span a very wide range, from $50 to $10,000 per task. Margins are determined by the cost structure (a baseline cost of $1,500 for ten hours at $150 per expert hour) and the degree of differentiation.

8. Q&A: What "data quality" really means

When people say data quality, they actually mean two things.

The first is realism. Reflecting the demands of customers who want to automate everything in the economy, it means building environments, apps, and tasks that accurately reflect the real distribution of that domain. This is why experts create the outlines and guide the data curation process.

The second is verifier accuracy. The training method is to roll out 100 trajectories from a model like Kimi K3 and score them with the rubric. Because there are countless paths a model can take, you have to confirm that the rubric scores match how a human would stack-rank those 100 trajectories.

For this they use a process called trajectory analysis. They roll out ten trajectories from the model being improved, score them, and then combine an agentic quality control system with human review to confirm every score matches the target. Sometimes human feedback evaluations or preference labels are used as evaluation tools for the automatic grader.

9. Q&A: Misconceptions about synthetic data

Brendan points out that there are many misconceptions about synthetic data.

"RLVR is actually a bet on synthetic data. Instead of humans writing SFT, you roll out synthetic model trajectories in bulk, score them all, and then have the model learn from those synthetic trajectories."

Models also play an enormous role in populating environments and creating tasks. Just as a lawyer writing a legal memo should be using Claude or ChatGPT, the experts building data rooms should actively use AI tools.

But humans remain essential because, by definition, you have to measure beyond the frontier of model capability.

"You can't tell the model, 'Build a legal environment, and tell me which of your legal memos are good and bad.' That's far too noisy and there's no clear signal. You need something with capability beyond that model's frontier."

10. Q&A: Why RL environments now, and what comes next?

Deep research environments came first because they used search tools. Search was the tool of the environment the model worked in, but experts did not need to populate apps. It was a lighter version of an RL environment.

The app trend became enormous in 2025 because people realized that the core bottleneck on model usefulness lies in how it uses all the context in a codebase and all the tools on our laptops.

"If you want this in the user distribution, you have to put it in the data distribution the model learns from too."

Two big changes are expected going forward.

The first is ultra-long-horizon tasks. Today's agents are mostly not trained to perform work that takes more than ten hours. Humans have to build tasks for work that takes 100 or 1,000 hours.

The second is the introduction of virtual co-workers.

"My favorite question is, 'What proportion of your work consists of tasks requiring interaction with another person?' Most people answer 60% or 70%. Yet the proportion of evaluations that measure how well a model interacts with other people is only about 1%. It's included a little in Tau Bench."

This points to an enormous realism gap in measuring how well agents perform the social interactions required to collaborate at work with various people and with other agents.

11. Q&A: Can rubric generation be scaled with models too?

An AI copilot working with experts to create tasks and verifiers can substantially raise efficiency, because the expert can converse with the trajectory to understand exactly what happened and where it went wrong.

But the model being improved cannot reliably write the rubric criteria for its own mistakes.

"FABLE (the model being improved) cannot reliably write the rubric criteria for its own mistakes. It might be half right and half wrong, and that level of noise is unusable from a training perspective."

Task creation is the process that needs humans most. Much of the environment can be filled with synthetic data, but tasks require human-written outlines. The exceptions are the coding domain, and cases where you can learn from tasks created by a model better than the one being improved (such as Kimi K3).

12. Q&A: RL environments for cyber defense

Cybersecurity is a field where verifiers do not always need humans, because you can set up attacker and defender agents.

"In cyber, humans are more involved in designing and setting up realistic environments, but building verifiers is less human-intensive."

Humans must design the environments themselves to ensure diversity, but verification can be solved through interaction between agents.

13. Q&A: How do you know the limits of a base model?

Asked why final model performance differs even with the same dataset, Brendan answers that parameter count and trainability matter. The key indicator is the gap between Pass@1 and Pass@16.

"If you roll out 16 trajectories and all of them are completely wrong, there's almost no chance that model will learn from that dataset. The ideal case is that it fails at Pass@1, but when you roll out 16 trajectories it gets one or two right. Then the model can learn from that dataset very effectively."

This is the general heuristic for judging how strong a base model needs to be in order to learn effectively from a given dataset.

14. Q&A: Build the data yourself, or partner?

A large part of Mercor's business is custom data, running teams that are entirely dedicated to core customers. This lets customers retain their competitive advantage while still benefiting from the infrastructure Mercor has built.

"Some companies try to build the entire talent network and infrastructure in house. But looking at what the frontier labs and the best models do is a good indicator that there are far more advantages in partnering with someone who has economies of scale."

The key is partnering with someone who has the platform, the talent network, and economies of scale all at once.

Conclusion

In this video Brendan Foody makes a persuasive case that RL environments are not merely a technical concept but a core factor reshaping the AI industry. An RL environment, made of worlds, apps, and tasks, is the method for reflecting the real distribution of work across the economy into model training data, and human experts remain essential because models cannot measure the limits of their own capability.

As Mercor's case shows, the arrival of open-weight models and post-training results that dramatically improve domain-specific performance with modest compute mean this technology is no longer the exclusive property of frontier labs but an opportunity for application-layer companies to build their own intelligence. The direction ahead is building more realistic environments including ultra-long-horizon tasks and virtual co-workers, in an era where the ability to build datasets becomes a company's moat.