This interview with Sail Research cofounder Neil Movva argues that AI's next competition is not a faster chatbot, but background agents that work autonomously for long periods and the extremely cheap token costs that make them possible. His strategy combines software optimization, heterogeneous chips, distributed data centers, and unreliable but inexpensive renewable energy to lower token prices by thousands of times. The goal is to make AI not an expensive expert reserved for hard questions, but abundant intelligence that is always working beside us.

1. Sail Research's Goal: The Cheapest Token Factory

Host Patrick O'Shaughnessy first asks what Sail Research actually builds. Movva gives a simple, striking definition. Sail offers open-source large language models through an API and aims to supply tokens for any customer task at a price the market cannot beat—a token factory.

It also provides cloud sandboxes, long-running virtual machines where agents can execute for hours, days, or weeks. Sail focuses less on chatbot inference where a person waits for an instant answer and more on infrastructure for agents that work independently over long periods.

"Sail Research is a token factory. Anyone can send requests through our API to use open-source large language models, and we will provide tokens at an unbeatable price."

Movva repeatedly returns to abundance. The fact that AI can think is already profound; the next task is making as many machines as possible think and giving as many people as possible access to that intelligence.

"When you make something ten times cheaper, it becomes a new product category. That is what we want to do with tokens."

Tokens are today's basic measure of AI usage, but may not remain the final "unit of intelligence." Customers may eventually pay for the outcome an agent creates, not the number of tokens. Different tasks require different reasoning time, tool calls, and attempts.

2. From Chatbots to Background Agents

The largest transition Movva sees is from instant-answer chatbots to autonomous background systems. AI infrastructure has been designed around a user entering a prompt and waiting at the screen, making lower latency the industry's focus.

Movva argues that "an answer you need not wait for" is a better goal than "a faster answer."

"The best latency is no latency at all. I want to wake up and find that the work was completed overnight. Perhaps I did not even need to request it."

He does not want users managing AI every five minutes. People do not manage coworkers that way. They assign large tasks and check a day or a week later. Human-agent collaboration should follow similarly human time scales.

"You do not manage a coworker every five minutes. You give them something large and check once a day, or usually once a week. That is the future of human-agent collaboration."

The technical basis is test-time compute scaling: allowing more reasoning time and computation can produce better answers. Once mostly theoretical, recent models can now perform tasks lasting around an hour. Fully autonomous multi-day work is early, but task duration is steadily increasing.

Movva expects real-time and background work to become similar in share soon, then settle around 90% background and 10% real-time over the long term. Human screen time limits interactive consumption, while AI running during sleep or other work is effectively unlimited.

3. Work Transformed by Long-Running Agents

Movva first names deep research. Problems requiring synthesis of more than 10,000 sources, not merely 100 or 1,000, do not fit a conversational model where a person waits.

Sail customer Parallel Web Systems, for example, wants to index the entire internet and track changes in real time—exabyte-scale data requiring tremendous intelligence and continuous computation.

Another central area is cybersecurity. There are far more ways to attack or break code than to generate it. AI can continually test all software, find vulnerabilities, and propose patches.

"Security is becoming proof of work. How secure software is increasingly means how much money you spent running AI to break it."

Movva doubts one giant model can solve every security problem. A small model may find a bug a large one misses, and models differ across vulnerability types. Security agents should autonomously attack using multiple models and search methods, then strengthen defense from the results.

For individuals, agents could become highly proactive assistants that continually examine email, messages, schedules, and habits and propose the next action. Unlike current voice assistants, they would understand life broadly and act first.

"If a machine is trustworthy enough and protects privacy, it could suggest your next action whenever you open the phone. Can we model what we will do next well? I think yes."

He distinguishes non-verifiable questions of taste, art, and beautiful writing, where AI has not fully solved the problem. Sail concentrates on quantitatively evaluable tasks with verifiable answers, leaving artistic judgment to people.

4. Nvidia's "Speed of Light" Culture

Movva worked at Nvidia in university, optimizing GPU kernels and forming his technical philosophy. He was present when Tensor Cores, dedicated units for fast matrix multiplication, were first introduced.

Matrix multiplication is central to mixing and calculating relationships among the many numbers in AI models. Nvidia began as a gaming GPU company, but in the mid-2010s saw researchers training deep-learning models on its hardware and invested seriously in AI.

Adding Tensor Cores was not easy. Chip area is expensive and limited; more tensor compute can mean sacrificing graphics functions. Movva remembers fighting for roughly 5–10% of die area to accelerate computer-vision workloads.

The defining Nvidia culture he learned was pursuing the "speed of light"—the physical upper performance bound of the hardware.

"If a machine can do something, we push it to the limit, break every bottleneck, and continue until we reach the best performance we believe is possible."

At Sail, he cares less about being a few percent faster than a competitor than about extracting the chip's absolute potential.

"I tell engineers to pursue 100% of the speed of light. I do not care about relative numbers. I care what this chip can do absolutely."

Nvidia's talent retention also impressed him. Many engineers from 2015–2016 remain, treating parallel computing and silicon design as lifelong work. Operations were remarkably frugal: employees even paid monthly into a "milk club" for coffee milk rather than receiving free lunch or milk. Movva recalls that frugality permeating the organization. 🥛

5. Why Throughput Matters More Than Fast Responses

Movva starts by adapting AI use to what GPUs do best. GPUs are most efficient when processing many tasks in parallel. Chatbots require one user's answer now, making batching difficult and leaving parallel capacity underused.

This reflects the fundamental trade-off between latency and throughput:

• Low latency requires handling one request immediately.
• High throughput requires batching many users' requests onto the GPU.
• Larger batches improve total GPU efficiency but make individual requests wait.

Movva compares cars and buses. A car takes one person directly and is faster for that person. A bus carries many people and stops repeatedly, making an individual trip slower but improving total road efficiency.

"Narrow and fast versus wide and slow is a classic trade-off throughout computer science."

Sail focuses on building the best bus. Long-running agents do not demand instant responses, allowing large batches and maximum GPU utilization, which lowers token cost.

Nvidia's NVLink connects multiple GPUs at high speed and is excellent for reducing latency. Splitting large matrix operations across GPUs speeds a response, but eight GPUs are not exactly eight times faster because of communication and partitioning overhead.

Movva accepts that NVLink is nearly essential for low-latency inference, but not for Sail's main workloads. Chips with slower interconnect can still be valuable if they perform matrix operations cheaply.

"Most of the time I look at the chip's hourly price and FLOPS per dollar."

FLOPS measures floating-point operations per second and approximates compute capacity.

6. Memory, KV Cache, and Roles for Different Chips

The discussion turns to AI's central memory bottleneck. Chips such as Cerebras and Groq pursue ultra-fast responses using large amounts of on-chip SRAM.

SRAM sits physically close to compute and reads quickly, but consumes die area and cannot easily provide large capacity. DRAM such as HBM around a GPU is much slower but much larger. Nvidia Blackwell has roughly 288 GB of HBM and only a tiny amount of on-chip SRAM by comparison.

Cerebras exceeds conventional chip-size limits by connecting an entire wafer as one compute device, providing vast SRAM and memory bandwidth. It can read large model weights quickly and potentially generate thousands of tokens per second.

The important complication is the KV cache, dynamic memory that expands as a conversation grows.

"Think of model weights as crystallized knowledge and KV cache as dynamic knowledge about the exact conversation we are having."

KV cache can eventually become larger than model weights and changes with user count and context length, making SRAM alone inadequate. Movva expects ultra-fast chips to serve as accelerators with distinct strengths rather than fully replacing GPUs:

• Cerebras/Groq-type chips: strong at fast memory access and large matrix operations
• Traditional GPUs: large external memory suited to long contexts and KV cache
• Future systems: hybrids assigning compute and memory phases to different chips

Transformers join compute-intensive and memory-intensive stages in one architecture. One chip is unlikely to be best at everything, making heterogeneous combinations realistic.

7. Transformer Strength and the Next Data Era

Movva explains why the Transformer, introduced in 2017, changed AI. Its center is attention, which lets a model assign weights to the parts of a data sequence most important for the current prediction.

"A transformer tries to find patterns in any arbitrary sequence of data. If a pattern exists, it will add parameters to find it."

Language is the richest sequential data humans continuously produce, so Transformers dominated language modeling. Crucially, the architecture scaled from millions to trillions of parameters.

Every token can refer to every other token. That can be inefficient, but powerfully captures any possible relationship. Until more selective architectures prove themselves, attention remains excellent.

On data, Movva calls the internet a one-time subsidy. It contained about 30 trillion tokens of high-quality text, or perhaps 300 trillion under a broad definition, but major models have already trained over most of it repeatedly.

"The internet was a one-time data subsidy. We got it free, and now we have seen almost all of it."

Future training will depend less on random user feedback and more on verifiable tasks and reinforcement-learning environments. In coding, mathematics, and scientific experiments, success can be measured while agents repeatedly try and improve.

"Give the model a difficult but verifiable task and measure whether it makes progress. The environment itself becomes data."

The path toward AGI may not be one general intelligence that instantly knows everything. It may involve accumulating abilities across verifiable specialist tasks and steadily closing gaps. The essential condition is a self-grading system through which an agent judges performance.

8. Software Optimization and Reviving Idle GPUs

Sail begins by using software to extract maximum efficiency from existing GPUs. It optimizes kernels, small programs executed on GPUs. If matrix multiplication followed by addition can be fused into one kernel rather than writing intermediate data to memory and reading it back, cost and time fall.

Movva expects AI to increasingly assist kernel development. Engineers will design an operation strategy on a whiteboard, explain it in natural language, and let a model implement rather than hand-writing every line.

"We write kernels on the whiteboard rather than by hand. We explain what the machine should do and let the model implement it."

He acknowledges that stronger coding models will make such optimization widely available. Sail's differentiation may be execution: connecting new chips to production sooner and redesigning the full system around them.

GPUs can reach 70–80% of theoretical performance on a happy path such as large matrix multiplication. Real Transformer inference does not always consist of large batched operations. Sail's work is designing the data center and software so GPUs spend as much time as possible doing what they do best.

Movva says the era requires programming not individual GPUs, but entire racks and clusters as one computer. Nvidia itself now presents rack systems of 72 units. Competition may center on who controls rack-scale computers most efficiently.

He is especially frustrated by idle GPUs around the world, trapped in private pools reserved for particular customers.

"It physically hurts me to see a GPU containing silicon and power simply sitting idle."

Sail wants to connect global compute, find vacant capacity, and efficiently assign many chip types within one system.

9. Opportunity Amid Chip Scarcity: No Bad Chips, Only Bad Prices

Patrick compares the cutting-edge Blackwell market to a black market: demand is explosive, supply scarce, and companies desperate for chips.

Movva believes Nvidia strategically allocates supply around long-term relationships and ecosystem health rather than raising prices without limit. Letting one cash-rich company take every chip might harm Nvidia over time. A startup promising to rent tens of thousands of GPUs for years is also difficult to trust. Capital and relationships matter.

Sail does not focus only on the latest GPU.

"There are no bad chips, only truly bad prices. At the right price, I will make any chip work."

AMD chips perform well overall but can remain underoptimized because their ecosystem and software support trail Nvidia. That is opportunity: lower market preference lets Sail buy cheaply and extract more value through its own optimization.

Movva is willing to use TPUs, AWS Trainium, AMD, and chips from new semiconductor startups. The point is finding each chip's comparative advantage.

"I like TPUs; we will make them work. I like Trainium; we will make it work. Every chip has comparative advantage, and we need to find it."

This is compute arbitrage: find hardware the market undervalues, optimize it better, and sell inexpensive inference.

10. Why Data Centers Should Be Small and Distributed

AI data centers were designed for massive training, where tens or hundreds of thousands of tightly connected GPUs and extremely fast, reliable internal networks are required.

Movva believes inference—especially long-running background inference—does not require that structure. Securing a new 100 MW or 1 GW power source in one US location is very difficult, while small resources around 1 MW exist widely.

He wants to connect distributed small data centers into one inference fleet. Liquid cooling means one megawatt of compute can fit in roughly eight refrigerator-sized racks rather than a giant warehouse.

"We will buy any chip, anywhere in the world, for any duration. I do not think anyone else has that flexibility and liquidity today."

Distributed facilities may be unattractive to conventional customers. Without diesel backup, multiple grids, and redundant fiber, uptime may fall to 95%, which would be catastrophic for a normal large data center.

Sail is willing to become the first buyer.

"I will be the first customer for 95% uptime."

There are two reasons:

1. Sail's control system can move work when a facility stops.
2. For an agent working for hours, one response arriving minutes later may not matter.

"We promise highly competitive average throughput. We cannot control p99 latency—the rare slow case. In return, we give overwhelming economics."

This does not suit a person waiting at a screen, but fits overnight research, security, and coding agents.

11. A Scavenger Strategy for Solar and Wind

The distributed strategy extends from chips to electricity. Movva believes US solar and wind remain underused, but their intermittency makes them unsuitable for conventional data centers demanding stable power around the clock.

Sail can tolerate outages lasting days or weeks if the price is low enough. If forecasts show prolonged low wind, cloud, and fog, work can move to another region in advance.

"The worst case is a long outage with no wind and persistent cloud. But that is predictable. Model the weather and move work when the data center shuts down."

He calls this the scavenger strategy. Instead of competing directly with OpenAI or Anthropic for premium power and cutting-edge GPUs, Sail gathers supply large companies ignore because it is operationally complicated.

"First we scavenge chips, then power for those chips. I do not want to bid against Anthropic or OpenAI for compute capacity."

He compares it to a mini-mill model: rather than one gigawatt factory, combine inexpensive power and chips across many small locations to build the cheapest collective token factory.

12. The Dream of Vertical Integration and Practical Operations

Patrick asks whether Sail might become fully vertically integrated across power generation, data-center construction, chip design, software optimization, and token sales.

Movva admits the founder in him wants to do everything. He has spent his life thinking about chips, energy, power, and data centers and wants the most efficient end-to-end system.

"I will not stop until I have built the most efficient system from beginning to end."

As CEO, he must remain practical. Owning everything requires enormous capital. Starting with highly leveraged software and partnering in power generation and data-center operation may be more rational.

Once Sail reaches sufficient scale and partners cannot provide custom solutions, it may build directly. Movva believes AI inference can become the largest market in computing history, creating economic justification for inference-specific hardware, power, and facilities.

13. The Largest Waste: KV Cache and Compute Allocation

Movva distinguishes optimized and inefficient areas. Sparsity, particularly Mixture of Experts, has improved substantially. Modern models activate only a small share of experts, saving compute.

The largest inefficiency is memory use in attention and KV cache. Current systems store large amounts of data per token, likely with redundancy. Movva believes reductions of 10 to 100 times may be possible, not merely single-digit percentages.

"We store several kilobytes per token in KV cache. We may be wrong by one or two orders of magnitude."

DeepSeek and others publish cache-compression research, with major annual improvements indicating room remains.

At a macro level, global compute is poorly organized. Nvidia may ship millions of GPUs, but many assigned to private clusters remain idle, wasting silicon and electricity.

14. Semiconductor Supply Chains and the Future of HBM Scarcity

Movva advises AI hardware founders to understand supply-chain bottlenecks, not only technical ideas:

• TSMC wafer capacity
• High-bandwidth memory such as HBM
• Advanced packaging capacity
• Data-center electricity

Nvidia is strong not because it is perfect, but because it balances trade-offs exceptionally well. A new company should not beat Nvidia everywhere, but build a sharp advantage around one bottleneck.

Movva especially emphasizes HBM scarcity. Memory fabs demand huge capital, and manufacturers burned by cyclical overinvestment are reluctant to expand aggressively. Scarcity can affect memory prices in phones and other consumer electronics as well as AI chips.

One long-term direction is reducing dependence on HBM. If architectures can offload more KV cache to flash, systems can bypass expensive high-bandwidth memory constraints.

"I keep working at the whiteboard on model changes that could move far more KV cache onto flash."

His geopolitical view differs somewhat from conventional fear. Losing TSMC access would be a serious shock, but Western process technology may not be overwhelmingly behind; the worst penalty could be roughly twofold in energy efficiency. He argues the gap is smaller than chip-industry conversation often implies.

15. Open and Closed Models Will Coexist

The final major topic is the relationship between closed and open-source models. Movva believes leading labs such as OpenAI and Anthropic rationally spend enormous sums to stay three to six months ahead.

That advantage may not remain forever. As AI-generated code and documents rapidly accumulate online, open models can absorb leading-model capability indirectly through public outputs without directly copying the models.

"A growing share of internet output is AI-generated. Fundamentally stopping the spread of information and model capability seems impossible. The question is only how fast."

Companies do not switch immediately to each new model; security, validation, and operational stability slow adoption. Open models can therefore remain competitive even with a three- to six-month technical lag.

Movva considers the disappearance of open source highly unlikely. If one leader withdraws, another will fill the position, and model training continually becomes easier.

His desired future is not a handful of giant model companies supplying all intelligence, but each company and user configuring and owning AI suited to them.

"I want abundant tokens and diverse harnesses. I want every company, even every user, to build their own agent."

Personalization need not require fine-tuning weights. It can happen through in-context learning that supplies the user's context and data.

16. The Goal Is Ultimately to Make Tokens Thousands of Times Cheaper

Movva closes by defining his job clearly: make tokens as cheap as humanly possible, using every supply-side lever across chips, power, land, data centers, and software.

"My job is to make tokens as cheap as humanly possible, and I will do it through every layer of the stack available to me."

His desired cost reduction is not small. A trillion tokens per day on some models can cost millions of dollars. Reducing that by 1,000 times or even toward a million times opens entirely new use cases. For some model sizes, trillion-token workloads are already approaching tens of thousands of dollars and may fall further.

When Patrick asks whether people truly want that much intelligence, Movva is emphatic.

"I will never believe the world lacks demand for intelligence. There is always demand for intelligence."

The issue is not demand for intelligence, but products and entry points that make it easy to use. He wants to change a reality where free users receive too few tokens and customers limit AI features out of cost anxiety.

Finally, Movva recalls influential mentors and professors. When he wanted only to build AI chips and asked why foundational subjects mattered, one professor told him to aim to understand the entire stack, from silicon gates to internet-scale services.

"He showed me the depth and beauty of understanding every piece of the puzzle rather than only one piece of the system."

That advice directly shapes his goal: not optimizing one chip, model, or data center, but treating the whole AI supply chain from silicon to power as one system.

Conclusion

Movva's vision is that the race to make AI overwhelmingly cheap and abundant matters as much as the race to make it smarter. The answer is not merely securing more cutting-edge GPUs. It combines undervalued chips, idle compute, small data centers, intermittent solar and wind, and long-running agents into new economics.

The message is simple: for AI to spread throughout society, intelligence must become not a rare, expensive consulting service, but a resource available at any time, like electricity.