AI 2026 · Part 2, Models
AI 2026
Read blogPart 2: Models
This is Part II in a three-part series looking at Chips, Models, and Agents.
Today is Day 1 of the singularity.
Call it what you will.
Singularity.
Intelligence explosion.
Cognitive revolution.
Maybe we’re building a god.
Let’s understand these Gods.
In this piece, I’ll show:
- Why pre-training compute gives AI labs terrible margins
- Why RL compute is the current frontier
- How Chinese algorithms are commoditizing models
- How AI is creating a $10T parallel digital workforce
1. Operating Margins of AI Labs
They’re the fastest-growing companies we’ve ever seen.
OAI and Ant aren't quite to Nvidia's size yet.
More importantly, Nvidia has ridiculously good margins.
For every $100B in revenue, Nvidia spends $25B on hardware (COGS) and $7B on salaries & opex.
That leaves $68B in operating profit.
Nvidia is famous for employing 5x fewer employees than the rest of Big Tech.
Only 45,000 people!
Anthropic’s estimated CY2026 revenue is $51B.
It costs them $12B to serve inference. Similar to Nvidia's cost for chips.They spend $7B on salaries & opex. Not all that different from Nvidia.
The biggest difference is that they spend $30B on compute for training R&D.
That leaves $2B in adjusted profit on $51B of revenue — a 4% margin.
OpenAI’s estimated CY2026 revenue is $29B.
Compared to Anthropic, they spend more on salary to make less revenue.
They spend nearly twice as much on inference ($20B) because they have 1B weekly users, many of whom are on a free tier.
And they still spend $32B on training compute R&D.
That leaves a $33B adjusted loss, a -114% margin. They're losing money while Anthropic makes money.
Compute is a massive bill for both OAI and Ant.
For OAI, we estimate $50B in 2026. Even back in 2024, it was pricey — they spent over $7B on compute.
They want commoditized compute to help with their margins.
Both for training,
inference,
for solving Millennium Prize problems,
Yes, this result cost millions of dollars.
— Noam Brown (@polynoamial) September 8, 2026
But remember that when @OpenAI announced o3 it cost ~$500,000 to score 87.5% on ARC-AGI 1. Today, Astra scores higher for ~$20.
In 2025 it took us and GDM an enormous amount of compute to achieve IMO gold. For the 2026 IMO, anyone with… https://t.co/iT2O8YRo8L
and the rest of math while we're at it.
How can a lab spend billions on training compute?
How does this make the models smarter?
Most importantly, how has training changed?
2. Long-horizon RL is the current frontier
Intelligence is the result of three inputs: compute, data, and algorithms. All of these are beginning to change. Let’s look at the progression.
The first LLMs used the most easily accessible data, human internet slop. I love Karpathy’s framing here: early models compressed the 100 TB internet into 100 GB of parameters. Learning is compression.
After pretraining, LLMs are a lossy JPG of the internet; a document creator doing next-token prediction. After some chat RLHF, it nudges them to become good at chatting. And voilà, ChatGPT.
It took work to ingest the internet, but models were still dumb. Or at least their intelligence was spiky:
MMLU was the first big benchmark. It’s a hard test. Specific domain experts only get 90% in their target field. 175B param GPT-3 got 40%. 1.8T param MoE GPT-4 got 80%. It’s now been completely saturated.
GPQA (PhD-level questions) quickly became completely saturated too. Humanity’s Last Exam is the current frontier.
To beat all of these benchmarks, we needed a new training paradigm, Chain of Thought. Thinking step by step. I love Bob McGrew’s framing here:
“GPT-3, GPT-4, and GPT-5 each used 100x more compute for training.
o1 is packing 10,000x more compute into an answer.”
Fundamentally, o1 didn’t need new data from humans. No new internet-sized reasoning library. Instead, o1 needed PhD’s (from Mercor et al) to create difficult tasks and judging criteria. It needed verifiable rewards in domains like math and coding (RLVR).
Then o1 sculpts its own reasoning library through millions of reasoning trajectories. And boom, the model learns to think better thoughts. RL is bitter lesson pilled.
But we don’t just want the AIs to reason. It’s not enough to solve math problems.
We want AIs to do useful work. To be agents. To add to GDP.
Mostly this means being able to do lengthy tasks. Not just chat. Not just reason. But creatively pursue a goal.
This is the cutting edge of data. Agentic trajectories, not just reasoning trajectories. More than anything, agentic trajectories need an agentic harness. They need a while loop:
while (task != complete):
use tools to complete taskWith MCP, grep (read), and str_replace_editor (write), LLMs can start to write arbitrary code.
Give them a gym of tens of thousands of Mac Minis, and they can use any arbitrary computer application.
These agentic RL trajectories have shifted the data frontier from learning the human corpus to learning model trajectories.
Which has shifted the compute spend of training large models:
I love how Jie Tang of Z.ai talks about it in his post "Thoughts About Scaling Law."
He shows how we’ve moved from scaling pretraining compute to RL compute:
Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training.
GLM-5.3 is our controlled experiment on that claim. Same base, same architecture, same total and activated parameters as GLM-5.2.
One month of scaling long-horizon environments and RL. The gains are not marginal. Well, scaling has more than one dial. We turned the post-training one this time because it had the most slack left in it — not because the others are finished.
You can think of RL as another dimension for scaling laws.
Thus far we usually just show FLOPs and model parameters.
Here’s the famous Chinchilla unhobbling that showed it’s better to pack more compute into fewer parameters.
We pumped up the FLOPs to 1e27 and parameters to over 1T.
Now we're scaling RL. Each bubble size is number of RL rollout tokens.
You can see GLM 5.3 gets bigger in terms of RL, not parameters!
Or you can view it as trajectory depth.
These days you’ll have 100 average tool steps in the RL training curriculum. While GPT-3 had ~none.
This is completely shifts lab spend to RL instead of pretraining.
Share of frontier lab compute by workload
Coming back to the three legs of the intelligence stool: compute, data, and algorithms.
Compute trains models on long-horizon RL tasks.
Data creates synthetic gyms for RL rollouts.
New algorithms emerge for this RL paradigm.
One of the best new RL algorithms is MOPD, Multi-Teacher On-Policy Distillation.
In MOPD, generalist models learn from trained RL experts.
These MOPD-trained agentic RL models are increasingly used to train themselves.
Claude now leads 26% of model R&D work at Anthropic.
Claude’s share of model R&D work
The median researcher at OpenAI spends over $200,000 in tokens per year.
Median OpenAI researcher · annualized token usage
Models training themselves is called recursive self-improvement, or RSI.
Right now 30% of training tasks are automated.
Based on these trends, AI will have automated model training by 2030 at the latest.
Vals RSI Index
Better models write better code.
Better code trains better models.
Nothing to see here…
We’re just getting started with RL as a training class.
RLHF created chat. RLVR created reasoning and code.
A new model, Jev, was trained on RLCD, Reinforcement Learning from Calibrated Decisions.
RLCD produces machine-readable structured outputs at 100x the speed. They’re starting with booleans (smart if statements) and will add other outputs like lists.
Jev demo · Karo
3. How China is commoditizing models
For a while now, Chinese models have lagged behind US models by 7 months:
Of course, OpenRouter doesn't process all tokens. It’s likely less than 10%. They’re in the tens of trillions, while OAI and Ant are in the hundreds of trillions.
But this is impressive. Especially because DeepSeek, Kimi, and GLM were trained on only 1e25 FLOPs, while US foundation models use 1e26 or 1e27.
By building in the open, Chinese models can learn more from each other:
I’m not sure if open-weight models will impact closed-weight models’ ARR. I think Ant & OAI have super strong flywheels of data, talent, and cost of capital. Perhaps most importantly, they have a great brand. Aggregation Theory still matters, even in the age of AI.
4. RL & the $10T labor market
Reinforcement learning has turned models from geniuses in a data center to agents on a mission.
We’re moving from tokens to tasks.
We’re creating a parallel digital workforce.
The size of the global workforce today is 3.75B workers making $66T, about half of global GDP.
How much will the AI workforce contribute?
Model companies already make more than $100B of ARR.
SaaS is $500B.
In the long-run, AI could outproduce the $66T from humans. (75% of Google’s code is already written by AI.)
For now, let’s call it $10T, or 1B humans of labor. How close are we?
In terms of salary, agents are making over $100B.
But what if we think in terms of labor, or human-equivalent AI workers?
We estimate 3.7M chatbot workers today.
2 million researchers.
Over a million other knowledge workers.
And over 4 million coders.
AI coding output is equivalent to roughly 11% of the 40M human developers worldwide.
These 10M AI workers are 1/6th of the 62M human knowledge workers in the US.
Driving and robotics lag far behind.
In summary:
- Compute gives AI labs terrible margins
- RL is a new paradigm for compute, data, and algorithms
- Chinese labs are competing to commoditize models
- In the long run, RL helps agents tackle a $10T labor market
But idk. Those are all specifics. Zoom out. The Earth is waking up. The Digicene is coming. Today is Day 1 of the Singularity.
The Earth is waking up.
Thanks for reading. Curious for your thoughts.
- Rhys
Looking for my next gig btw. Here’s how I can help. Would love to chat!
Subscribe for Part 3: