2026 in a nutshell:
I skip checking X for one day. The next day, I open it: “Whoa, what just happened? The world has totally changed.”
Given how quickly AI infrastructure is changing, here is my attempt to make sense of what is happening and where the market may be going next.
My core thesis is simple:
AI infrastructure is unbundling at every layer. The future winners will make that choice operable.
From one bundled decision to four independent choices
Last year, choosing an AI provider meant choosing the entire stack.
For a while, I thought: OpenAI has the best models. Why would you use anything else? If frontier intelligence produces materially better results, of course you would want the frontier model.
Later, I felt the same way about Anthropic. Claude Code was a compelling gateway drug into the Claude ecosystem. Once it pulled you in, why wouldn’t you keep adopting more of the surrounding products and workflows?
But in 2026, we are decidedly in a new world.
Every layer of the AI stack now has a growing number of credible choices. What used to be one bundled decision is becoming four independent choices:
- Model: Which capability do you want?
- Harness: Which agent loop should translate that capability into action?
- Inference: How should the model run?
- Compute: Where should it run?
This is not simply a shift from closed models to open models. It is a shift from one integrated product to an architecture designed around substitution. No layer looks winner-take-all.
Model choice is real
At the model layer, there are more and more genuinely capable options.
A few years ago, open-weight models trailed frontier systems by a wide margin. Today, that gap is measured in months rather than years. In cybersecurity, the UK AI Security Institute estimates that the gap between leading open and closed models has narrowed to one to two quarters and is closing rapidly (UK AI Security Institute). We see a similar story in other domains, too.
The shift also shows up in usage.
Chinese open-weight models have grown from a tiny fraction of token usage on OpenRouter a year ago to more than 60% today (Associated Press). Of course, OpenRouter does not represent the entire AI market. A company buying inference directly from a provider, for example, does not appear in that data. But OpenRouter represents a meaningful slice of developers who value the ability to compare and switch between providers. This momentum signals a step change in the quality and maturity of open-weight models for many use cases.
Yet, no single Chinese model has permanently won. The mix rotates rapidly among DeepSeek, Xiaomi, Tencent, Z.ai, MiniMax, Moonshot, and others.
That is exactly what makes routing, evaluation, and portability valuable. When a credible new model can appear every few weeks, flexibility becomes part of the product. According to Ramp, 52% of organizations in its observed model-serving and inference category now use OpenRouter, up from 14% two years ago (Ramp). A neutral model gateway has become a valuable control point.
The major infrastructure platforms understand this. NVIDIA and Microsoft, among many others, are publicly supporting frontier open-weight models. That makes strategic sense: infrastructure companies benefit from a diverse ecosystem of model builders and consumers rather than a world with only two main customers.
The model is only part of the agent
The second layer is the harness: the software infrastructure and runtime environment that enable a frontier model to act as an agent. Identity, memory, tools, persistence, and other components turn a real task into something the model can execute.
The easiest way to understand the importance of this layer is to hold the model constant. Composio ran the same Kimi K3 model through eight harnesses using the same provider, reasoning level, tools, task instructions, connected data, and scoring rules. Pass rates ranged from 68% to 88%. The model stayed the same; the harness created a 20-point spread (Composio).
This remarkable result suggests a useful framing:
The model determines the capability ceiling. The harness determines how much of that capability reaches the task.
There are two schools of thought about where this goes.
One says the model wins. Make the harness thin. Bake more planning, reasoning, and tool use directly into the weights. The model is the magic.
The other says the harness wins. Models become increasingly substitutable, while context, permissions, tools, memory, workflow state, and proprietary feedback remain an independent product layer.
Honestly, I think the jury is still out.
The model and harness are increasingly co-designed
The evidence increasingly points toward the harness and model evolving together. Better harnesses create better trajectories and feedback. Those trajectories help train specialized models. Stronger models then allow parts of the harness to simplify.
The best agent companies own the learning loop between the two.
Harvey is a good example. It introduced Tenet, a model based on Kimi K3 and post-trained for long-horizon legal work. Tenet completed almost twice as many held-out Legal Agent Benchmark tasks as the underlying model and 20% more tasks on the contracts component (Harvey).
Harvey shows the power of model selection, domain-specific environments, post-training, evaluation, and harness design working together.
We see similar ideas in coding and application development. Cursor, Cognition, and Vercel are building specialized models alongside product environments tailored to the work those models need to perform.
This creates two broad ways to win in the harness layer.
The first is to own the finished agent: build the product experience, own the workflow, and tune the intelligence to a particular domain.
The second is to enable agent builders: provide training infrastructure, evaluation environments, reinforcement-learning loops, and open harnesses that let other companies build specialized agents.
Inference and compute are becoming heterogeneous
Inference is not one workload.
Prefill and decode have different characteristics. Interactive agents, batch jobs, and multimodal applications have different latency and throughput requirements. Sensitive workloads may need to stay inside a customer-controlled environment. Others may be best served in a provider’s cloud or even on a local device.
The optimal setup may change across phases of a single request.
That is producing a heterogeneous market:
- Neutral serving and routing platforms
- GPU clouds and neoclouds
- Alternative chips and accelerators
- On-premises and customer-cloud deployments
- Local and edge inference
It is a pretty wild time in the compute market. Startups are making enormous capacity commitments on the assumption that all of this infrastructure will eventually find a buyer.
I hope all those bets work out. Regardless, the structure of every layer has fundamentally changed, and there is no turning back.
Choice creates new control points
The most interesting implication of this unbundling is that choice creates new control points and therefore new winners. Throughout the stack, choice in one layer can create an opportunity in another.
At the model layer, choice creates demand for a neutral gateway that can select, route, price, and fail over between models. Stripe is acquiring OpenRouter for $8B+.
At the harness layer, specialization creates demand for infrastructure connecting environments, rollouts, training, weight synchronization, and serving. Fireworks says more than 95% of the tokens it serves now come from customer-specialized models (Fireworks). It raised funding at a $17.5B valuation in July 2026.
At the inference layer, heterogeneous hardware creates demand for systems that can map each phase of a workload to the right silicon. Gimlet raised funding at a $3B valuation in September 2026.
This demand for architectural flexibility extends to deployment. Customers increasingly want to choose not only which model and harness they use, but where the resulting workload runs.
Does it run in the vendor’s cloud? My cloud account? An existing VPC? On-premises? In a restricted environment? On a local machine?
At the compute layer, those choices create demand for bring-your-own-cloud (BYOC) control planes that can deploy, secure, observe, and update software wherever a customer needs it to run. This is Alien.
Alien lets the customer retain control of the environment and sensitive data while the vendor continues to operate the product like managed software.
Unbundling does not eliminate platforms. It shifts value toward the neutral control planes that enable choice.
Making choice operable
There will not be one universally best model, best harness, or best place to run. All of this choice is good for builders. It gives us more ownership, more customization, better economics, and less lock-in. But unmanaged choice becomes operational complexity.
Someone still needs to evaluate the options, route the workload, preserve context, enforce policy, manage failures, track costs, decide where the data lives, and keep the software updated across environments. That is the infrastructure opportunity.
The future of AI is choice. The winners will make that choice operable.