13 min read

Razor's Edge: Schrödinger’s AI Compute Box Dilemma

Razor's Edge: Schrödinger’s AI Compute Box Dilemma

Over my 30 or so years in markets, I've learned one basic thing about supply debates and that's that: extreme shortage and extreme oversupply can and often do exist at essentially the same moment in time. What separates them isn't necessarily the underlying fundamental conditions of whatever market (oil/housing/bandwidth/shipping/potash) is in focus, but the reality the observer is able to see at a specific moment in time. It's a lot like superposition in physics, where multiple possible states coexist until a measurement resolves the outcome. (yes, as you can see from the image, Apple's Dark Matter is making an impression on me these days)

AI compute may be the most fascinating example of this we have ever come across in markets. We have an undeniable shortage of compute today, driven in no small part by a handful of super resource rich tech companies locked in a winner take most race to have the most capable AI models that they can monetize across all forms of economic useful output. None of these companies can afford to stop investing for fear of losing, even though collectively they may be building far more capacity than the eventual market can economically support in any productive basis. The scarcity and oversupply can exist simultaneusly and the prisoner's dilemma is what keeps both realities alive.

Are We Already In A Compute Glut?

Nvidia and Broadcom, the two biggest suppliers of AI chips, have delivered robust rev growth and ever rising fwd forecasts over the past 12 months. Yet Nvidia's stock is roughly flat over the past 52 weeks, while Broadcom is down more than 12%.This is happening while seemingly every notable big tech CEO/Founder is insisting that compute shortages are going to persist indefinitely.

You have companies announcing hundreds of billions of dollars in infrastructure spending, GPU rental rates that supposedly signal years of scarcity, and executives/hedge fund managers talking about gigawatts of compute as though they were oil reserves. And yet the equity market has become noticeably less enthusiastic about the two giants supplying much of the hardware behind this expansion.

As you can see from the the image, this note is inspired somewhat by the Apple TV series Dark Matter, where a mysterious box allows its occupants to enter a state of quantum superposition, with different possible realities existing until one is experienced. That to me feels like a pretty good metaphor for the current AI infra market.

Behind one door is a world of permanent compute scarcity, rising rental rates, Elon SpaceX maxi moments, extraordinary returns on capital at scale, and seemingly (inserting worst phrase ever) infinite demand for intelligence.

Behind the other door, depending on how you manage your emotions, is a world of collapsing compute prices, very bad infra capital returns, and abundant knowledge work capabilities for all mankind.

The Dark Matter part is that both realities emerge from exactly the same starting conditions and thus the forces producing today's shortages are the same forces creating tomorrow's oversupply.

Shaun Maguire and the $40 billion gigawatt

This phenomena was in focus over the weekend thanks to Sequoia's Shaun Maguire, who published a bottoms up financial model of SpaceX through 2028.

He projects SpaceX revenue rising from approximately $51 billion in 2026 to $274 billion in 2027 and $731 billion in 2028. The growth largely comes from AI compute infra.

His assumptions for terrestrial compute look roughly like this:

These are Maguire's modeling assumptions not spacex explicit guide, but u don't need even a rocket scientist probably can't distinguish the difference these days. His model also includes orbital compute, which rises to approximately 2.8 GW by the end of 2028, and which Elon says will exceed terrestrial capacity additions in short-order. Maguire immediately got his fair share of criticism on the model and his intentions, but the key focus for me was the almost mathmatical constant of rev per gw as SpaceX gw of compute grow exponentially in an incredibly short period time. He also assumes that new gigawatts of infrastructure can eventually be built for around $30 billion each.

Anyway, if you can build something for $30 billion and generate $40 billion of annual revenue from it and scale that to a trillion in 12 months, the ROIC is obviously extraordinary. Elon, as u wud naturally expect, chimed in that SpaceX could do even better!

But this is where I think some actual real investing debate needs to happen.

Like what happens to the price of compute when everyone is building the same thing at the same time, using hardware that becomes dramatically more productive with each generation?

Maguire also argued that compute pricing would behave more like oil production than the overbuilt broadband infrastructure of the dotcom era. I think that analogy finally deserves some serious scrutiny.

Compute isn't a good oil or any traditional physical commodity comp

The compute market has recently been described as being in contango with forward GPU rental rates exceeding spot rates. This shit will give you a bit of a headache because the key shortage focus before was that it was in backwardization ie compute today being more expensive then compute tmrw. But now some investors interpret the contango curve as evidence that future compute shortages will be even more severe than today's and thus a more bullish demand signal.

So yea, at this point the physical oil trader terminology really has become nothing but useless noise.

Why?

Beacuase a barrel of oil delivered in 2027 is essentially the same physical commodity as a barrel delivered today.

A Gpu/Xpu hour delivered in 2027 will represent a very different amount of computing capability. Generally the useful computing output per chip is expected to go up, and that's a flops measure and u can see the numbers quickly by just asking supreme to do the work..

  • Blackwell (B200): Delivers roughly 18 PFLOPS of FP4 inference compute at a Thermal Design Power (TDP) of approximately 1,000 Watts.
    • Efficiency: ~18 TFLOPS per Watt [1, 2]
  • Vera Rubin: Ramps raw compute up to 50 PFLOPS of NVFP4 inference compute while pulling roughly 2,000 Watts of chip-level TDP.
    • Efficiency: ~25 TFLOPS per Watt [1, 2, 3]

The Raw Metric Multiplier: At the individual silicon tier, the shift to native 4-bit floating-point execution and TSMC's advanced lithography nets an immediate ~1.4x improvement in raw FLOPS per watt. [1, 2]

But system-level throughput per watt is the real gain....

A raw silicon calculation does not account for data center bottlenecks. When individual chips wait for data to transfer from memory or network layers, they burn "idle power," driving down real efficiency. [1]At the system scale (comparing a Vera Rubin NVL72 rack to a Blackwell GB200 NVL72 rack), the platform achieves a 5.4x to 10x improvement in real-world performance per watt due to major sub-system overhauls

And oh yeah, this differs from training to inference and by workloads so there will be caveats, but you get the problem with respect to oil. When an automobile engine becomes more efficient it requires less oil to travel the same distance. The efficiency improvement typically occurs downstream of the commodity, and in this case that would be model/soft/architecutre level stuff. A barrel though remains a barrel. Now the ability to get more oil out of the ground per $ of capex is what hydraulic fracking gave you, and generally the tech itself is now just viewed as a surplus gain for all.

In the land of compute, efficiency improvements occur at several different levels and compound rapidly. A system delivering four times as much computation per watt, combined with software that requires half as much computation per task, could theoretically deliver eight times as many tasks per watt.

Basically, the productive output of a gigawatt of compute is not a mathematical constant.

In fact, Jensen would probably tell Maguire rev per gw should always go up for NVDA powered DC. But let's not go down that rabbit now. What I think matters is that its pretty obvious that the relationship between arithmetic chip operations and useful model capability are also no constant.

People need to ask what exactly are we pricing really?

A GPU-hour? Cost per million tokens? A arbitrary completed coding task? An autonomous agent successfully completing a workflow in how many steps? An employee wage/hour equivalent?

You have seen many smart folks make a mess of this lately with their Ant/OpenAi token vs spend color and their good old fashoned Jevon's tweets.

The oil analogy becomes even more annoying when someone argues that improvements in useful intelligence per watt automatically justify higher revenue per watt. Yes, they might justify higher value to ur end customer.....

But value and price are not the same thing.

And if multiple suppliers can deliver the same useful capability at a fraction of the previous cost, competition can transfer much of that efficiency gain to customers. The fact that compute becomes more valuable technologically does not mean the owner of the hardware captures that value in exponential ROIC.

And this brings us to the supply numbers at the heart of all of this.

The semiconductor industry may end up increasing "compute" supply nearly tenfold in two years....

Rather than debating generic gigawatts as if they perfectly fungible barrels of oil, I think we need to try and understand how much actual computing capability the semiconductor industry is producing.

I had the Supreme Intelligence tackle this...

Epoch AI has assembled estimates of AI accelerator production covering Nvidia, AMD, Google's TPUs, Amazon's Trainium and Inferentia, and major Chinese suppliers.

The data normalize the hardware into H100-equivalent computing capability using peak dense 8-bit arithmetic performance.

It isn't a perfect measure of useful compute, but it provides a consistent starting point for comparing very different architectures.

Here is the approximate progression:

Global AI compute supply20252026 estimate2027 scenario
New H100-equivalents produced13.6M26–31M90–140M
Cumulative stock~20M~46–51M~100–190M

The 2025 numbers are historical estimates. The 2026 figures extrapolate observed first-half production. The 2027 figures are scenarios based on semiconductor shipment forecasts and assumptions about chip performance and deployment.

The distinction matters. These are not all equally reliable observations, and cumulative chip sales are not necessarily the same as energized, operational compute.

But the direction is extraordinary.

The industry is already on course to manufacture roughly twice as much new computing capability in 2026 as it did in 2025.

And 2027 could be another enormous step higher.

DIGITIMES Research forecasts approximately 9.1 million high-end Nvidia accelerators shipping in 2027, with Rubin representing the overwhelming majority.

Using a more conservative 80% Rubin mix, that could represent approximately 68 million H100-equivalents of computing capability from Nvidia alone.

AMD's MI400 family could contribute another 10–13 million under an illustrative production scenario.

That's nearly 80 million H100-equivalents from Nvidia and AMD before counting Google, Amazon or other custom ASIC suppliers.

And ASICs have become a very big deal.

Google was already the second-largest source of newly sold AI computing capability in Epoch's 2025 estimates. Amazon continues expanding Trainium, while DIGITIMES forecasts approximately 15 million custom ASIC shipments across the industry in 2027.

Physical ASIC counts cannot simply be added to GPU counts because performance varies significantly across architectures. But the contribution is substantial.

Under a central scenario of approximately 110 million H100-equivalents of new compute manufactured and deployed in 2027, the global cumulative stock could approach 160 million.

That's approximately eight times the estimated end-2025 stock.

A more aggressive production and ASIC-performance scenario could approach ten times.

In other words, if the semiconductor production forecasts are achievable and the power infrastructure can absorb them, the world could end 2027 with roughly 8–10 times the peak arithmetic computing capability it had at the end of 2025.

That is a remarkable supply expansion to contemplate in an industry whose investors are increasingly discussing permanent scarcity.

And remember this is hardware driven perf calc. It doesn't include measuring software efficiency improvements which will increase useful output even faster for some workloads.

SpaceX's 10 GW demonstrates both sides of the paradox

Supreme walking through the H100 equivalent math is good contextual grounding for the SpaceX debate. Musk has discussed adding approximately 10 GW of Nvidia-powered compute during 2027. Against Nvidia's projected 2027 production of approximately 9 million high-end accelerators, SpaceX alone could absorb roughly a quarter to nearly half of annual output. That some serious demand, and if you H100 equivalent it against what all the big 6 have today; it would be like SpaceX adding anywhere between 40%-100% (depending on the reuben mix) of their combined 2026 AI compute capacity by itself. The oil market reset viciously in 2015 with a mild 2% supply/demand overshoot from shale. Here SpaceX is gonna deliver anywhere between one and three 2026 OPEC's on its own by the end of 2027.

Now, the Oil price collapse happened because US shale was like 65% of supply growth and oil needs to be stored and has a delivery curve. I just established compute isn't as clean as a compare and the Ai demand curve elasticity has its own superior argument versus mature oil, but you get my drift. It's potentially a lot of supply to add in a very short time no matter how bullish u are demand.

And consider the other side.

If Nvidia by itself can manufacture GPU's with compute equivalent of multiples of 2025 and 2026 output, the industry needs somewhere in the neighborhood of 20–40 GW of Datacenter's to absorb that production. AMD and the custom ASIC suppliers would require additional capacity. Maybe you undershoot bigly on power, but that does not eliminate the underlying increase in computing capability being manufactured. In fact, it can temporarily conceal it.

So, contrary to popular beleif, a shortage of energized datacenter capacity can coexist with an enormous increase in the productive capability of the chips waiting to be deployed.

The industry can be genuinely short of usable compute today while simultaneously creating the conditions for massive future excess supply. I.e. rental rates and component supplier expectations can diverge. Basically, which seems to be a struggle for some to process, neither observation contradicts the other.

Now, this is what I would call accepting the big picture element of the AI infra mkt.

I think there is a more in the weeds area to turn one's attention to today...

How much compute is being consumed by duplicative training?

This is the part of the Ai infra debate that I think receives far too little attention. Investors spend enormous amounts of time discussing token demand, GPU rental rates, Jevon's paradox, intelligence, inference efficiency and the economics of frontier models vs open source.

But honestly little time is spent on analyzing how much of the industry's existing compute consumption is actually dedicated to serving end user demand and how much is being consumed by multiple companies independently training frontier models that increasingly offer similar capabilities a few weeks later?

OpenAI and Anthropic are major sources of commercial frontier model demand. Nobody is going to dispute that. Meanwhile, Google, Meta, SpaceX and other companies continue investing substantial amounts of capital developing competing frontier capabilities. Google has been debated ad nauseum on this front as a company defending a multi trillion dollar franchise shud rationally spend considerably more on model development than its standalone end model revenue would justify. But that doesn't make the economic distinction disappear or potentially how much worse it might be getting when combined with peers. Companies with relatively little commercial serving share may still be consuming extraordinary amounts of compute trying to catch up with the leaders.

Training compute is consumed because a company has decided to pursue a technological objective whose measured progress is inference serving compute rising susbtantially. That is the general premise. Both can create value. Both can be subsidized. But they have very different economic drivers. The training race depends on the perceived future value of competitive advantage. Serving demand depends much more directly on actual adoption and usage. The industry currently provides little transparency into how this will evolve at all.

Current winners in training obviously end up with substantial and very profitable inference demand, but at what point does winning get defined?

I would imagine that is when 3-4 other players stop chasing frontier model training and when the capability saturation of tasks translates into far less training for those with the current dominate serving positions. But for 3-6 to quit the outcome can't be total seccession to the leaders. You would need some sort of stalemate that resolves this in a manner which licensing models reduces some players costs training costs to zero and the winners costs substantially or some sort of public diffusion. And when you consider how much compute is going to training right now, figuring this out matters enormously for anyone discussing supply/demand balances in compute infra.

Like u can easily argue there could be substantial shadow supply hiding inside massive duplicative training clusters. Like if the current split is 60/40 training/inference, which tbh I have no clue on, but let's just go with that based on research to date. Now, let's assume frontier model economics begin to rationalize some because of task capability saturation for avg work and feature parity across top providers. And let's say one big player just decides to exit chasing the frontier and partners. This could reduce training demand notably in a very short time and imbalance the market on its own.

I asked Supreme to give me a prospective demand analysis of a training to inference unlock of note..

Suppose the industry begins with 100 units of computing capability at the end of 2025.

By the end of 2027, it potentially has 1,000 units.

Assume inference initially accounts for 40% of compute consumption.

Now imagine training's share declines as the frontier race rationalizes, leaving inference responsible for 70% of total consumption.

Inference compute demand would need to increase from 40 units to 700 units.

That's 17.5 times as much inference compute consumption in two years to maintain the assumed workload mix and utilization.

And if software efficiency doubles the amount of useful output delivered per unit of inference compute, the corresponding useful-output requirement could be approximately 35 times the original level.

These are not forecasts. They are sensitivities illustrating the scale of the absorption problem.

Which brings us back to Jason Dessen's box..

Schrödinger's Compute Dilemma

Behind one door is a world where AI demand expands so rapidly that even an extraordinary increase in computing capability cannot satisfy it.

Training expenditures remain elevated. Inference demand grows explosively. New applications emerge. Power constraints keep utilization high. Compute rental rates remain strong. You cure cancer and cell aging and hunger and work all at once.(we can just be optimists assume that such changes will be chaos free for society) Everyone is both rich and satisfied at the same time and GTA6 and Memefluencers are the new output measures.

Behind the other is a world where the same enormous investment cycle produces a dramatic increase in computing capability just as software efficiency improves, model differentiation narrows and some duplicative training programs become totally unnecessary. Inference demand grows rapidly, but not rapidly enough to absorb the combined increase in hardware productivity and available capacity. You pay less for professional services of all kinds but you also have all kinds of new economic challenges. Universal high income starts to feel more like a road to serfdom for JD/MBA's.

Both worlds are consistent with today's shortages.

Both are consistent with enormous capital expenditure commitments.

Both are consistent with the belief that AI will become much more important to the global economy.

The difference is perspective at a point in time...