Razor's Edge: Rogue Agents & Jevon's Router Jubilation

Agent's Gone Wild Great For Governance

This post caused a ruckus over the weekend...

It's a fun read and made me realize the agent dream gonna have some challenges. Agents are marketed like perfect obedient human employees who just do endless work which is why I I imagine some folks would see this incident this way. But its becoming obvious to me that these task minions have huge compliance limitation that hasn't been explained clearly as RL chases task compleetion capability at all costs. Now I think its gonna become pretty obvious to a lot of folks. Like chat llm drift was 'hallucination' and really not biggest deal because human is observing all times but w outcome based agents doing unmonitored work the 'rogue' element clearly unavoidable and problematic part of architecture.

The training process rewards task completion, and the model learns whatever policy reaches the reward. This is what makes them so appealing and useful. An agent that only does what you literally specified is just a great script, and while handing script writing ability to all is great it's not the agentic unlock. The 'AI dream' value of an agent is exactly its willingness to find paths/solutions you didn't specify. And scaling is about capability which really goes hand and hand with more persistence. So, the gap between obedience and capability is just gonna widen with compute/frontier improvement. That's a problem, and AI isn't gonna fix it.

As far as I'm concerned, this gap cannot be closed by better models anytime soon which is not what you want hear for agent adoption. A smarter model is at this point just a more capable hacker, which means whatever soft constraints you wrote into its prompt gonna decay even faster in value. So, capability gains make the governance problem WORSE and NOT BETTER.

At this point, you could maybe argue for Robocop esque directives which really just capability reduction or monitoring agents etc, but I just don't think that is gonna fly for enterprises. The safeguards that should be implemented must be deterministic, external, and architectural. Basically, boring ass software in complete human hands needs to handle this. This is good news for Servicenow, Okta, Gitlab( ultimate upgrades not DAP), Cloudflare, Rubrik and several others. Written about this before, but I think clearer now more than ever that theme stays strong. I bought some Rubrik today as rewind narrative around agents resonates the most with me as far as allowing the capability to work without nerfing/blocking with too strict rails.

Jevon's Jubilation

This post is really bad. I wouldn't care if it hadn't gone so viral as some sort of bullish llm demand read. So, Luna price cut 90% triggered 13.8x token consumption for it. What the data shows is it largely took share from open-source models in same price band. Token demand on the platform over same 3 week window roughly in-line w organic Openrouter growth. So against baseline, the discount program's net effect on total token consumption is approximately zero.

Which means the tweet thread got the finding exactly backwards. The experiment they ran is actually a decent test of short-term induced demand, and the conclusion wasn't bullish. A 90% price cut on frontier model produced no detectable expansion of the aggregate pool, only very predictable share shift within it. And the post promo volume holding above program pace is contaminated by Sol's discount starting Aug 17. And if that to me didn't look great with 2/3 loss immediately and them qualifiyng what retained as whale's which makes you just think distillation.

To be clear I'm not arguing a bear case either because maybe there is not much to read here other then the customers who could respond can't because they already have Openai entp contracts and subbing what is running on open source isn't worth it. Or maybe the routing ecosystem can't really respond yet to something like this. Who knows...

But it sure isn't bullish data related to compute/datacenter capex and it's definitely not stupid Jevon's. I think people need to realize most of the history of tech where per unit cost collapses is not aggregate $ spend mega bull land....

Storage: $/GB fell about a millionfold, consumption of bytes exploded, and yawn for a very long time...

Bandwidth: cost per bit collapsed 99.9%, traffic grew relentlessly, and telecom revenue and capex have been flat-to-down in real terms for two decades

Lighting: consumption up four orders of magnitude over two centuries, share of income spent on light down 100x.....duhh..

Server market 2000–2017: FLOPs delivered grew maybe a thousandfold, server spend basically sideways.

AI obviously very different right now because of the workloads, and that's really what will determine things from here.

So, pretty please, with cherries on top...No more Jevon's Paradox!