Back to Podcast Digest
Theo - t3.gg24m

Oh no (the new Grok model is good)

TL;DR

  • Dramatic price advantage: At $2/million input and $6/million output tokens, Grok 4.5 costs 5-10x less than Fable while delivering comparable coding performance.

  • Exceptional token efficiency: The model uses only 2 million tokens per coding task versus 7.2 million for Fable, making it both faster and cheaper in practice.

  • Transparent about contamination: xAI disclosed that Cursor's codebase was accidentally in training data, tainting one benchmark but showing rare honesty.

  • Surprising 3D capabilities: Outperformed all competitors including Fable at 3D modeling in Three.js, a first for any model Theo has tested.

  • Missing orchestration skills: Cannot delegate to sub-agents or coordinate complex multi-step work like next-gen models Fable and GPT-5.6.

  • Massive competitive leap: xAI went from irrelevant to competitive in one release, the biggest jump Theo has seen from any lab.

The Breakdown

A Model That Caught Him Off Guard

Theo opens with genuine surprise. He had been testing a mystery model in Cursor for 24 hours, assuming it was a new Composer release, only to learn it was Grok 4.5. The benchmarks back up his experience. On the Artificial Analysis code index, Grok 4.5 sits neck-and-neck with GPT-5.5 and just below Fable while beating Opus 4.8, a massive jump for a lab that was previously an afterthought.

The Training Story and Cursor Partnership

Grok 4.5 is a 1.5 trillion parameter mixture-of-experts model, triple the size of previous Grok iterations. xAI trained it on tens of thousands of GB300 GPUs with heavy investment in data filtering and curation. The RL training covered hundreds of thousands of multi-step software engineering tasks. Cursor, now partnered with xAI, contributed trillions of tokens of interaction data showing how developers actually work with codebases.

The Contamination Admission

One benchmark is conspicuously missing from the marketing. Cursor Bench showed Grok 4.5 beating Fable 5 High at a fraction of the cost, but there was a problem. An earlier snapshot of Cursor's actual codebase was unintentionally included in the training data. xAI disclosed this openly and removed the data from future models. Theo appreciates the transparency, noting other labs might have buried this detail.

Pricing That Undercuts Everyone

The pricing is aggressive. Grok 4.5 costs $2 per million input tokens and $6 per million output tokens, compared to Fable at $10 and $50 respectively. Under 200K context, it is 5-10x cheaper. Above 200K up to 500K, pricing doubles, which Theo finds slightly greedy but understandable given GPU resale economics. The model is remarkably token-efficient, using only 2 million tokens per coding task versus 7.2 million for Fable.

Real Work on Lake Bed

Theo tested Grok 4.5 on hardening his Lake Bed project for release. He threw messy, multi-part requests at it, referencing different numbered lists, asking for multiple PRs simultaneously, and even pasting screenshots for fixes. The model handled all of it without getting confused or fixated on old context, something even GPT-5.5 struggles with. It felt pleasant and fast, though it did not catch everything Fable would.

Unexpected 3D Prowess

The surprise standout capability is 3D. Theo asked it to convert a 2D fish game into 3D using Three.js, and it modeled creatures, geometry, and a full environment. The results were not production-quality, but they crushed every other model he has tested, including Fable and GPT-5.6 with access to the same tools. He calls it the first model to be almost decent at 3D modeling in game engines.

The Generation Gap

Despite the impressive performance, Theo draws a distinction between model generations. Fable and GPT-5.6 can orchestrate sub-agents, delegate work, and tackle complex multi-step tasks. Grok 4.5 struggles there. His analogy: xAI just released the best PS2 game ever, but the PS3 has been out for two months. It is a phenomenal achievement for the previous generation, but not quite the new tier.

Was This Useful?

Share