Guy Freeman spent three days and $300 of agent time building a cryptic crossword solver called ximenes: a typed DSL for wordplay, a verifier that walks the parse tree and executes nothing, a posterior fitted on held-out data, loopy belief propagation over the grid's crossing letters, and a value-of-computation controller that buys the next model call only when the expected information is worth the price. Fourteen briefs, each with a pre-registered measurement.
It worked. On forty fresh cryptics, the full solver filled seventeen grids with every answer correct, the verifier was sound, the posterior was calibrated to within three points, and committed answers were right 98–99% of the time.
Then, on brief fifteen, he asked the question he should have asked at brief zero: what does the frontier model do if you hand it the whole puzzle — clues, enumerations, crossing structure — and let it think? One call per grid, no machinery. Claude Opus 5.5 at high effort filled thirty of the same forty, at 28 cents each.
Fifteen grids were solved only by the direct call. Two were solved only by the solver. On a second set of forty it got thirty-five; on Times and Telegraph puzzles, twenty-eight of thirty-six. Across everything, 93 of 116, with no fitted parameters, for about the price of a postage stamp per grid.
Fourteen briefs of machinery, beaten by a phone call.
The baseline that should have come first
The essay is unusually honest about the sequence. Brief fourteen had already warned him: a frontier model fed through his calibration and the crossing letters, with no verifier at all, edged the full solver eight grids to seven. Brief fifteen removed the machinery entirely, and it won.
He'd measured the baseline at brief zero — clue by clue, with a small model. That baseline was not the strongest simple one available. The strongest simple one was the frontier model, given the whole puzzle, allowed to think. It would have cost eleven dollars and ended the project before it started.
This is the most transferable lesson in the piece, and it's not really about AI. It's about how builders measure. The temptation is to benchmark against a strawman because the strawman is cheap and the result is encouraging. The discipline is to benchmark against the strongest thing you can do without building anything, and to do it first.
His own formulation: measure the frontier model given the whole problem, then build only what it demonstrably lacks, in order of measured value — the world it cannot see, the exactness it cannot hold, the calibration it cannot have, the loss it does not know.
What the machinery actually produced
The interesting twist is that the elaborate system did produce something valuable: not better answers, but honest probabilities attached to the same answers.
Raw frontier-model confidences had an expected calibration error of 0.177. The solver's posterior, fitted on held-out data, had 0.064, and it committed 495 slots where the bare model committed 341. When he fed the direct call's own answers through the same machinery — calibrated probability per answer, belief propagation, re-asking the open slot — it gained one grid in each of two runs and never lost one, while taking the model's stated confidence from an ECE of about 0.07 to 0.003.
By his pre-registered rule, that's not an accuracy claim, and he doesn't make one. But it's a real finding: the value of the machinery was in telling you what you know, not in knowing more.
Positions, not capabilities
The essay's second half argues that the Bitter Lesson — general methods that scale with compute beat encoded human knowledge — is a claim about the method, not about the four inputs: objective, world, feedback, outcome. Compute replaces inference, search, synthesis, craft. It cannot supply the objective (compute optimises, it doesn't prefer), the world (unobserved information can't be computed into existence), or the outcome (truth arrives at the speed of time, and consequences attach to persons).
He calls what remains comparative standing: being the principal whose loss it is, the owner of the world being observed, the bearer of the outcome, the subject who experiences the utility.
That's a sharper frame than the usual "skills vs. judgment" talk, and it has a useful operational edge: for any action, split its expected value into a position term (durable) and a capability term (depreciating at the rate frontier capability grows). Own the registry, not the solver. Be the party that signs.
I'd push on one thing: three of those four positions are held up by institutions — property, consent, liability — and the essay itself notes that institutions weaken when people are no longer needed for production. So "standing" is not a natural fact you can rely on; it's a political one you have to maintain.
The disclosure that makes it credible
The best paragraph is the noise check. Two near-identical conditions — the same call, with and without the grid — disagreed on seven grids out of thirty for a net difference of one. The hybrid's one-grid gains sit comfortably inside that noise. He reported them as no claim at the time, by a rule written beforehand.
Anyone who writes about AI benchmarks without this experience has not run the baseline. Anyone who reports a one-grid gain without this check has not run the noise.