The Scoreboard

Parts I through III of this book were written in a library. This part comes from a front.

Between December 2024 and July 2026, a run of contests that had stood for years — one of them a benchmark built precisely to outlast the machines — were decided, and not in our favor. Software now outscores people at several of the things the previous eleven chapters taught you to love. There is no honest way to write a book about natural intelligence in 2026 without putting that on the table first, in full, with dates.

So Part IV is a war report in three dispatches. This chapter is the ledger of what fell. Chapter 13 walks the moats that held. Chapter 14 covers the contested ground in between, where the line was still moving as this book went to press.

Three rules govern all three chapters, because this part carries a special obligation: it will be read after some of its numbers have gone stale, and it intends to be respected anyway.

Every claim is dated. "As of July 2026" is not a verbal tic; it is the load-bearing phrase of the whole part. A capability claim without a date is advertising.

Every capability is tagged. [fallen] means machines match or beat people on the dated evidence. [eroding] means the human lead is real but shrinking on a measured clock. [durable] means a five-year human lead is still a defensible bet — defensible, not sacred.

Claims that do real work carry a grade. ⚖ marks a primary-sourced result, ⚠ a secondary or contested figure, ⚙ a mechanism — why something happens — and ✦ a rule of thumb. Where a number comes from a leaderboard aggregator rather than a lab, the text says so.

The tags do work that adjectives cannot. "AI is amazing" and "AI is overhyped" are both moods; [fallen], dated December 2024, is a checkable statement about the world. This Part trades moods for statements.

If you are reading this in 2028 and some line below has been overtaken, good. That is what the dates are for. Judge the method, not the snapshot. And no cheering in either direction: a scoreboard that flatters the home team is not a scoreboard.

The test built to outlast machines [fallen]

Chapter 3 ended with the Abstraction and Reasoning Corpus — the Sphinx's gate rebuilt for machines. Here is what happened to it.

ARC-AGI-1 (2019) made a clean bet: small grids of colored cells, a few examples of a transformation, then a new case — find the rule. No trivia to memorize, no training set to mine; each task asks for a fresh abstraction. The design goal was never to be hard the way chess is hard; it was to measure how efficiently a solver acquires a brand-new skill from a handful of examples — the thing children do all day and machines, for years, could not do at all. The bet held for five years, through a fifty-thousand-fold scale-up of the language models thrown at it ⚖. For half a decade the machine scores sat near the floor while nearly everything else in AI moved. Then, in December 2024, OpenAI's o3 — spending heavily on computation at test time, adapting to each task on the fly — reached 87.5% in its high-compute mode. Five years of nothing, then everything inside a season.

The successor was built harder. ARC-AGI-2 (March 2025) was calibrated on more than 400 human testers in San Diego: every evaluation task is solvable, within two attempts, by at least two ordinary people, and the average individual test-taker scores about 60%. At launch, pure language models managed 0–4%. Sixteen months later — July 2026 — the frontier leaderboard showed roughly 85% (GPT-5.5), an aggregator figure ⚠, above the average human and at the grand-prize bar.

Now the asterisk that matters: cost. On the competition's efficiency track, where entrants get about $50 of compute for 120 tasks, the best score was 24.03% (NVARC, December 2025) ⚖, and the private-set grand prize remained unclaimed as of July 2026. Machines can now do this kind of abstraction; doing it for pennies is another matter. A person solves these puzzles on a sandwich. That thread — watts, words, the economics of thought — is picked up in Chapter 13, because it is one of the things that held.

The verdict, stated precisely: static abstract reasoning, as a capability, fell by July 2026. The efficiency argument survives. And the benchmark's own designers moved the flag to interactive worlds — ARC-AGI-3 — where the score inverts spectacularly. That battle belongs to Chapter 14.

One year after "humanity has prevailed" [fallen]

Competitive programming fell in two acts, twelve months apart, and the dates deserve to be written out in full.

Act one: July 2025, the AtCoder World Tour Finals, heuristic division — open-ended optimization against the world's best. The human champion, who competes under the handle Psyho, beat OpenAI's entrant and posted three words that traveled around the world: "humanity has prevailed."

Act two: Tokyo, July 9, 2026. At the same finals one year on, an OpenAI model swept all five problems and scored 8,300 points; the best human managed 4,300 ⚠ (press-reported figures). Not edged out — nearly doubled. The same year's ICPC World Finals, the premier team contest for university programmers, had already gone the same way: an OpenAI system solved all twelve problems (2025).

Note what kind of contest Tokyo was. The heuristic division has no single right answer — entrants chase the best score they can find in a vast search space. It fell anyway, and that matters for the pattern at the end of this chapter: what makes a task machine-winnable is not a unique answer but automatic scoring. An open-ended problem with a scoreboard is still a closed problem.

The lesson is not that the 2025 victory was hollow. It was real, and it was earned against the best system then built. The lesson is the interval. The distance from "humanity has prevailed" to a clean sweep was exactly one year. Keep that interval in mind whenever this book says [eroding]: the AtCoder door is what eroding looks like from the inside — a genuine human win, honestly celebrated, with a countdown attached that nobody could read.

The proof line [fallen]

Olympiad mathematics — the summit Part III trains toward — held longer than most predictions and then fell on schedule with the rest.

July 2025: two systems, DeepMind's Gemini Deep Think and an experimental OpenAI model, reached the International Mathematical Olympiad's gold-medal threshold of 35 points out of 42 ⚖. Gold-level performance on the world's most selective mathematics contest, from machines, within the working lifetime of everyone who ever said it was impossible.

It did not stop at contests. By 2026, frontier systems had settled 9 of 353 open problems from Erdős's famous collection, with the proofs checked in the Lean formal-verification system, and had disproved Erdős's planar unit-distance conjecture (announced 2026) ⚠ (press-reported). An amateur working with GPT-5.4 Pro closed a problem about prime sets that had stood open for sixty years (2026) ⚠.

Read that last sentence again: an amateur. The machine supplied technique; the person supplied the question, the persistence, and the judgment of what counted as finished. Across the 2025–26 mathematics results, the human role narrowed to three things — choosing the problem, setting the strategy, and verifying the result. File those three away. They are the skeleton of the next chapter.

And the Lean detail is not a technicality; it is the mechanism on display ⚙. A formal proof checker is the cheapest, strictest verifier there is — a referee that works for free, never tires, and cannot be charmed. Give a searching machine that referee and it can try a million dead ends a night and keep only what compiles. Mathematics did not fall despite being rigorous. It fell because it is rigorous.

As this book went to press, IMO 2026 was underway in Shanghai (July 10–21, 2026), with prediction markets pricing an AI gold at about 82% ⚠. The result lands on Chapter 14's watch list; this book states its bets before the results come in, not after.

The white coat, narrowly [fallen]

Clinical diagnosis was supposed to sit safely behind licensure and lived experience. On one specific version of the task, it did not. A Harvard/Beth Israel Deaconess study (April–May 2026) ran an o1-class model against real emergency-department records: 67.1% diagnostic accuracy for the model, versus 55.3% and 50.0% for two groups of expert attending physicians ⚖ (press-reported study). The researchers' own epitaph for the exam era: "we're already at the ceiling" of multiple-choice benchmarks.

That sentence deserves a slow read. The multiple-choice exam — the instrument a century of professional licensure is built on — stopped being able to distinguish machine from expert somewhere before mid-2026. Whatever we still want to claim about human judgment in medicine, the exam is no longer the place we can claim it.

Now the width of the claim, exactly. The model read records; it did not run a department. The study was text-only. The same class of systems tends to over-order tests. And the gap between a correct diagnosis on paper and a treated patient in a working hospital remains wide (Stanford–Harvard State of Clinical AI, 2026). What fell is reading a prepared chart better than the median expert reader — a static, fully specified, checkable task. Hold on to those three adjectives. They are about to become this chapter's thesis.

The rest of the ledger [fallen]

Six more entries, compressed into a table because this Part respects your time. Each is real; each carries its own asterisk — and the asterisks are where the next two chapters live.

The pattern of falling

Lay the fallen side by side and one profile emerges. Contest programs. Olympiad proofs. Abstraction grids. Exam questions. A prepared patient chart. Debate prompts. Prose in a known author's voice. Every one of them is:

Static. The task sits still while you work on it. The chart is already written; the problem statement does not fight back; no supplier lies to you halfway through. Time is not an opponent.

Well-specified. The goal arrives from outside, complete. Prove this. Diagnose this. Maximize this score. Nobody on the fallen list had to decide what was worth wanting — the hardest move in the setter's craft (Chapter 8) had already been made for them.

Verifiable. A machine can check the answer cheaply and exactly. The proof compiles in Lean or it does not. The program scores 8,300 or it does not. The grid matches or it does not.

The third property is the engine ⚙. Where checking is cheap, training is possible: generate a million attempts, keep what the checker blesses, repeat until the checker is always pleased. A cheap verifier converts raw computation into skill. That is why the theorem — the crown jewel of human reasoning, the summit Part III treats with reverence — fell before the unfamiliar kitchen did: a proof can be machine-checked; breakfast cannot. Chapter 14's vending-machine result makes the same point with money.

There is an inversion of prestige buried here, and it stings. Humans ranked these tasks by how hard they are for us: the olympiad above small talk, the proof above the kitchen. Machines sort tasks along a different axis entirely — how hard they are to specify and to check ⚙. The two rankings are nearly perpendicular. That is why the fall order looks upside down from inside a university and exactly right from inside a training run. Moravec pointed at this tilt decades ago; 2024–2026 measured it.

Three corollaries, each earned by a dated example:

Falls are sudden. ARC-AGI-1 sat near the floor for five years, then hit 87.5% inside a season (December 2024). AtCoder went from human champion to clean sweep in exactly twelve months (July 2025 → July 9, 2026). Capabilities arrive as step functions more often than ramps, and a long quiet before a fall tells you almost nothing about the date of the fall ✦. The suddenness has a mechanism ⚙: capability accumulates invisibly — in scale, in tooling, in test-time tricks — while the score sits still, like water rising behind a dam. The score measures the spill, not the reservoir; when the wall is finally topped, the jump looks instantaneous, though the filling never was.

Falls are one-way. Nothing on this list has un-fallen. Benchmarks get retired and rebuilt harder — ARC-2 after ARC-1, ARC-3 after ARC-2 — but no demonstrated capability has been lost. The scoreboard is a ratchet. Any plan that needs a fallen capability to un-fall is not a plan ✦.

A fallen benchmark is not a deployed replacement. The ~85% ARC score (July 2026) ran on a budget far beyond the $50 that bought 24.03% (December 2025). The 67.1% diagnosis figure came with over-ordered tests and no ward to answer to. The designed drugs had zero approvals as of mid-2026. Capability falls first; economics, integration, and liability fall later, or not at all. A good deal of Chapter 13 lives in that gap.

One more thing belongs in an honest ledger, and it is not a number. If something on the fallen list was your summit — the skill a life got organized around — this book will not talk you out of the sting. The best heuristic programmers alive were beaten at the thing they had built their lives around, in public, twelve months after their champion's finest hour. That is real, and it is not the end of the analysis: what we loved about these skills was rarely the checkable part, and the checkable part is all that fell. The next chapter makes that claim precise.

So, a reader's triage — three questions to put to any task, including the one you are paid for ✦:

1. Does the task sit still while you work on it?
2. Is the goal handed to you, fully specified?
3. Can a machine check the answer for pennies?

Three yeses is the profile of everything that fell by mid-2026. Every no is a moat — and a moat, remember, is not a wall. It is ground that stays defensible while you stand on it and work.

The moats are next.

Sources

All accessed July 16, 2026. Press and aggregator items are marked; treat their figures with the confidence noted in the text.

• ARC Prize Foundation — ARC-AGI-2 specification and human calibration: arcprize.org/arc-agi/2
• ARC Prize 2025 results and analysis (December 2025): arcprize.org/blog/arc-prize-2025-results-analysis · technical report: arxiv.org/html/2601.10904v1
• ARC-AGI-2 frontier score ~85%, GPT-5.5 (July 2026) — aggregator, secondary: benchlm.ai/benchmarks/arcAgi2
• AtCoder World Tour Finals 2026 sweep, 8,300 vs 4,300 (July 9, 2026) — press: the-decoder.com/openais-ai-beats-every-human-at-atcoder-a-top-competitive-programming-contest/
• International Mathematical Olympiad 2026, Shanghai (July 10–21, 2026): imo-official.org/editions/2026/
• Erdős problems settled and conjecture disproved (announced 2026) — press: scientificamerican.com/article/ai-uncovers-solutions-to-erdos-problems-moving-closer-to-transforming-math/
• Harvard/Beth Israel Deaconess emergency-diagnosis study (April–May 2026) — press: fortune.com/2026/05/04/harvard-study-ai-outdiagnose-doctors-openai-o1-preview/
• Humanity's Last Exam leaderboard (July 2026) — aggregator, secondary: llm-stats.com/benchmarks/humanity's-last-exam
• Conversational persuasiveness of personalized GPT-4, Nature Human Behaviour (May 2025): nature.com/articles/s41562-025-02194-6
• Higher-order theory of mind, Frontiers in Human Neuroscience (2025): frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1633272/full
• DialToM (2026): arxiv.org/pdf/2604.20443
• Fine-tuned AI writing versus MFA-trained experts, CHI (April 2026): dl.acm.org/doi/10.1145/3772318.3791276
• Isomorphic Labs first-in-human AI-designed drug trials (2026) — press: mlq.ai/news/isomorphic-labs-launches-human-trials-for-ai-designed-drugs/
• Waymo Safety Impact, 220.6M rider-only miles (March 2026): waymo.com/safety/impact/