A field in nine movements · 1943 — 2026
Eighty-three years from a paper about neurons written in wartime to systems that write code, fold proteins and argue with you. Two winters, three revolutions, and one architecture that swallowed the field whole.
Scroll to begin
The question
“I propose to consider the question, can machines think?”Alan Turing · Mind, October 1950
Turing knew the question was badly formed, so he replaced it with a game — an interrogator, a machine, and a rule that only the answers count. Everything since has been an attempt to build a player.
What follows is that attempt, in order: the confident decades, the two collapses, the unfashionable idea that kept working, and the point in the 2010s where compute, data and one architecture arrived at the same time.
The field has never moved at a steady rate. It moves in a decade of nothing, then eighteen months of everything.
The machine is imagined before it is built
Before a single neuron was simulated, the idea was formalised: thought as computation, the brain as logic, the machine as a candidate mind.
Idea Warren McCulloch & Walter Pitts
The first mathematical model of a neuron. Networks of simple threshold units, they argue, can compute any logical proposition. The neural network begins as a piece of logic, not biology.
Idea Donald Hebb
“Cells that fire together wire together.” A physical rule for how memory could be stored in connection strength — the ancestor of every weight update since.
Idea Alan Turing
Turing replaces “can machines think?” with an operational test, then answers the objections in advance. He estimates a machine might pass by 2000. He also proposes learning from a child-machine rather than programming an adult mind.
System Marvin Minsky & Dean Edmonds
3,000 vacuum tubes and a surplus autopilot mechanism simulate 40 neurons learning to escape a maze. Reinforcement, implemented in hardware.
System Arthur Samuel
Samuel's IBM 701 program improves by playing itself, storing evaluated positions. In 1959 he coins the term “machine learning”.
Symbols, logic, and enormous confidence
Dartmouth names the field. Programs prove theorems, solve puzzles, and speak in therapist-ese. Researchers predict human-level machines within a generation.
Milestone McCarthy, Minsky, Rochester, Shannon
Ten researchers, two months, one proposal sentence: that every aspect of intelligence can “be so precisely described that a machine can be made to simulate it.” The field gets its name: artificial intelligence.
Idea Frank Rosenblatt
A learning classifier with a convergence proof, built as physical hardware — the Mark I, with 400 photocells and motor-driven weights. The New York Times reports the Navy expects a machine that will “walk, talk, see, write, reproduce itself.”
System John McCarthy
A language where code and data share one structure. LISP becomes the substrate of symbolic AI for thirty years — and gives the field its own hardware industry.
System Joseph Weizenbaum
Two hundred lines of pattern-matching, posing as a Rogerian therapist. Users confide in it and refuse to believe it is mechanical. Weizenbaum spends the rest of his life warning about exactly that reflex.
System Feigenbaum, Lederberg, Buchanan
The first expert system: infers molecular structure from mass spectrometry using chemists' heuristics. The insight that shapes the 1980s — power comes from domain knowledge, not general reasoning.
Winter Marvin Minsky & Seymour Papert
A rigorous book proving single-layer perceptrons cannot compute XOR. Multi-layer networks are left as an open question — but the field reads it as an obituary. Neural network funding evaporates for fifteen years.
System SRI International
A wheeled machine that could perceive a room, plan a route through it and push blocks around — the first system to reason about its own physical actions rather than symbols alone. The project also produced the A* search algorithm, which outlived the robot by fifty years.
System Terry Winograd
Fluent English dialogue about stacking coloured blocks. Astonishing inside its toy universe, and completely unable to leave it — the clearest demonstration of the scaling problem to come.
The bill arrives
Combinatorial explosion meets finite hardware. The Lighthill report guts UK funding; DARPA pulls back. Neural networks are declared a dead end.
Winter Sir James Lighthill / UK Science Research Council
A commissioned review concludes AI has failed to deliver on any of its promises and that combinatorial explosion makes real-world scale unreachable. UK funding is cut to almost nothing; a televised debate turns the verdict public.
Winter US Department of Defense
Open-ended AI grants are replaced by mission-specific contracts after speech-understanding programmes miss their targets. Money flows only where a deliverable can be named in advance.
System Edward Shortliffe
A blood-infection diagnostic that outperforms junior physicians, and is never deployed — liability, integration and trust prove harder than accuracy. A lesson relearned continuously for fifty years.
Knowledge, hand-carved
If intelligence is knowledge, then type the knowledge in. Rule engines enter the Fortune 500, Japan bets a nation on the Fifth Generation, and backpropagation quietly returns.
Milestone Digital Equipment Corporation
A rule engine configures VAX orders with 2,500 rules, later 10,000+. Reported savings around $40M a year make AI a line item in corporate budgets for the first time.
Milestone Japan / MITI
A ten-year, ~$400M national programme for parallel logic-programming machines. It triggers defensive funding in the US and Europe — and delivers almost none of its goals.
Idea John Hopfield
Associative memory as an energy landscape borrowed from statistical physics. It brings physicists into neural networks — and wins a share of the 2024 Nobel Prize in Physics, forty-two years later.
Idea Rumelhart, Hinton & Williams
Multi-layer networks can be trained by propagating error backwards through the layers. The maths existed earlier (Linnainmaa 1970, Werbos 1974); this is the paper that makes the field believe it. Every model on this page descends from it.
Idea Sutton, Barto, Watkins
Temporal-difference learning and, in 1989, Q-learning. Value estimation from delayed reward — the machinery behind AlphaGo and, later, learning from human feedback.
Brittleness at scale
The LISP machine market collapses in months. Expert systems prove impossible to maintain. The word "AI" becomes unsayable in grant applications — so the work continues under other names.
Winter Symbolics, LMI, Xerox
General-purpose workstations from Sun and Apple become faster and far cheaper than specialised AI hardware. A half-billion-dollar industry disappears in roughly a year, taking the field's credibility with it.
System Yann LeCun et al., Bell Labs
Convolutional networks trained with backprop read handwritten ZIP codes, and by the mid-90s process a significant share of US cheques. The architecture that wins 2012 already works — it is starved of data and compute.
Idea Cortes & Vapnik
Elegant theory, convex optimisation, excellent results on small data. For a decade, the sensible choice — and the reason neural networks stay a minority interest.
Idea Hochreiter & Schmidhuber
Gated memory cells solve the vanishing-gradient problem for sequences. Largely ignored for fifteen years, then it powers Google Translate and Siri — until attention replaces it.
Statistics wins, unglamorously
Chess falls. Search, spam filters and speech recognition ship to billions. Convolutional nets and LSTMs exist but lack data and silicon. ImageNet is assembled and waits.
Milestone IBM
200 million positions per second, custom chess chips, and human-tuned evaluation. Not learning, but proof that a machine can beat the best human at a task long treated as the signature of intellect. The public verdict: chess must not have required real thought.
Milestone Stanford Racing Team
212 km of desert, driven autonomously, one year after every entrant failed. Machine learning on sensor data, not hand-written rules — the ignition point for the autonomous vehicle industry.
Idea Hinton, Osindero, Teh
Layer-wise pre-training makes deep networks trainable again and rebrands a discredited field. Within six years the pre-training trick is unnecessary — plain backprop plus GPUs is enough.
System NVIDIA
A programming layer that lets ordinary code run on the thousands of small cores inside a graphics chip. Written for physics and finance; five years later it turns out to be the substrate the entire field had been waiting for.
Milestone Fei-Fei Li et al.
14 million images across 20,000 categories, labelled by crowdworkers over years. The unfashionable bet that data, not algorithms, was the bottleneck. It was.
Milestone IBM
Statistical NLP over an offline corpus beats two champions on live television. The commercialisation that follows — Watson Health above all — becomes the decade's cautionary tale about demos versus deployment.
GPUs, data, and a 10-point drop
AlexNet halves the ImageNet error rate on two gaming GPUs. Within five years vision, speech and translation are rebuilt on neural networks. Then a paper about attention rewires everything again.
Milestone Krizhevsky, Sutskever & Hinton
A convolutional net trained on two consumer GPUs cuts ImageNet top-5 error from 26.2% to 15.3%. The margin is so large the field reorganises around it within eighteen months. GPU compute becomes the strategic resource it remains today.
Idea Mikolov et al., Google
Words as dense vectors where geometry encodes meaning. Embeddings become the default interface between language and neural networks — and the first widely visible evidence that models absorb human bias with human data.
Idea Ian Goodfellow et al.
Two networks in a forger-and-detective loop. Photorealistic synthesis arrives years earlier than expected, and with it the first serious public argument about synthetic media.
Milestone He, Zhang, Ren & Sun, Microsoft
Residual connections make 152-layer networks trainable, pushing top-5 error to 3.6% — below the measured human rate. Skip connections are now in essentially every deep architecture, transformers included.
Milestone DeepMind
Go has more legal positions than atoms in the observable universe; brute force was never an option. Policy and value networks plus tree search win 4–1. Move 37 — an opening no human would play — is the moment the field starts talking about machine creativity.
Idea Vaswani et al., Google
Drop recurrence, keep attention, and the whole sequence trains in parallel. The transformer is the single most consequential architecture of the era: every frontier model since — text, image, audio, protein — is a descendant.
System DeepMind
Chess, shogi and Go from the rules alone, with zero human games. Superhuman chess in nine hours. Self-play establishes that a learning signal can be manufactured rather than collected.
Bigger is, embarrassingly, better
One architecture — the transformer — absorbs text, code, protein structures and pixels. Scaling laws turn research into an engineering budget. Then a chat box goes public.
System Devlin et al., Google
Pre-train on unlabelled text, fine-tune on anything. NLP benchmarks fall in a single year, and “download a large pre-trained model” becomes the default starting point for every language task.
System OpenAI
1.5B parameters, coherent paragraphs, and a staged release citing misuse risk. Criticised at the time as hype or paternalism; it establishes the release-policy debate that now governs every frontier launch.
Idea Kaplan et al. / later Hoffmann et al.
Loss falls as a smooth power law in compute, data and parameters. Capability becomes forecastable from a budget — which converts AI research into capital expenditure. Chinchilla (2022) corrects the recipe: most large models were badly under-trained on data.
Milestone OpenAI
175B parameters, ~3×10²³ FLOP of training. Few-shot prompting means one model handles tasks it was never trained for. The API turns a research artefact into a platform.
Milestone DeepMind
Protein structure prediction at near-experimental accuracy at CASP14, ending a 50-year grand challenge. 200M+ predicted structures are released free. The 2024 Nobel Prize in Chemistry follows.
System OpenAI
Shared text–image embeddings, generated pictures from prompts, and code completion trained on public repositories. Multimodality and programming — the two capabilities that will define commercial adoption — arrive in the same year.
Idea Ouyang et al., OpenAI
Human preference data turns a text predictor into something that follows instructions — a 1.3B aligned model preferred over the 175B base. Alignment technique, not scale, is what makes the technology usable by non-experts.
System CompVis / Stability AI
A capable image model released with public weights, runnable on a consumer GPU. It opens the open-weights era — and the still-unresolved fight over training data and copyright.
Milestone US Department of Commerce, October
Export controls block the sale of NVIDIA's top data-centre chips to China, and are tightened repeatedly afterwards. Which country can train which model turns into a licensing question — and becomes the strongest incentive yet for a domestic Chinese chip industry.
Milestone OpenAI, 30 November
A research preview wrapped in a chat box reaches ~100M users in two months. Nothing in the underlying model is new; the interface is. Public understanding of AI splits permanently into before and after.
From demo to infrastructure
Frontier models become products, then platforms, then agents that act. Reasoning at inference time opens a second scaling axis. Capability, cost and controversy all compound.
Milestone OpenAI
Professional-exam performance, image input, and a technical report that discloses no architecture, dataset or parameter count. Capability leaps and transparency retreats in the same document.
System Meta
Frontier-adjacent weights, downloadable. Fine-tuning, quantisation and local inference become a global hobbyist and startup ecosystem, and the open/closed split becomes the industry's central structural question.
System Sora, Veo and the video labs
Text-to-video crosses from novelty to product. By 2026 the leading systems produce eight to twenty second shots with synchronised audio, characters that survive a cut, and physics that is convincing rather than correct. The labs start calling them world models and disagreeing about what that claim means.
Milestone Hopfield & Hinton / Hassabis, Jumper & Baker
Physics for neural networks, Chemistry for protein structure. Machine learning is formally absorbed into the scientific establishment — three decades after both fields were considered dead ends.
Idea OpenAI o1 / o3
Train the model to think in long chains before answering, and spend more compute per question. A second scaling axis opens: performance now improves with thinking time, not only training budget. o3 reports 87.5% on ARC-AGI-1 at very high compute cost.
System DeepSeek R1 and the open ecosystem
Strong reasoning released with open weights at a fraction of the assumed cost triggers a global re-pricing of AI capability — and a market shock. Capability leadership and cost leadership stop being the same race.
System Industry-wide
Models stop answering and start operating: running tool chains, editing repositories, browsing, and holding multi-hour tasks. Measured task horizons — the length of job a model completes reliably — have been roughly doubling every seven months.
Milestone DeepMind & OpenAI experimental models
Systems reach gold-medal standard at the International Mathematical Olympiad working in natural language under contest conditions — a benchmark expected to hold for years longer.
Milestone Moonshot, Alibaba, DeepSeek
Kimi K3 ships with 2.8 trillion parameters and open weights; Qwen3.8-Max follows at 2.4 trillion, DeepSeek-V4 Pro at 1.6 trillion. For the first time, the largest model whose size is publicly known is one anyone can download — and every one of them comes from a Chinese lab.
Milestone Figure, Physical Intelligence, Tesla, DeepMind
Vision-language-action models put a transformer directly in the control loop: one network sees, reasons and moves. Counterpoint counts more than 50,000 humanoids working commercially in 2026, up from 16,000 a year earlier. Small numbers, steep curve — and the same “it worked in the demo” problem that ended the expert-system boom.
Milestone Global
The binding limits shift from ideas to electricity, land, HBM supply and advanced packaging. Multi-hundred-billion-dollar datacentre programmes are underway, and grid interconnection queues become an AI story.
The mechanism
Every system on this timeline resolves to the same loop: read the sequence so far, predict what comes next, add it, run again. Six steps, on repeat. Training is that loop run backwards a few trillion times, nudging the weights whenever the guess is wrong.
The sentence is cut into tokens — common words stay whole, rarer ones split into pieces. Each token is looked up in a fixed vocabulary and replaced by its id. The model never sees letters.
Hardware · 2006 — 2026
Nothing on this timeline happened because someone had a better idea alone. Backpropagation waited thirty years for hardware that could run it at scale, and the hardware arrived from an industry built to draw video-game shadows. Here is how a model is trained, on what, and who sells it.
The same forward pass, run backwards
Training is inference plus an answer key. The model guesses the next token, the guess is compared with what actually came next, and the error is propagated backwards through every layer to work out how much each weight contributed. Every weight is then nudged a fraction in the direction that would have been less wrong. Do that a few hundred thousand times a second, for three months, across a hundred thousand chips.
Estimates for a 2026 frontier run. Compute is 65–75% of a training budget that now starts around half a billion dollars.
US$ billions per fiscal year
NVIDIA did not set out to build the AI industry. It built a parallel processor for rendering, then in 2007 shipped CUDA — a way to run ordinary code on it. For five years that was a niche for physicists. Then two graduate students trained AlexNet on two gaming cards, and the entire field discovered that the machine it needed was already on sale.
A general-purpose programming layer on a graphics chip. Nobody in AI notices for five years.
AlexNet is trained on two GTX 580s in a bedroom-scale setup. The result is so far ahead of the CPU-based field that GPU training becomes non-negotiable within a year.
Jensen Huang hand-delivers the first purpose-built AI supercomputer to a small nonprofit called OpenAI.
The V100 adds silicon that does nothing but matrix multiplication — the one operation a transformer needs. The chip stops being a graphics part.
Hopper lands the same year as ChatGPT. Lead times stretch past a year, allocation becomes a strategic asset, and the US restricts sales to China.
Data-centre revenue reaches $194B for the fiscal year and NVIDIA is the most valuable company in the world. Blackwell gives way to Rubin.
Status, mid-2026
6-gigawatt deals with OpenAI and Meta; 5–7% of the market. Claims 30% more tokens per dollar than Rubin.
Over 75% of Gemini runs on TPUs. Not sold — rented, which makes it invisible in market-share tables.
Handles more than half of Bedrock token throughput. Cheap by design, not fastest.
From nothing to 50–60% of the Chinese market in three years, ~$12B. Stuck on a 7nm process and short of memory.
The quiet one: designs the hyperscalers' in-house chips. Custom ASICs are a bigger threat to NVIDIA than AMD is.
Different architectures aimed at speed per token rather than training throughput. Real, small, fast-growing.
Every challenger has competitive silicon on paper. What none of them has is nineteen years of CUDA: the libraries, the kernels, the fact that every framework was written against it first and every engineer already knows it. Switching cost, not clock speed, is what holds roughly 80% of the market.
Instrumentation
The ideas behind today's systems are mostly forty years old. What arrived recently is the ability to spend: on silicon, on data, and on the electricity to run both. Three measurements tell most of the story.
Logarithmic · FLOP · estimates, order of magnitude
1950 — 2026
Drag the chart sideways →
From proposed to saturated
Human-level handwriting recognition
28.2% → 3.6% error; human ≈ 5.1%
Language understanding suite, beaten in 13 months
Built to be “too hard”. It was not.
57 subjects: 43.9% (GPT-3) → 90%+
Google-proof PhD questions; experts ≈ 65–74%
Abstraction puzzles; 87.5% at extreme compute
The current frontier — still open
Every test built before 2024 to be “too hard” has been beaten. The interval between a benchmark's publication and its defeat has collapsed from fourteen years to under two.
US$ per million output tokens · log scale · OpenRouter, August 2026
A million output tokens cost $60 from GPT-3 in 2021. DeepSeek-V4 Flash charges twelve cents — a five-hundred-fold collapse. The frontier tier did not follow it down: Claude Opus 5 is $25. Intelligence is deflating at the bottom and holding its price at the top.
Size · 2018 — 2026
For five years the answer to "how good is it?" was a single number, and the number only went up. Then the closed frontier stopped publishing it, and mixture-of-experts split it in two. Here is every disclosed figure from GPT-1 to the open-weight releases of 2026, plotted.
Parameters · logarithmic · disclosed and reported figures
2018 — August 2026
Drag the chart sideways →
GPT-1 shipped in 2018 with 117 million parameters. Kimi K3, eight years later, holds 2.8 trillion — roughly 24,000×. The line is not continuous: it runs up through the closed frontier until 2023, then continues only in open-weight releases, because those are the only ones still attaching a number.
Mixture-of-experts split the figure in two. Kimi K3 holds 2.8 trillion parameters and activates 104 billion for any given token — 16 experts out of 896. DeepSeek-V4 Pro holds 1.6 trillion and uses 49 billion. Every dumbbell on this chart is the gap between what a model is and what it costs to run.
No closed frontier model has published a parameter count since GPT-4 in 2023. Every figure on the right of this chart comes from a downloadable release, and nearly all of them are Chinese: Moonshot, Alibaba, DeepSeek, Z.ai, MiniMax. Meta's 2026 open contribution is a 30-billion-parameter dense model.
Where we are · early 2026
A snapshot is the wrong shape for a field that reorganises every eighteen months — but here is the honest state of play, including what is not working.
Pre-training scale still buys capability, but the fastest gains now come from reinforcement learning on reasoning traces and from spending more compute at answer time. A hard question can cost a thousand times more to answer than an easy one.
The dominant interface is shifting from a conversation to a delegated task: an agent with tools, a filesystem, a browser and a budget. Reliability over long horizons — not raw intelligence — is the live engineering problem.
Video generation reached synchronised audio, characters that survive a cut and convincing momentum in 2026 — while still getting physics subtly wrong. What the models have learned about the world is now a research question, not a rendering one.
Vision-language-action models collapse perception, reasoning and control into a single network, and humanoids moved from demo to pilot. The bottleneck is no longer intelligence: there is no internet-scale corpus of a robot picking something up, so the data has to be made.
Cost per unit of capability has fallen by roughly two orders of magnitude since 2023, while the cost of training the very best model keeps climbing. The result is a barbell: near-free commodity intelligence, and a handful of billion-dollar training runs.
Open-weight releases now hold the largest published parameter counts, ship million-token contexts, and undercut closed APIs by two orders of magnitude on price. What is still contested is the top of the reasoning leaderboards — a gap measured in months, and argued over every quarter.
Advanced packaging, high-bandwidth memory and grid interconnection have become the rate-limiting steps. AI is now a heavy-industry story as much as a software one.
The EU AI Act phases in through 2026–27, national safety institutes run frontier evaluations, and export controls shape who can train what. Regulation is being written while the technology reorganises itself underneath it.
Every headline test built before 2024 is saturated. Evaluation has moved to long-horizon agentic tasks, contamination-resistant held-out sets, and human expert comparison — all of which are slow and expensive.
Continual learning, robust long-horizon planning, reliable factuality, sample efficiency, and genuine novelty are all proposed as the missing ingredient. The honest position is that the field does not know which limits are fundamental and which are one paper away.
Forecasts · with citations
What follows is not prediction, it is a spread of positions — each one held by people who published their reasoning and can be checked. They disagree, and the disagreement is the honest part. Numbers are as stated by the source; the reference list at the end links every one.
Epoch AI puts training runs of about 2×10²⁹ FLOP within reach by 2030 — roughly the jump from GPT-2 to GPT-4, again. Power, chips, data and latency are all judged surmountable at that scale. The constraint they name is willingness to spend, not physics. [16][19]
METR measures the length of task a model finishes reliably, and finds it doubling on a regular clock. In 2026 the AI Futures Project shortened its estimate of that clock from 5.5 months to about 4. Surveyed experts put a 50% chance on eight-hour software tasks falling by 2030. [17][21][22]
Grace et al. asked thousands of published AI researchers the same question across successive surveys: the median for high-level machine intelligence moved from 2061 to 2047 in two years. The median moved; the disagreement between respondents did not narrow. [14][22]
Villalobos et al. project that the stock of high-quality human text is consumed on roughly the current trajectory within this decade. Epoch's counter-argument is that synthetic data for reasoning and multimodal corpora push the ceiling out. Both are projections, not measurements. [13][16]
LeCun has argued since 2022 that autoregressive prediction cannot reach human-level intelligence, and that what is missing is a world model — a system that predicts in an abstract space and can plan. If he is right, the scaling curves are measuring progress along a road that stops. [10]
The IEA projects datacentre consumption more than doubling to about 945 TWh by 2030 — more than Japan uses today — with AI the main driver. In the United States, datacentres account for nearly half of all electricity demand growth to 2030. [18]
Acemoglu works upward from task-level exposure and productivity estimates and arrives at no more than a 0.71% total factor productivity gain over ten years. His reading is that AI widens the gap between capital and labour income without moving aggregate growth much. [15]
The International AI Safety Report, now in its second edition, is written by 96 experts under Yoshua Bengio and commissioned by governments. Whatever else happens, the argument about frontier risk now has a citable, jointly-authored baseline. [23]
Every figure on this page is the source's own. Where sources conflict — and on data limits and timelines they conflict sharply — both are shown rather than averaged.
Unresolved
Does scaling continue to pay, or does the curve bend?
Can a system learn continuously without forgetting what it knew?
What is the correct evaluation when the benchmarks are exhausted?
Who is accountable when an autonomous agent causes harm?
Is there enough electricity — and who does not get it?
Does understanding require a body, or only enough text?
Apparatus
A glossary of the terms this page uses without explaining them, and the full list of sources behind its numbers.
Terms, in the order you meet them
Every citation, linked