The 2019 report described an ImageNet-like moment for language: models learned from large collections of unlabeled text and transferred that knowledge to other tasks. Progress in games and biology showed the reach of learning systems, while the limits of each demonstration still mattered.
BERT, ELMo, ULMFiT, and other approaches showed how representations learned from text could improve several language tasks. This reduced reliance on training a separate model from scratch for each problem. The report saw the potential for new commercial applications as the approach scaled.
DeepMind beat a world-class StarCraft II player 5-0. Its system learned from human games and then a league of competing agents. The report explicitly cautioned that restrictions on the action space meant StarCraft II should not yet be called solved.
The first AlphaFold used neural networks to predict amino-acid distances and angles that informed a three-dimensional structure. It improved on earlier methods, but the report noted that the system did not yet address protein complexes.
Competitive play exposed progress at a larger scale
OpenAI Five won two games against the world champion Dota 2 team in April 2019. In the subsequent Arena event, the report recorded a 99.4% win rate across 7,257 competitive games involving 15,019 players. The system trained through self-play, combining games against its current policy with games against older versions. The result extended the evidence beyond a single exhibition match.
Game performance is specific to the match conditions. The protein result refers to the original AlphaFold, not AlphaFold 2 or later systems. Language-model progress did not establish general reasoning ability.
Pretrained language models learned reusable representations from large text collections, improving a range of downstream tasks. The report compared this development with the earlier impact of ImageNet-based transfer learning in vision.
Large text collections let models learn reusable language representations before adapting to a particular task. The report highlighted this as a way to reduce dependence on training a separate representation for every application.
The report discussed BERT, ELMo, ULMFiT, and OpenAI’s Transformer among the approaches bringing pretrained representations to NLP. These were examples from the 2019 research landscape.
The report recorded a 99.4% win rate across 7,257 competitive Dota 2 games involving 15,019 players. That was evidence from a particular game and event, not a measure of performance across arbitrary real-world tasks.
The game combined imperfect information, a large set of possible actions, real-time control, and strategic decisions over a long horizon. The report highlighted this combination of challenges when explaining AlphaStar’s result.
The report described initial supervised learning from human games, followed by reinforcement learning in a league of agents. Those stages combined demonstrations with experience generated through competition.
Protein complexes were not covered by the system described in the 2019 report. Predicting an individual folded protein was distinct from modeling the larger assemblies in which proteins often function.
Historical snapshot published June 28, 2019. This web edition was prepared on 2026-10-11 from the online deck and original launch posts. Findings and forecasts retain their original time frame.
2019 report, PDF page 11Original 2019 report. This edition has no printed slide numbers. References use one-based PDF pages.
2019 report, PDF page 13Original 2019 report. This edition has no printed slide numbers. References use one-based PDF pages.
2019 report, PDF page 20Original 2019 report. This edition has no printed slide numbers. References use one-based PDF pages.
2019 report, PDF page 21Original 2019 report. This edition has no printed slide numbers. References use one-based PDF pages.
Benaich, Nathan, and Ian Hogarth. “Pretraining changes language AI.” State of AI Report 2019. Historical report snapshot; web edition prepared 2026-10-11.
