STATE OF AI REPORT 2023.
The most widely read and trusted analysis of key developments in AI.
Explore the full report
Generative AI becomes the center of the industry
The sixth State of AI Report captured the first year of the ChatGPT era. GPT-4 led the frontier, Llama helped galvanize an open model ecosystem, and generative AI attracted capital while the wider startup market slowed. Early productivity gains were real in particular tasks, but retention, reliability, and governance remained open questions.
Published on October 12, 2023, this edition captures what we knew then. The findings and predictions below retain that historical perspective.
Read the original Air Street Press launch essay and website launch post, or explore the 2023 report by topic.
Let’s dive into the key stories.
GPT-4 leads as open models gather momentum
GPT-4 was the reference point for language-model capability in the 2023 report. At the same time, Llama 2 made capable models more accessible, and experiments with smaller models showed how much carefully chosen training data could matter. Progress came with growing uncertainty about what closed labs were actually building.
Curated training data made small models surprisingly useful on selected tasks. The evaluation setup mattered as much as the size comparison: a story-generation result assessed with GPT-4 was not the same experiment as testing broad reasoning or factual reliability. The report’s examples supported specialization, rather than universal equivalence with larger models.
Language models begin to use tools and build skills
The 2023 report followed language models beyond generating text. Tool use let them retrieve information, call software, and act in an environment. Systems such as Voyager showed how code execution and stored skills could support longer sequences of work, while leaving open how well those results would generalize.
Voyager stored successful code as reusable skills, so later behavior depended on more than the language model alone. Its curriculum, environment feedback, and skill library formed an interacting system. This makes it important to evaluate the whole agent setup rather than attribute every outcome to a model’s next-token predictions.
The generative AI boom runs through NVIDIA
Demand for generative AI made access to compute a central industry issue in 2023. NVIDIA’s financial results reflected the rush to build infrastructure, while the report’s Compute Index showed how deeply its hardware was embedded in AI research.
NVIDIA’s revenue and its research-paper footprint both showed a strong position, but they were not interchangeable. Revenue described sales in a business segment; the Compute Index described hardware reported in papers. Keeping those denominators separate makes the evidence more useful for understanding both the business and research ecosystem.
Productivity gains meet a retention problem
The 2023 report found evidence that generative AI could make particular kinds of work faster. Turning that usefulness into a durable business was less settled. Consumer products still struggled to retain users, while generative AI attracted enough investment to support the wider AI funding market.
The writing study measured time and rated quality on defined professional tasks. Those results made the benefit concrete, while leaving other questions open: how often the tasks occurred, how much checking they required, and how assistance changed a broader workflow. The experiment did not measure an entire organization’s productivity.
Web knowledge begins to inform physical actions
The 2023 robotics story connected two kinds of progress. Models trained on internet-scale data began to help robots interpret objects and instructions, while specialized learning systems achieved exceptional performance in tightly defined physical tasks.
RT-2’s large model connected language and vision to robot actions, but its reported control frequency made latency visible. The physical system could not evaluate capability in isolation from timing and infrastructure. A decision that was useful in principle also had to arrive fast enough for the intended behavior.
AI designs proteins and interprets genetic variation
Beyond chatbots, the 2023 report documented AI methods working on scientific problems with distinct data and validation requirements. Diffusion models generated protein structures, AlphaMissense predicted the effects of genetic variation, and learned weather models improved on established forecasting tasks.
AlphaMissense expanded the set of genetic variants for which researchers could consult a predicted effect. The report also emphasized how few variants had confirmed clinical classifications. The resource could guide investigation, but its scale should not be confused with the number of clinically established conclusions.
Governments choose different paths for AI rules
In October 2023, governments were still choosing how to regulate the new generation of AI systems. Some leaned on existing laws, others developed AI-specific frameworks, and international governance remained at an early stage. Control over advanced semiconductors was already a concrete instrument of state policy.
A summit, a code-of-conduct discussion, and a proposal for a new institution could all signal political attention. They did not create the same obligations or enforcement powers. The 2023 report captured an international process taking shape, with many institutional outcomes still unsettled.
Safety becomes a mainstream question with unresolved methods
AI safety moved into government and public debate in 2023, while technical work exposed persistent weaknesses in the methods used to steer models. The report separated increasing attention to the problem from evidence that the underlying reliability and control challenges had been solved.
RLHF optimized a signal derived from human judgments, but the signal could omit important values or be exploited by optimization. The report treated this as a structural challenge. Improving performance against a training reward was not the same as establishing reliable behavior under every new circumstance.
Ten forecasts from the first year of the ChatGPT era
The 2023 report made ten forecasts for the following twelve months, spanning entertainment, agents, finance, competition policy, governance, and AI hardware. This archive keeps the original wording and time frame so readers can distinguish what was expected then from what became clear later.
The retrospective separated YES, NO, and partial judgments. Keeping those categories avoids improving a forecast record by rounding a nearby event into a complete success. The accompanying evidence and original wording are necessary to understand why a prediction received its label.
Three of the forecasts published in 2023:
- Self-improving AI agents crush SOTA in a complex environment (e.g. AAA game, tool use, science).
- Financial institutions launch GPU debt funds to replace VC equity dollars for compute funding.
- An AI-generated song breaks into the Billboard Hot 100 Top 10 or the Spotify Top Hits 2024.
The report is authored by Nathan Benaich and Air Street Capital
For our writing throughout the year, subscribe to Air Street Press.
Cite the report
Benaich, N. and Air Street Capital (2023). State of AI Report 2023. https://www.stateof.ai/2023