All topicsSTATE OF AI REPORT.

Can robots learn an unfamiliar task from a video?

By Nathan Benaich (Air Street Capital)

Skild’s S1 performed unfamiliar tasks from video demonstrations without updating its weights. With 100,000 hours of pre-training, it scored 66% on the company’s cumulative per-step success measure, against 9% for a language-prompted baseline trained on the same data and compute.

Source: Skild AI: Introducing S1.

Evidence you can use

Unseen tasks in Skild’s internal evaluation

S1 release, 2026

Unseen tasks in Skild’s internal evaluation
How the task is specifiedCumulative per-step success
Language-prompted baselineSource9%
Video demonstration in contextSource66%

Company-reported results at 100,000 hours of pre-training, using the same data and compute. The metric is cumulative per-step success across tasks. Human intervention was used to recover from failures during rollouts. These percentages are not end-to-end autonomous task completion rates.

Sources: Skild AI: Introducing S1.

Data & methodology →Download CSVJSON

Robot learning from demonstrations and simulated environments

Learning from demonstrations could reduce the task-specific training each deployment needs. World models offer another route. Wayve's GAIA-4 turns recorded driving scenes into simulations in which an AI driver's steering and braking change what it sees next. Other road users follow their recorded paths, but the driver can explore the consequences of its own actions.

Odyssey-3 draws on visual pre-training to learn controls for robots, cars, and games. In the company's experiments, robot arms learned from tens of hours of demonstrations and recovered from missed grasps without being shown those recoveries. These early results suggest that physical AI may need fewer demonstrations of every situation it could encounter.

Frequently asked questions

Answers drawn from the report and the sources below.

How can world models help train robots?

Wayve’s GAIA-4 turns recorded driving scenes into simulations in which an AI driver’s steering and braking change what it sees next. Other road users follow their recorded paths, while the driver can explore the consequences of its own actions.

Source: Wayve: GAIA-4.

Sources and dates

2026 report snapshot. Preview revised 2026-10-07. Individual data periods and source checks are listed below. This is not a claim that every source was updated on that date.

  1. Skild AI: Introducing S12026 release. Primary source checked 2026-10-07.
  2. Wayve: GAIA-42026. Retained from the launch essay.
  3. Odyssey: Introducing Odyssey-32026. Retained from the launch essay.
  4. State of AI Report 2026, slide 56: Teaching robots requires data about how to act2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  5. State of AI Report 2026, slide 57: For π0.7, context makes imperfect robot data useful2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  6. State of AI Report 2026, slide 58: A robot turns five minutes of play into reusable skills2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  7. State of AI Report 2026, slide 60: With a longer memory, a robot can improve long-horizon task completion2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  8. State of AI Report 2026, slide 61: Simulation is a bedrock of robotic reality2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  9. State of AI Report 2026, slide 62: A humanoid learns stair climbing in four hours of simulation2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.
  10. State of AI Report 2026, slide 145: One year on: Waymo tripled to 220M rider-only miles and serves 500k rides a week2026 report snapshot. Read against the report PDF on 2026-10-07. Study-specific limits retained.

Cite this page

Benaich, Nathan. “Can robots learn an unfamiliar task from a video?.” State of AI Report 2026. Published 2026-10-08.