World models move AI from predicting words to predicting physical scenes

Language models predict the next word in a sentence. A newer class of systems, called world models, predicts what a scene will look like next. They take visual input and generate digital environments that follow the rules of physics. According to a Wired report published on 8 November, these systems are becoming the training ground for home robots and driverless cars.

This matters because physical machines are expensive and slow to train in the real world, and some situations are too dangerous to stage. A simulation that looks real and reacts to actions lets a robot or a car practise millions of times before it moves at all. Data still has to come from somewhere, though, and Wired reports that much of it now comes from people filming their own daily lives for small payments.

From the next word to the next frame

Oliver Cameron, CEO of the world-model lab Odyssey, described the difference to Wired. "With a language model, you have a sentence, and it will predict the next word in the sentence," he said. "With a world model, you give it a visual state of the world, an image, and it will then predict the next visual state of the world." The 3D output can then be used as a training environment. "They're simulations, but they look and feel real," Cameron said.

Prediction alone is not enough. Jamie Shotton, chief scientist at Wayve, which builds world models for cars, told Wired that a world model "must be able to react to the actions that we take in the world, and so it must understand physics, cause and effect, and how other objects and people change their actions in response to our own."

Training robots and cars in replicas

Niantic, the company behind Pokémon Go, sold the game last year and now works on reconstructing the real world through Niantic Spatial. Its chief technology officer, Brian McClendon, told Wired the company wants to use its store of location data to build digital replicas. His example: someone photographs the place where a robot is meant to work, software turns the photos into a 3D model, and the robot then runs "millions of simulations" to learn that space without physically moving through it.

Wayve applies the same idea to driving. Its generative world model is trained partly on dashcam footage, and the company uses it to recreate hazardous roads and unsafe conditions that would not be practical to test on real streets. With no human driver to take over, the speed and accuracy of the software's reactions is the whole job.

Wired also notes that video game developers are studying the technology, since physics and 3D rendering are central to games. Tencent is among the companies building its own world models.

Where the money is going

The field is drawing large sums. Yann LeCun, formerly Meta's chief AI scientist, co-founded Advanced Machine Intelligence (AMI) Labs in Paris. In March it raised $1.03 billion in a seed round at a $3.5 billion valuation, according to Tech.eu, which called it Europe's largest seed round to date. Alexandre LeBrun is CEO and LeCun is executive chair. Backers include Nvidia and Samsung.

In September, AMD agreed to buy World Labs, the startup Fei-Fei Li founded in 2024, for $8.2 billion, TechCrunch reported. Li, known for her work on ImageNet, is set to join AMD as executive vice president and chief scientist. World Labs' first product, Marble, is pitched for entertainment and for building simulated environments to train robots. The deal is expected to close before the end of 2026, subject to regulatory approval.

Wired expects 2027 to be a major year for home robotics and driverless cars, both of which depend on progress in this area.

The people recording the world

To simulate the physical world, labs need recordings of it. The Wired reporter described strapping an iPhone to their forehead and filming hours of video at home: hands folding laundry, washing dishes, pouring a drink. They sold the footage through an app for what they called "a fairly measly payout". Wired expects that more gig work of this kind will appear as more companies try to build better world models.

This is a different kind of data work from labelling text or rating chatbot answers. It asks for first-person video of ordinary household tasks, which are the tasks home robots are meant to learn.

What it means for work

World models change how physical machines learn. Testing in the real world and staging dangerous scenarios can move into simulations that can be repeated without limit. That affects test drivers and robotics engineers first: some on-road and on-site testing becomes generating scenarios and checking results. If the robots and cars trained this way work as promised, the tasks they aim at are driving and household chores such as laundry and dishes, which today are done by drivers, cleaners and domestic workers. A new layer of work is growing in the meantime: people paid by the clip to film their own hands doing those same chores, often for little money. Open questions remain. Nobody yet knows how well skills learned in simulation hold up in messy real homes and streets, who owns the recordings of private spaces, or whether this data-gathering gig work becomes a lasting job category or a short phase.