Overview
- Waymo and DeepMind used a post‑training method that lets the model generate synchronized 2D video and 3D lidar for the same scene.
- Engineers can direct simulations with driving action control for what‑ifs, scene layout control to alter roads and traffic, and language control to change time or weather.
- The model can turn ordinary dashcam footage into interactive environments that emit matching multimodal sensor data and can render longer sequences, including 4x playback.
- Edge‑case testing covers tornadoes, floods, roadway fires, a snow‑covered Golden Gate Bridge, and even an elephant obstructing the road.
- Waymo says the tool speeds scenario creation and helps prepare vehicles for tougher markets like Boston and Washington, D.C., while reporters note its effectiveness hinges on realism and it does not yet simulate fleet‑wide city events.