The 3D graphics card company Nvidia recently released this mind-bending bit of research, developed in conjunction with some of the University of California, Berkeley, team behind the best dance video of the year. It’s a driving simulator that allows you to pilot a car through a city. Why is that a big deal? Because that city, from its buildings to its streets to its cars, is being created in real time by AI.
advertisement
Like most AI that generates images, the software was taught on a training set. It learned from driver camera footage of vehicles moving through several major cities. This footage was segmented, meaning that it was labeled with “car” or “tree” so the AI could learn to identify details. And by sheer repetition, it gradually learned what these items should look like and where they should go. Then, as is customary in machine learning, two algorithms duked it out, drawing these items and attempting to fool one another with their forgeries. What resulted is the demo you see here. Nvidia was able to paint a blobby 3D world with a sharp, realistic veneer.
Actual video (left) and AI rendered video (right). [Image: Nvidia]Technically, the demo is impressive for a few reasons. One, it’s a great proof of concept illustrating what’s possible: Yes, we can generate convincing city blocks through automation alone. Architects and city planners could make great use of this technology one day, rendering rapid prototypes to test theories before cementing them in real bricks and concrete. Two, the footage you see isn’t rendered over hours or even days of processing, as so many good AI constructions are today. It’s being generated by the AI model in real time, in an interactive, 3D environment. Granted, the simulation is powered by a $3,000 graphics processor. Even so, it’s rendered just like any video game on your Xbox.
