It was the year when self-driving cars got boring. When iPhones became confusing. When the army spent $480 million on Microsoft Hololens headsets. When we finally got to see the long-awaited Magic Leap…and it was a dud. When virtual reality headsets like Oculus Go had their cords cut from bulky tower PCs.
We’re not living in the future yet, but 2018 was an important year for demonstrating how all these technological puzzle pieces will fit together, eventually.
There were plenty of other fascinating but lower-profile projects that you may have missed, too, born from big box retail stores and basements alike. With that in mind, here are some of the most important projects of the year in graphics, interface, and user experience.
Deepfakes–and the war on what’s real
Deepfakes was undoubtedly one of the most important tech moments in 2018. It was a project that launched with incredible contentiousness, and small wonder: the fan-created interface allowed anyone to train an AI to swap one person’s face in a video with another’s. At first, it was used for pornography, but we quickly saw Deefakes alter pop culture, too, from besting Industrial Light and Magic’s work on Star Wars to creating a fake President Obama. We were the only publication to interview Deepfakes this year. Don’t miss it.
It’s easy to forget that Microsoft is now the world’s most valuable company. Yet somehow, its Microsoft Research arm remains a gem of weird and fascinating work that’s gone untouched even as Microsoft has grown. The group regularly churns out interesting experiments, but the Canetroller takes the cake. It’s essentially a cane that works in virtual reality, allowing someone without vision to tap their way through simulated 3D environments. It’s a reminder that the next digital revolution cannot leave anyone behind. And it’s a marvel of engineering, to boot.
Augmented reality holds incredible theoretical promise, but it faces major UX hurdles. After all, how many of us really enjoy squinting at the world through our phones? Or wearing those bulky glasses? The Portable Lumipen, a research project developed between Sony and University of Tokyo, solves those problems in a very simple way: it projects AR graphics right onto real life. Powered by an eye-, world-, and body-tracking projector that hangs around your neck, Lumipen can, say, spot a scurrying cockroach and float a label over its head in a terrifyingly smooth 1,000 frames per second. It was a reminder that AR still hasn’t found it perfect form, but it’s getting there.
The Force Jacket
It sounds like Star Wars fan gear, but in reality, the Force Jacket is a project by MIT, Carnegie Mellon, and Disney Research that uses a series of air bags to make you feel sensations on your torso–anything from the experience of having supersized muscles to the horror of bugs crawling all over your skin. It appears to make you look something like the Stay Puffed Marshmallow Man, but it’s also a glimpse at the future of tactile interfaces in the digital world.
We’re living through an obesity epidemic. The problem is deep and complex, and tied to lobbying, politics, subsidies, and our agricultural system. But researchers at the Tufts University School of Engineering think tiny mouth sensors could help some people. They debuted a 2 x 2 mm sensor that sticks on your tooth, tastes your food, and transmits its contents back via radio to your phone. The technology is in its early days for sure, but it points to a future where circuits live not just on, but in your body. Such tech will open up a new pipeline of personal data we can barely imagine today.
GridDrones is a fleet of flying robots that float in front of you. Grab one, and you can move it, placing it anywhere you like in midair. Grab many, and you can choreograph your own show with nothing but your two hands. Maybe we won’t all have our own drone displays one day, but GridDrones teases the next wave of matter: programmable, flexible, and as easy to use as any dumb object today.
This Mindbending iPhone
3D phones are a gimmick, sure, but oh, what a gimmick!
Media artist Peder Norrby built an app to give his iPhone X a trompe-l’oeil effect, simulating a 3D image on a 2D plane. The effect is absolutely reality-shattering interface magic.
Nvidia’s AI-drawn playable city
There’s crazy and there’s crazy. This is crazy. (The second crazy. The crazy with more emphasis. Not the first, more boring crazy.)
Nvidia, alongside the the University of California, trained an AI to draw the buildings, cars, and surfaces of city streets. No big deal? Well this system did so in real time, constructing cities from scratch. It’s just the sort of graphics engine that developers will need to build the infinite expanses of virtual reality. It’s just the sort of tool that will one day allow a single designer to craft a deep, immersive world on their own.
Imagine an Uber with no driver. That’s Waymo One, which launched in the Phoenix area to the public late this year. While the robo cars still feel far from human, every bit of the interface design reinforces trust in the machine, and a calmness in the cabin. We reported an exclusive deep dive with the design team on how Alphabet created the robot mind, and somehow, managed to make us bored even while heading at breakneck speed into the future.
Spatial hologram meetings
We’re the biggest skeptics when it comes to teleconferencing or (*swallows bile*) telepresence. But when it’s designed by MIT Media Lab and Samsung alum Jinha Lee, we’ll give it a shot. With $8 million in funding, his young company Spatial came out of stealth in 2018 to show a very impressive, shared hologram conference room experience that is sure to only get better.
Friend UI and the rise of Dark Mode UX
Friend UI is a beautiful smartphone interface built around a simple premise: the modern OLED displays in many phones don’t require energy to show black pixels, so why not make more pixels black to extend battery life?
Friend UI was just a concept, but dark UI was all over 2018. It’s in Android’s Power Saver mode, along with the Pixel’s always-on display. And it’s in the latest MacOS Mojave (for which Google just released a Dark Mode version of Chrome to match!). Given that battery technology is stuck in a rut, dark modes are probably here to stay.
Metalimbs Fusion tele-robot backpack
Imagine having two extra arms, controlled by someone else. That’s the premise of Fusion–the latest iteration of a project called MetaLimbs–out of Keio University and the University of Tokyo. Fusion puts a pair of robo arms and a camera over the human’s shoulder, so someone half a world away can have a first-person view of their body, either adding a few helping hands, or even puppeting someone to teach them a new skill.
Google’s insane AI-powered everything
This year, Google’s flex moves were all about how AI could empower the next wave of UI–like its flight tracker that can predict when your plane is delayed before the airline tells you. It also put a lot more AI into your pocket, running more code locally on your phone. This enabled new features like Call Screening on the Pixel 3, which answers phone calls and transcribes messages in real time. On top of that, Pixel AI now allows you to shoot night photographs that look like they were taken with reasonable amounts of light–solving one of the biggest human factors problems with smartphone cameras today.
Airbnb Backyard housing
We still don’t know a lot about the project, but Airbnb’s Samara futures division has been developing housing, working under code name Backyard, and its first units will go live in 2019. It may not sound like an interface, but Airbnb is looking at buildings holistically, with a strong focus on connectivity and even, potentially, reconfigurability. That may mean robotic walls that can create a walk-in closet on a whim. A guy can dream, at least.
Walmart.com–and the Amazon pushback
Load Walmart.com if you haven’t in a while. It’s…decent! The redesign launched this year, and it’s a powerful example of how brick and mortar retail is finally firing back at Amazon with digital design. Not to be outdone, Target debuted 2-day shipping nationwide for the 2018 holiday season and has been investing in spreading its new, experiential-focused store design, which is tailored to realities of online and offline shopping today. In each case, major retailers are thinking carefully about how their customers use their websites. Maybe 2019 really will be the year you cancel that Prime membership.