Among the attributes credited for Apple’s famous customer loyalty is a network of stores where curious or frustrated consumers can meet the company face-to-face.
The 3D design software maker Autodesk is trying to achieve something similar online with a help service that allows people to interact with what sure looks like an actual human. The company says that next year it will introduce a new version of its Autodesk Virtual Agent (AVA) avatar, with an exceedingly lifelike face, voice, and set of “emotions” provided by a New Zealand AI and effects startup called Soul Machines. Born in February as a roughly sketched avatar on a chat interface, AVA’s CGI makeover will turn her into a hyper-detailed, 3D-rendered character–what Soul Machines calls a digital human.
The Autodesk deal is Soul Machines’ first major gig, following a pilot project with the Australia National Disability Insurance Agency from February to September, 2016, and some proof-of-concept demos, like a recent one with Air New Zealand. Greg Cross, Soul Machines’ chief business officer, says the company is working on eight other projects with “major brand names” but declined to name them.
In demonstration videos, the new AVA’s face reacts with all the physiological subtlety of the best Hollywood CGI characters. No wonder, since she’s born of the same technologies. Before founding Soul Machines in 2016, engineer Mark Sagar was a movie special effects guru at outfits like Sony Pictures and Peter Jackson’s Weta Digital. He won two Oscars, in 2010 and 2011, for his work creating lifelike CGI facial animation in films including Jackson’s King Kong and James Cameron’s Avatar. Among his achievements was codeveloping a system called Light Stage that scans a human face and body in extreme detail, with lighting from multiple angles, in order to create lifelike 3D models.
Hollywood productions may take the scans as a starting point, such as morphing impressions by actor Zoe Saldana into the blue-face Neytiri in Avatar or of Mark Ruffalo into the green-faced Hulk in The Avengers. But Soul Machines, using additional software that Sagar developed at the University of Auckland, keeps true to the original, retaining even the pink splotches, clogged pores, and errant eyebrow hairs of its human models.
That embracing of flaws may allow digital humans to avoid the “uncanny valley” of creepiness, when a robot looks oh-so-close-to, but not quite, human. Soul Machines’ imperfect beauty has to be more than skin deep for expressions to be believable, though, with computer modeling of bone structure, muscle twitches, and other subtleties.
Sagar wants to go even deeper. Since leaving cinema, he’s been trying to recreate the human nervous system in software. Analyzing facial expressions to discern a smile, even a subtle one, and analyzing voice to pick up a pleasant tone, Soul Machines’ software provides a hit of virtual dopamine to AVA’s nervous system. As in a human, this triggers a relaxed demeanor in AVA. Don’t be mistaken, though: This is very far from artificial sentience, Sagar cautions. “Everything is radically simplified from the real thing, and even how the real thing works is not understood,” he says. “This is just sort of current thinking on how some of these models work.”
But AVA builds off another project of Sagar’s, called Baby X, which actually does aim to simulate human consciousness, with all its wonders and flaws, like anger and impulsivity. Those wouldn’t be good attributes for a customer service bot.
“There’s not much autonomy [for a digital human] because you don’t want it. However, all the subtleties of the behavior come from the Baby X model,” says Sagar. “If it were an actor, Baby X is just improvising, whereas [a digital human] is being told: I want you to walk here, I want you to say this, and I want you to deliver it with this tone.” Crucially, AVA and other digital humans are incapable of feeling anger. They are built to be bottomless wells of empathy, no matter how nasty a customer gets.
All that digital artistry may seem like overkill for the system that tells people where to find their software downloads. But a bot with a winning personality is all that saves Autodesk from a customer service apocalypse.
“We’ve reached a capacity constraint where, the number of calls coming in–we just don’t have enough people,” says Gregg Spratto, Autodesk’s VP of operations. “Our products are not the easiest ones in the world . . . So we get a fair amount of questions that are just about how to get up and running or which product to choose,” he says.
That’s all gotten worse as Autodesk shifted from selling software that runs forever to selling a subscription service to its customers. Now a glitch with an online login means a highly paid person can’t do their job. Turnaround times of a day or two were common. “If the resolution time is 38 hours, 37 hours of that would be just waiting for the case to get picked up,” says Spratto.
Even the current version of AVA has helped a lot, he says, handling about 40,000 customer help requests a month via the text chat, with an average turnaround of about five minutes. “It would take a small army of our internal agents to handle 40,000 conversations per month,” says Spratto. Built on top of IBM’s chatbot software Watson Conversation API, AVA can currently handle about 500 types of requests, identified from running machine learning on recordings of live help calls to identify common problems customers have.
Autodesk is also experimenting with Watson Tone Analyzer to get a read on a customer’s mood, but Rachael Rekart, who manages Autodesk’s “machine assisted service engagement,” hopes for more. “What soul machines adds onto that are conversational elements,” she says, “like inflection in your voice, tone, being able to pick up on a signal like furrowing your brows–or sometimes you raise your voice. And responding to that.”
With her new face and brain, AVA’s getting a promotion from call center staffer to brand ambassador. “We really strongly feel that if AVA is to be the front door into Autodesk, people need to feel like AVA is a great first solution and not yet another deflection mechanism,” says Spratto.
It’s a leap of faith for Autodesk to go with the Kiwi startup, but Soul Machines made a big impression with its first project, thanks to another two-time Academy Award winner. Cate Blanchett served as the voice model for the Australia National Disability Insurance Agency pilot project. She recorded over 20 hours of phrases that Soul Machines could manipulate into a broad range of dialog for a digital human called NADIA. The brunette avatar’s appearance came from scans of an anonymous model, but Blanchett’s elegant tones, and star power, helped generated international press coverage.
For AVA, though, the looks are especially important. The new face of Autodesk, made from scans of actor Shushila Takao, is elusive, with a light tan complexion that’s hard to pinpoint. “That was very intentional,” says Spratto. “What we intentionally tried to do is make AVA appeal to as broad a cultural spectrum a possible . . . She could be African American. She could be Asian. She could be Latino. She could be a Caucasian.”
What AVA doesn’t look like is a nerd boy’s fantasy. “We had a group of UX [user experience] folks who did go out and survey customers,” says Spratto. “And essentially what they came back with was, they wanted her to look very cartoonish and overly librarian sexy. That was the exact opposite of what we wanted.”
AVA does stick to the robot stereotype of being a woman, but Spratto says that he’s open to someday having a staff of virtual assistants, including men. Soul Machines does include a single male avatar out of three (including AVA) on its home page.
Feeling satisfied with what customers see, Spratto says he’s now worried about their willingness to be seen. The kind of tech-savvy users that make up Autodesk’s customer base are more likely than most to put black tape over their computers’ webcams. Now Autodesk is asking customers to turn the camera on for a face-to-face with its new robot–one designed to analyze their every micro expression. “I think that is going to be the biggest challenge of it,” says Spratto. “And I think a lot of our success will be dependent on how we convince people to give that a try.”
Soul Machines, of course, sounds confident that people will opt for a face-to-face. Gregg Cross says that, in a survey of the 10,000 people who participated in the pilot project with NADIA in Australia, 74% “were happy for her to be their primary customer experience.”
But just adding voice will help, says Spratto. Of the Autodesk customers that use live support, about half choose to text with an agent and half choose to speak. “At a bare minimum, having a voice capability makes AVA more available to those who would rather speak as opposed to [text] chat,” he says. He also thinks AVA’s ability to discern tone in people’s voices will help provide customers with a more understanding service experience.
Despite looking and sounding real, AVA is just an automaton, reading the lines her masters write. And getting the words right has proved tricky. Autodesk recently set a summer intern who majors in creative writing to the task of making AVA’s computer-generated texting dialog appear more human, and Spratto says a humanizing editor will probably become part of the permanent staff. “There’s character development. We have a backstory about AVA, so that the dialog she speaks is actually authentic,” he says.
Autodesk sent me a 650-word profile of AVA. A digital native (of course), she’s in her late 20s or early 30s, and will grow as the company does. AVA embodies five Autodesk brand traits: being real, open, smart, trusted, and inspirational.
AVA isn’t just Autodesk’s platonic ideal. She makes mistakes, the backstory says, but she’s open to constructive criticism and will never blame the customer. Like any human, AVA us even wrestling with competing desires. “She wants to keep moving forward, wants to see what she could be and how her growth could help the company and customers, but wants to make sure she is as reliable and trustworthy as possible,” her profile reads. “There’s a trade-off, and she’s aware of this.”
That’s all just to build a better chatbot. “What we’re really trying to do with Soul Machines,” Spratto says, “is make AVA as close to a human experience as possible, while all the time not pretending that AVA is anything other than a robot.”