Enough With The Sexy Virtual Assistants

UX designer Cathy Pearl talks to Doreen Lorenzo about the gender politics of voice user interfaces–and what so many designers get wrong.

Enough With The Sexy Virtual Assistants

Doreen Lorenzo: You’ve been working in design for more than 15 years now. Tell us about how you got here. How did you decide to get into design?
Cathy Pearl: In a way you could go back to when I was a kid. My dad liked to buy gadgets—he bought a family computer when I was eight, and I got very involved in programming. One of the things I was really interested in was trying to get the computer to have a conversation with me. So I wrote a program that today you might call a primitive chatbot, where you could type something to the computer, and the computer would respond back to you.


Fast-forward all the way to 1999, when I started working for a company called Nuance Communications, which is a speech recognition company, and that’s what I was doing as my job, trying to get a computer—in this case a phone system, an Interactive Voice Response System (IVR) where you call in, you say things, it says things back to you. I did that for eight years and was ready to move on and try something else. I then spent a few years consulting. Eventually I ended up where I am now at, all these years later, trying to get the computer to talk to me again.

Did you always have an interest in design? How did you make those two come together?
When I went to college, I wasn’t really sure what I wanted to do. I did love programming, but I didn’t really want to be a computer science major. I went to UC San Diego, where they had a major called cognitive science, which I’d never heard of before. It’s basically this great mix of neuroscience, linguistics, psychology, and computers—all centered around the question of how the human brain works. And that just fascinated me. I remember the first class I took freshman year, the homework was to go home and think about what makes things funny. And I thought: that’s the greatest thing ever! And of course I went back to class and he did not have the answer, which was very disappointing. But I just loved that course of study, and I minored in computer science.

Afterwards I decided to pursue my master’s. In my second year of graduate school I got to take classes I was more interested in. I took a class called human-computer interaction, and it kind of blew my mind because it was the first time I’d heard that there was an actual discipline of studying and making things work better for humans. Not just necessarily computers or software, but really anything—a stove, a door, anything at all—and there are people who have thought about that and really done their best to not just design function, but to design something that people could easily and efficiently use. Eventually I went to Nuance, and I was put in the role of voice user interface designer. That was the first time I’d had the title of designer, and I really loved it.


How have you seen the design space evolve over the years?
Speaking about voice interface designs specifically, as far as design goes, what has changed a lot is that in 1999 speech recognition was really in its infancy, and it didn’t work very well. And I was amazed when you could do things like say “checking account” or “savings account,” and it would know the difference. And so if you fast forward to now, in 2016, really even in the last five years, speech recognition has improved so much that now we have this whole new universe in front of us that we didn’t have before, where it’s not so constrained.

You can pick up your phone and you say, “Okay, Google,” and you can ask it anything. Not that it will work, but you can try to ask it anything, and it will often at least get the words right that you said, then fails to do what you want, but what’s changed so much is that now we can tackle these much larger and more interesting design spaces where we can truly try to emulate a conversation between a person and a computer, rather than just this very constrained interaction where you can say, “x or y” to accomplish your task.

You also have a strong side interest in the science of relationships and this intersection of data and dating. Tell us a little bit about how you became interested in the topic and how it impacts you, how you think, and your design choices.
I’ve always been really fascinated by what attracts people to one another. And I’ve always been interested in my friends: how did they meet their spouse, why were they attracted to that person but not another person? To me it’s one of the most fundamentally interesting pieces of being human. Why do you like this person, but not that one? People on paper can say, ‘Well, these are my top 10 things that I must have in a spouse,’ or, ‘These are the top 10 reasons I like my spouse.’ But that’s not really it. If you look at what people actually are attracted to, it’s often so hard for people to describe or explain. For some reason I have this dream that someday maybe we can explain at least part of it, and we can take that information and can help people find the right match for them by helping them eliminate some of the difficulty of finding the right person if we can understand it more.


What are some of the insights you’ve found in your research on this topic and how do they apply to what you’re working on with
I think a lot about this question of how you can apply design principles from dating. Right now I work in the health care space—our product deals with helping people with chronic health conditions. And I think some of what people need from relationships is also similar to what people need from their care provider, which is someone to acknowledge how you’re doing, how you’re feeling, and really listen to you and feel understood. People just need to feel understood, whether it’s by their spouse or by their doctor or their friend.

For example at we have this avatar, this virtual nurse. And she can help in that same way, because she can acknowledge that people are having a hard time or they’re struggling; they’re in pain. And we’re not trying to fool anyone into thinking that they’re talking to a person or anything like that at all. But that doesn’t mean we can’t offer support and empathy to someone, even through these virtual means, these virtual avatars. And if you can help people feel better emotionally, and connect with them emotionally, there’s tons of research to show that that will give them a better health outcome. People are social creatures. There’s more and more research that comes out that says one of the keys to long life is having lasting good relationships with people. And I think we can build on that. And it applies not just to romantic relationships, but really to all kinds of relationships.

Let’s talk about data, because it’s really about data. How do you think that influences your design choices? How do you think that plays a role? Everybody talks about big data, and you seem to have found a way to really take that data and make it tangible from a design perspective.
I think in my ideal world I would make most, if not all, of my design choices based on data. I’d get lots and lots of users in, I’d do lots of A/B testing, I’d iterate, prototype, do all these things. But when you talk to most designers, most of us just are unable to do that. It’s very difficult to run lots of user tests, find the right people, etc. So I think what ends up happening is that as a designer, you have to become very comfortable in the more qualitative data world, not so much as the quantitative because it can be difficult to get enough quantitative data especially when you’re starting out.


What exactly does do, and what is your role there?
With, we have a virtual nurse. Her name is Molly, although we call her different things depending on where she lives, because people have a lot of feelings about what their avatar is called. It’s an app on your phone or your tablet, and as I mentioned earlier, we’re mostly focused on patients with chronic health conditions such as heart failure, diabetes, addiction, etc. And the way that we help these patients and their clinicians is that they can do things like a daily check-in with a patient, so Molly can help the person walk through those tasks every morning—take their weight, take their blood pressure, answer some questions—and all that data gets fed back to the clinician. But the nice thing about it is that our system will flag when something is wrong. So let’s say you’ve got 5,000 patients. Well maybe only a few of them are having a problem, and you need to know which patients are having a hard time, because that’s who you need to call. That’s whose medication you might need to adjust. And by allowing that sort of triage system to occur, more people can have someone looking after them, caring for them, and hopefully prevent bigger medical events. The hope is to keep these patients monitored, engaged, following their health routine, and make them healthier and happier, and of course save money.

How do you build emotion and that emotional trust with the user? I mean it’s clear, now, what you’re describing; people feel that. They feel like they’re communicating. It is that emotional resonance. What do you think are some of the things that you’ve done that have created that?
So this conference I was at recently, the Virtual Assistant Conference, talked a lot about that. They talked a lot about personality and emotional engagement. And the thing is that humans are going to ascribe personality and emotions to anything. We do it with our pets, we do it with inanimate objects—it’s something that is very much a part of us. So then we wanted to get more explicit around that. So we tried to make sure that we design her personality and what she says and the way she says it to be empathetic and caring and acknowledging. So I would try to make sure, for example, that we have conversational fillers: I ask you a question, and you tell me an answer. I’ll probably say, ‘Thank you.’ And then I might ask you the next question. So we don’t want her to be robotic; we want her to use common human conversational cues. For example, when you’re asking a bunch of questions, you might say, ‘Well my first question is this.’ And then at the end you might go, ‘Finally, I’m going to ask you this.’ And all these little conversational cues that humans do very naturally, I try to consciously program into what Molly is going to be saying to the user to make it feel a little more like a real conversation, because that’s going to build more emotion and more engagement.

How do you think women contribute to health care-related design in particular? Do you think there’s a difference in how women design vs how men do?
I definitely have thoughts about it. I truly believe, very strongly, that women are socialized to always be thinking about the needs of others and looking out for other people. It’s very common that women are more often caregivers, they’re more often nurses, home care nurses; it’s usually women who end up taking care of their elderly parents—I mean, again, this is stereotyping, obviously. Some women are not very caregiving and there are lots of men who are, but I think typically women are socialized to take on that role, and it’s a very thankless role that women are kind of put into it. And because of all that experience that women have had over all this time, I think that anyone with that experience is going to design a better app in this situation because they’re not just going to think about it from the doctor’s or the clinician’s perspective. Somebody with experience caring for someone, or being more on the patient-facing area is going to have additional insights and–building that into the design–is going to end up with a final product that’s more emotionally aware, that’s more engaging. You’re going to have these two apps that are functionally, perhaps, the same, but the one that’s going to have this more emotional component and the more engaging component is going to be the one that succeeds.


What are some of the design trends that you see today that you think are great and what are some of the ones that you’ve already had enough of?
As far as the virtual assistant realm goes, in my experience, 99% of them are female, and if they have an avatar they have a young, sexy woman avatar. If they have voice, they have a sexy voice. And I just want to say to all virtual assistant designers out there, just stop. Let’s not make the default assumption that an assistant must be female and subservient and sexy. Your user base is not all 14-year-old boys, so let’s move on from making those avatars.


In terms of design trends I want to embrace and continue, some of what I was talking about earlier with speech recognition improving so much, I think we are really on the cusp. We are in the infancy of these conversational systems. I mean yeah, we have the phone systems and things like that, and we have Siri and Google now, but they’re not really conversational yet. But the speech technology has gotten so much better that we can start to build them now. So the more that designers can learn how to design conversations properly, I think the easier some of the interfaces that we have are going to be able to get to use. So let’s keep figuring that out and keep designing for it.

Do you see design playing a prominent role in the future of your industry?
I hope so, and I do think that design is being treated with a little bit more respect. Google, for example, is a very engineering-focused company; it was built on engineering, and engineers are king, but that’s shifting a little bit, where they’re starting to recognize the importance of design. And I think where we’re getting to now is that there’s a lot more people who have titles in design. The title that I have now never would have existed five years ago. It’s a great sign that people are taking design a little more seriously.

Do you think that it’s possible to do this retrofit of design into a company’s DNA, or does it need to be part of the core of who the company is?
I think it’s possible. I like to believe that anyone and anything can change if so motivated. But it’s hard. You know, obviously if you come from an atmosphere where the leaders of the company are really, really pro design and have made sure to put that in place it’s going to be easier. But I think any company, whenever it wants to, can step back and say hey, okay, we’re going to take a breather; we’re going to actually listen to what our users are saying right now and then we’re going to feed back some of that into the design. And you could do that any time. The earlier the better, but it’s never too late, in my opinion, to just stop and decide that design is something your company cares about. I think it has to be buy-in, like if one person is saying this is a great idea and the other person’s saying, ‘Yeah, yeah, whatever,’ that’s not going to work. But if people are willing I think it could be done at any time.


Do you think of yourself as a creative person?
It’s kind of like I’m creative within constraints. And I think I’m creative in that way. If you give me a problem to solve, I’ve got the constraints around it. Now I’m going to be creative in how I figure out the solution.

Do you think that empathy comes more naturally to women than men?

I do think that it typically comes more easily to women, and again I think that is because women have been socialized from day one to pay attention to how everybody in the room is feeling. Is your friend upset because you didn’t share with her? I think girls are much more pushed into the ‘make sure everybody’s happy right now’ mode. Does everybody have a drink? Is everybody good? And I think that’s sort of a double-edged sword as a leader, because on the one hand, to be a good leader you do really need emotional intelligence, you need to know how your team’s doing because emotions are not something on the side for people’s work or relationships or anything. Emotions are an integral part of whatever they’re doing. If somebody’s having a hard time, feeling a certain way, and you’re oblivious to it, I think that’s going to hurt you as far as your leadership goes. But on the other hand it can get very tiring to always be aware of how everybody’s doing and worried: like, ‘Oh, that person’s not doing so good today; I wonder what’s up. I wonder if I can help them.’ And sometimes you just want to focus on your work and be oblivious to all that, and that can be hard.

So I have one question I always ask. Are you a hugger?
I am not a hugger. Actually I should ask you this question. What did you think I was going to answer that question as?


That you weren’t.
(Laughs.) What gave it away?


About the author

Doreen Lorenzo is Assistant Dean at the School of Design and Creative Technologies, and Founding Director of the Center for Integrated Design, both at The University of Texas at Austin.