Designing Software And Hardware At Once

Integrated experiences are inherently both physical and digital, but “integration” doesn’t mean doing everything at once.

Designing Software And Hardware At Once
[Image: Flickr user Anil Jadhav]

Digital goods are increasingly being bundled with physical goods to make them easier to merchandise. FastCo.Labs brought together three designers at the NYC-based creative agency Code and Theory to discuss what makes an integrated user experience worthwhile. Here they explore why the convergence of digital and physical products–from smart TVs to newspapers–has prompted a need for a smarter approach to multidisciplinary design.


Dan Gardner, cofounder & executive creative director UX: So, the whole point of why we’re debating physical and digital design is because we feel like each represents one thing. But both invoke a feeling. Both invoke an emotional response.

Mike Treff, managing partner of the product design group: I actually concur, but I also think there’s definitional issues with this title. I don’t think it’s about what came first. It’s more about: When did the duality between physical and digital actually start to actually matter?

Gardner: Look at TVs. Today, you can do more than just browse content. So this discussion is more about understanding how products with physical and digital components–like TVs–are starting to evolve into a much more interactive and meaningful experience.

Geoff Baldwin, lead industrial designer: When you think of digital interaction, you generally think of something on a screen. What’s so ironic about the lack of successful TV interaction design is that at its core, TV is just a giant screen. So the question becomes: Why is the interaction design on TV so poor, and why is the physical hardware design so poor?

Treff: Historically, the TV experience was purely consumptive: up, up, down, down to get to the content you wanted to see. As the TV experience became more interactive, and included experience elements like On Demand, Pay Per View and second screen interactions, the industry’s response was to solve most interaction problems with hardware solutions. So now you’ve got a lack of congruity between the hardware and the interaction design that the hardware is meant to manipulate and display.

Baldwin: Right. Every time you added a feature to the content, you needed another button on the remote to keep up with the UI.


Gardner: Well, you didn’t need it. That’s just what happened.

Baldwin: Yeah, it’s what happened. So now, it’s gotten to the point where you have a million buttons on a remote. Thankfully, the digital interface is starting to catch up with the physical interaction. Designers now have the opportunity to completely rethink what a remote control is and what a digital entertainment experience should be.

Treff: So the question becomes: At what point does the interplay between the digital and the physical become critical to the product experience?

Gardner: I think it started with the fact that we have different types of screens for every single situation we’re in. However, the user expects a consistent experience across all screens. How I hold and interact with the physical object that surrounds the screen—that expectation is becoming a blurred line.

Baldwin: It’s funny. It used to be that we designed products and worried about what the logo looked like on the physical product. Now, the physical product is becoming the talisman for the digital experience.

Gardner: If the experience is the identity, then part of that experience is the screen, and part of it is the physical object. I think people are still slow to catch on to this.


Baldwin: Yeah. It is weird. Look at the history of branding: Your brand used to be a mark. Then, it became a product. Now, branding is all about how people use a product, like Google–

Treff: Exactly. The verb of “Googling” is the brand.

Gardner: It also goes back to the purposefulness of good design. If you design with purpose, you will, of course, get an aesthetically pleasing design. But at the heart of it, you should be solving a usage problem. The design solution has to extend to both physical and digital. It’s one in the same.

Baldwin: Exactly. One becomes a catalyst for the other. And it’s not always only in one direction.

Treff: Agree completely. I think this is analogous to newspapers. In the past, when newspapers were just physical products, everything that affected the reading experience was centrally located: Designers and content creators were all in the newsroom. It was a much more integrated process. The end product had to be done at a certain time, and then it was shipped to a printer.

When newspapers went online, the digital team was siloed away. Then the digital ad team was siloed away. Then the technology team was a thousand miles away. Suddenly, everyone was segmented and there was no person to sit at the locus and make it cohesive on the input side so that it was cohesive on the experience side.


Gardner: Right. The point is that creativity can’t be limited to one silo. Frankenstein systems don’t really work that well.

Baldwin: One thing we always say is that design isn’t finished until it’s developed. Sure, you can 3-D print anything, or render something in minutes, but fulfilling the physical design still takes blood, sweat, and tears, just like it always has. You can’t fulfill it with just strategy.

Treff: So, for physical and digital design–how should the approach be consistent? How should it be different?

Gardner: In the ideal sense of all design, they should be moving in parallel. You have design and engineering happening at the same time, and you’re learning from both and making decisions along the way.

Treff: It’s like when we do a tech assessment in the definition phase of a project. We don’t just look at the platform. We look at things like: What are the right tools? What’s the right stack? What’s the right framework? It’s really critical work that happens before we design anything.

Gardner: Obviously, there’s no perfect solution. Ultimately, the hardware shouldn’t be separate from the digital screen, the digital front-end shouldn’t be separate from the engineering, and the physical hardware shouldn’t be separate from the mechanical engineering.


Baldwin: Exactly. Design is messy. You can’t be afraid to start at the end. You can’t be afraid to Wes Anderson a project. At the end of the day, you want to do the most efficient amount of trial-and-error as possible. And that doesn’t necessarily mean it’s going to be a linear process. Physical and digital design move on different timelines.

Gardner: The reality is you need a stimulus. The catalyst has to come from somewhere. It could come from the hardware or the software, but it has to be human. I think in any design process, you can’t be afraid to go back a step. Once you’re afraid to go back, that’s when you have a problem.

Baldwin: Actually, the car is a great example. Knobs and buttons are designed into the car before the content and user experience is even considered. So, it’s not just figuring out the digital experience, it’s figuring out the user experience: How do I interact with this thing physically, digitally, emotionally?

Gardner: Right. User experience isn’t digital interaction design. It’s user experience.

Look at the actual physical buttons and knobs–why do you need them? Maybe it’s because you can’t drive a car and look at a screen at the same time. The point is you have to figure out the starting purpose to the design. That’s the user experience.

Baldwin: Right. I don’t think we should think of a knob as a knob. We should think of it as a connection point to a digital experience. When you stop thinking about a screen as a screen, you can start to think about it as a digital canvas for a physical interaction.


Gardner: Yes. The output.

Treff: Physical and digital design should both stem from the user experience. It all needs to be cohesive. Bringing an idea to life in a way that is meaningful and productive is all about reducing complexity and focusing on simplicity.

Gardner: Right. It’s how you do it, and it goes back to having an integration philosophy. It’s hard enough to define what simplicity is. The process to get there is hard. You need an integrated approach, or else you’ll fail. You see that time and time again.

Baldwin: I think people mistake integration for doing everything. In reality, integration is about creating focus. It’s about creating products and interactions that have empathy for each other.

Treff: I love it when you say that. Every time it gets me really warm and fuzzy.

Baldwin: In the end, great design is when a product knows what it’s good at, the feature knows what it’s used for, and they understand how they fit together.