At yesterday’s Google I/O we heard about the latest batch of products and releases coming out of the Googleplex this year. And while Google’s engineers are hard at work cranking out new features, another team at Google is busy dreaming up what to do with them. Google’s Data Arts Team is an elite group of creative coders, led by creative director Aaron Koblin, whose sole task is to imagine the most inspiring ways to use Google technology.
They’re the team responsible for the 100,000 Stars project and that one HTML5 Arcade Fire music video everyone loved so much. They were the first to envision and experiment with the potential of WebGL with another interactive music video, Three Dreams of Black, and continue pushing the capabilities of web technology through the popular Chrome Experiments website.
But why does Google invest in the development of what essentially amounts to a bunch of art projects? Because they know that getting developers excited to build on Google technology is the key to the company’s success, and the Data Arts Team is tasked with showcasing the creative potential of these tools.
We spoke with the Data Arts Team’s Creative Director, Aaron Koblin, and Creative Producer, Val Klump.
How do you decide which experiments to showcase?
Val Klump: There are two kinds of Chrome Experiments on the website. There’s the kind that users submit, or creative coders submit, which is like 95% of the projects, maybe 98% of the projects. And then there are some that we do ourselves at Google. The ones that our users submit tend to be one person who does it as a side thing, whereas we do it professionally so they’re much bigger projects. And we also have a kind of "in" with the Chrome team so we can see what’s coming down the pipe as far as technology. Our team tends to try and build big flagship projects that show what the newest features in Chrome can do creatively. So with The Peanut Gallery, for example, we knew that the web speech API was coming.
What was the inspiration behind The Peanut Gallery and who came up with it?
Aaron Koblin: It was really a team effort. We knew that Chrome was going to be releasing the Web Speech API, and that it's incredibly fun to play with. We were talking about a lot of different ideas—there's so much that can be done with voice control. I brought up how fun it is to talk over films and completely change what the people were saying. Originally we were thinking about subtitles and foreign languages, but ultimately silent films and inter-titles really felt like the best way to go.
What’s the purpose of these projects, apart from just being really cool?
What’s the relationship like between the engineering team and your team of creative developers?
Aaron Koblin: One of the best parts of working at Google has been the dialog between us as developers and the engineers who are building the technologies we're experimenting with. Often we're using the technologies while they're actively being developed and we can help identify potential issues and provide example use cases which may not have been on the radar. There have been several times where learnings from Chrome Experiments have started conversations about how Chrome itself should function. One example I remember from the very early days of Chrome was a window tiling spirograph that I put together. It spawned well over 100 Chrome windows and moved them around the screen in rapidly changing patterns. I excitedly showed it to a number of folks only to find, quite soon after, an update which limited the number of windows—a smart move for the user.
Do you ever open-source your projects?
Val Klump: We frequently do. Sometimes it’s a little too complicated because our code is a mess. Sometimes we release the code open-source and we clean it up and put it on GitHub. But not always, because it’s a lot of work to do it. Usually if someone asks for it we send them the code. That happens occasionally.
What projects do you have coming down the pipeline?
I can’t talk about any specifics. I can say that there are some technologies that we’re particularly interested in. Web video is one. There’s been some really nice work done by a number of developers with web video. And then WebGL is always another. I think we’ve only scratched the surface with WebGL. It’s been out for two years but it still hasn’t hit the mainstream.
What about Web video is so exciting?
Web video lets you use your webcam as an input device. It’s not just streaming video but you can actually use that stream and analyze that stream, so you can look at all different colors, you can look at brightness levels, and you can play with all that data and do all sorts of cool stuff with it. People have done things like however bright the pixel is, they can extrude that pixel, so that you get all this 3-D depth in the image. You can play with time and color, so one guy did a project where the RGB colors in your webcam update at different rates resulting in a weird collage effect where one color will move when you move your face and then the others will follow. There’s some really cool stuff that’s been done with that. But I don’t think we’ve even scratched the surface creatively. There’s more we can do.
Why do you think Google invests resources in creative projects?
We want to show what the web is capable of doing. It’s not just about reading articles online or checking your email but it can be a really amazing, interactive platform creatively, too. We see this as a platform for the future, for everything, not just communication but creativity. That’s why we’ve done projects with interactive film, interactive music videos, some games, like gaming is going to move more and more to the web as the browser becomes more capable. Google sees it as a platform for much more than it is now. And we’re trying to showcase that with the Chrome Experiments.
What are some of the biggest challenges that you guys face when working on these projects? I mean obviously working with experimental technology can’t be easy.
That’s definitely the hardest part. For the web speech API, no one had done any creative projects with it before. And for Exquisite Forest, no one had ever built an animation tool before that had that kind of scale and needed that kind of stability. And it’s a real struggle. Once you’ve done it, you realize, "I could have saved so much time." It took us maybe four or five months to build it and then once you’ve done it you can build that in maybe two months. Because you haven’t done it before, and because there’s no template of other projects out there that people have used that we could use to just make a shortcut to launching a project, we have to build it ourselves.
That’s a massive undertaking. Is it rewarding?
It is, because we’re frequently the first people to do something with that tech. I like to think we sort of set the standard for what it’s capable of doing. And then obviously our hope is that people will surpass it. Which happens sometimes but it doesn’t always happen as quickly as we hope it would.