Why Science Research Labs Are Getting Into Video Game Development

Players of a video game called EyeWire have cracked a 50-year-old mystery about the way we see. The technique is catching on.

Why Science Research Labs Are Getting Into Video Game Development
[Image via Wikimedia Commons]

The tangle of nerves and ganglia behind your eyeballs is like jungle, so it was no trivial task when Princeton computer science professor Sebastian Seung and a team at MIT set out to map them in 3-D. They were hoping to learn more about the eye’s transmission of data to the brain, but along they way they were stymied by their own inadequate software. So they built a video game, called EyeWire, in the hopes of getting a community of untrained people to virtually map our most delicate and beloved organ.


It worked. With the help of the game’s 120,000 or so volunteer players over the last year, Seung’s team was able to clear up a question that has perturbed science for 50 years: Just exactly how do the nerves in the eye process visual information?

The answer was in the 3-D model, now complete: Retinal neurons sense motion in much the same way that computers recognize speech. You can read the findings here, published in the journal Nature last month. It was largely thanks to EyeWire’s players, who, with no formal training in computational biology, helped produce a scientific discovery. They were citizen scientists, contributing to science by playing a video game. So it’s no surprise this model of research might be catching on.

Scientific Research and Visualizations

The ability to create 3-D images has always been a commodity in modern biomedical science departments. Coders routinely clean up and analyze digitized, high-quality CT and MRI images. Libraries, like VTK and ITK, have helped numerous researchers reveal fresh information about the body’s structures through image rendering and visualization. Back when the libraries came out, finding competent coders to cross over from computer science departments to medicine was a delicate question.

Now, finding developers to staff labs might become more commonplace. Seung’s team is currently looking to hire more developers for EyeWire. And the team has brought public attention to the utility of citizen science games in furthering research. The White House even invited the EyeWire team to share its experiences with other scientists, signaling a growing public interest in citizen science games.

Getting the right people to create the games might still prove difficult. Experienced developers may find the pay difference between industry and academia off-putting. And it is hard to pinpoint one, specific profile of a potential applicant since the citizen science area is relatively new.

When asked what the MIT team was looking for, Will Silversmith, full stack developer in the Seung Lab, said, “We’re just looking for someone who’s, like, very talented.” It is easier to start out vaguely then setting one’s sights on a person who has equal parts biology and computer science know-how, someone who might not exist.


These Citizen Scientists Aren’t Scientists, But Gamers

EyeWire’s user experience is centered on the game. The EyeWire gamers’ contribution to the Seung Lab’s research was attributed to no more than coloring in neurons with their computer mouses. But it was important to the researchers. At the end of the day, the EyeWire team sees the players as extensions of the lab.

“Some do just want to play, like, a casual game, and we have seen some of those. But we have seen that almost all the players are very driven by the fact that they want to contribute to science,” says Silversmith.

The EyeWire team takes the design of the game seriously. “We do a lot of looking at other online games. We’re looking at, say, the on-boarding experience, or looking at how they display leader boards, or looking at, even small things, you know, when you see the pixel size of the completion bar,” says Amy Robinson, creative director of EyeWire.

“We’re moving toward more flat design and sort of going toward more low-poly style animations of the cells that the players are reconstructing,” she says.

Click to enlarge

Simplifying EyeWire’s landing page took great care. An earlier version of the page contained information about the group’s research and the goal of participating in the game. A player had to confront all of this information even if all she wanted to do was sign up to play. All the links distracted people from the game.

So Robinson and the team decided to take everything away from the landing page except for the log-in and registration fields and a colorful image. In all, the team spent a month redesigning that one page, and they have doubled their registration rate since then.


Science Communication Through Design

Infusing the EyeWire site with intense graphics is another large part of its design ethos. “I think, often, design decisions are plated as just being beautiful. They’re also ideally dually functional,” says Robinson. Good design attracts people to the project and gives them a way to share the science with others.

The game’s interactive graphics were written in WebGL. Silversmith says that WebGL is useful because it lets the browser use the graphics card of the user for the displays. This JavaScript-based technology lets online games, like EyeWire, thrive since comparable graphics quality would have previously been found in desktop-only applications. Other graphics were written with the library three.js, and the rich color images were made with Autodesk’s 3ds Max.

The workflow among the designers and developers varies from day to day. “Basically, we build out a feature and refine with a designer. And then we keep, sort of, iterating the design with the working feature until we get something to beta-test with our players,” says Robinson.

Citizen science games are just one way for science to appeal to a non-scientific audience, like gamers. Other modes of communication may be just as effective to get the public interested in scientific research.

“One thing that I think is really useful for almost every lab out there is, like on the design side, communicating how science works and what it is,” Robinson says.

The winning infographic, created by Chris Whittaker and Laura White of Ashfield Healthcare Communications, and Craig Armstrong of CreativeFusion.

Designers in a research lab could help create a simple animation and create visuals, like an infographic. Last month, EyeWire and partnered with the microscopy company FEI to host an infographic competition, to find the best graphic to communicate the scale of the brain’s structures. Entries came in until April 30th, and the winner was announced on May 15th. Like EyeWire, such infographics make science that much more accessible to the wider public.


Human Power vs. Computer Power

Seth Cooper is the creative director of the Center for Game Science at the University of Washington. The center’s main work tasks are to innovate games to promote citizen science and understand how data from players could improve video games, in general. Cooper and his colleagues mainly output scientific discovery tools, similar to Seung’s EyeWire, and early math education apps.

A few years ago, the Center for Game Science helped put out FoldIt, a citizen science game where players compete against one another to fold proteins into their most stable configurations. Fast Company honored the game on its Most Innovative Companies list in 2012.The University of Washington’s biology and computer science departments collaborated on it, recognizing that people could sift through the complex structure of a protein better than a computer could.

In general, people are thought to be better at recognizing visual patterns than computers, and FoldIt’s creators hope that its players will live up to the claim. They are currently collecting data on how well its players fold proteins compared to existing computer programs, like Rosetta. The players would ideally complement the computer’s computational power with their own skills.

“It’s basically trying to combine humans and computers together to solve problems that neither humans nor computers would be able to solve alone,” says Cooper. He says that humans have the problem-solving and spatial reasoning skills to do the high-level structural analysis, while the computer would just do the low-level finishing touches, like placing the atoms in their optimal location.

Moving From Desktop Games To The Web

The FoldIt creators largely based the game on the software Rosetta, which predicts the 3-D structures of proteins computationally. But this method is slow since the computer has to search through a random space of possible solutions to find a protein’s best configuration. Eventually, Rosetta evolved into the screensaver app Rosetta@home, whose users donate their unused CPU time to run the protein-folding program when their computers go idle, for example. Still, crowdsourcing computer power did not make the make the protein-folding process go faster.

FoldIt aims to let humans take over some of the thinking. Cooper and his colleagues wrote FoldIt to interface with the computations in Rosetta, which were largely written in C++, and chose OpenGL over DirectX to render the protein graphics because OpenGL could work easily across platforms. The interface was ready for these citizen scientists.


But FoldIt is a desktop application, where, when you open it, a ton of files have to be downloaded onto your system, which also complicates sending updates to the user. “It is a pretty big download and install process to get started,” says Cooper.

Newer developments in gaming tech have merged with the web. The center’s newest game, Nanocrafter, uses web-based gaming technology. Nanocrafter, which focuses on synthetic biology problems, works entirely in the user’s browser. Gamers do not need to download any extra software onto their computers. It was written with Flash’s Stage3D and ActionScript3 and the Starling library, all of which allows the computer to do the computations of the individual molecules in parallel to the game play. Cooper and the team took a totally different approach to developing Nanocrafter than building FoldIt.

EyeWire, FoldIt, and Nanocrafter are just some of the citizen science games out there. Other notable ones are FunSAT from the University of Michigan and Hope College’s Pebble It and Algoraph, but there are more.

“It seems like computational biology is a pretty fruitful area for these kinds of things,” says Cooper, since the games play on the spatial reasoning skills of players. “We might still be in a phase where we need to see where this approach is going to be most useful,” he adds.

It takes an especially motivated developer to convince a scientist that he or she belongs in the lab. Silversmith applied three times to finally secure his position on Seung’s team. And he even read Seung’s book, Connectome, before applying.

On deciding to leave industry for academia, Silversmith says, “I realized that there were very few things that have any meaning. While I wasn’t sure what those things were, I resolved to find those things. I eventually decided it would make sense to study the thing that decides what is meaningful, the brain. I also figured that if I was wrong, there was so much cool stuff going on that at least I wouldn’t get bored.”