A Head For Detail

By: Clive ThompsonWed Dec 19, 2007 at 8:14 AM

Gordon Bell feeds every piece of his life into a surrogate brain, and soon the rest of us will be able to do the same. But does perfect memory make you smarter, or just drive you nuts?

Enlarge

A few of the things on Gordon Bell's mind: From ancestral photos to his MIT diploma to his robot driver's license, it's all in there. And we do mean all.

But the truly intriguing part about Facetmap is that it shows how Bell's information is connected. I start poking around, clicking on Gemmell's blob, then drilling down to a particular email Bell sent him on February 25, 2005. As I zoom in, the software automatically creates new blobs showing everything else Bell did on that day--emails to other colleagues, photos Bell took, Web sites he viewed. It feels like flying freely through Bell's memories, flitting anywhere I want. And it re-creates those same loosey-goosey linkages that tie memories together in our real-life minds. Were this my own computer, I could zip back to read a Wall Street Journal article from three months ago, then teleport over to all my email from that day, then notice I'd forwarded the article to someone along with a couple of cool ideas, ideas I'd forgotten I'd ever written. It's like software for productive daydreaming.

Facetmap is based on a truth psychologists have long understood: We organize our memories by time and people. Those categories, Czerwinski says, are the pathways into the forest and the portholes from one memory to another.

"The way you remember things is associative," she says. "You think, 'There's all this stuff in my life that's related to Clive, or to Gordon.' Or you think, 'There's all this stuff that happened last fall.'" If you vaguely remember a book but can't recall the title or author, the first thing you're likely to begin with is the friend who told you about it; so you hunt through all the emails you got from him.

One of Czerwinski's colleagues at Microsoft, Susan Dumais, analyzed how people search for things on their computers and found that about one-quarter of the queries were for someone's name. She also explored ways to use "landmark" events to index our memories. "You'll think, 'Oh, I'm sure it happened right before the wedding, or just after Hurricane Katrina,'" Dumais says. She developed an experimental piece of software that embedded those landmarks into search tools so that you could, say, start with a major event and then see all the email or Web pages you looked at that day. It drastically improved the ability to find things in the distant past, she says.

These sorts of tricks are already helping Bell find his memories. Gemmell has written a piece of software that works much like Facetmap. It's less graphical (it looks more like a regular Windows search) but just as powerful, as Gemmell illustrated with an example during my visit to his office: At one point last year, Bell had considered selling some property, so he surfed a bunch of real-estate Web pages and chatted with Connie, his broker. A few months later, he wanted to revisit those pages but couldn't figure out the right key words to bring them up. He had Connie's number, though, so he located the copy of his call to her. Then he checked the rest of his activity for that call, and presto: There were the Web sites, too.

"It works via how your memory works," Gemmell says. "It's like, 'I don't remember the specific words on that real-estate page.' My memory is just, 'It's the page I saw when I was talking to Connie.' We have to make 'search' more like the way we actually think."

Instant Replay

I came away from these demos eager for this stuff to come out of the lab and into the world. But the fact is that our everyday search tools are already moving slowly in this direction. In the past year, free "desktop search" programs by Google and Microsoft--which scour your hard drive--have begun incorporating sophisticated filters that let you work in similar ways: You can start by looking for a person, then find all the memos you've written to them, then quickly zoom in on a day.

One day when I met Bell in San Francisco, I got a chance to see his life through his eyes. He'd worn his SenseCam to work that day, and when he plays the images back rapidly, it's like watching a crude, stop-motion movie: buying coffee at Starbucks, grabbing a paper, entering his building, and finally dropping down at his desk.

"If you lose your keys, you can scroll back and figure out where you put 'em," he jokes. In fact, Bell seems like the sort of guy who might lose his keys a lot. He'd regularly get halfway through a sentence, then cut himself off and race along another tangent, only to have his hamster brain veer away halfway through that thought, too. The guy's obviously crazy-smart, I thought. But no wonder he loves having a camera record the messy details of life.

Yet here's the problem with the pictures: They pose an even bigger search dilemma, because computers can't "see" the contents of a photo. It's impossible for Bell to hunt for "pictures of my desk at work," or "that tall blond guy I met at the party"; at best, he can sort them by date or GPS coordinates. And while he has added keyword "tags" to many shots, it's time-consuming and still not terribly accurate. Even he admits he rarely peruses any of his thousands of SenseCam pictures.