What Do Dreams Mean? This Big Data Project Could Crack Them

An experimental sleep app backed by renowned sleep scientists could change the way the science of dreaming is done.

What Do Dreams Mean? This Big Data Project Could Crack Them
[Image: Flickr user Charles Dyer]

Since the discovery of REM sleep in 1953, scientists have been waking people in sleep labs to understand how and why we dream. These artificially captured dreams are ones you would have slept through otherwise, not the natural dreams that affect a person’s waking life. Not to mention that waking people up one at a time doesn’t exactly yield a fast-growing dataset. For those reasons and others, dreaming has remained mostly a mystery–but a new project could change that.


What scientists have always needed is a tool to gather natural dreams on a huge scale. Dream reports have traditionally been classified by hand, using a taxonomy system from the ’50s. Only recently have algorithms been used to filter reports automatically for patterns, much the way word tracking tools have been applied to literature. But it’s hard to be certain of “norms” in dreaming without enormous samples, which (as we said) have been nearly impossible to collect.

An amped-up alarm clock app on your smartphone may not seem like a scientific tool. But that’s what top dream researchers have signed on to support: Shadow: Community of Dreamers is a mobile app, crowdfunded with $50,000 on Kickstarter, which will wake people, collect dream reports by typing or talking, anonymize them and beam them into a searchable, analyzable online set.

“If we want to make the world’s largest database of dreams,” says Hunter Lee Soik, Shadow’s 31-year-old founder, “We need the world on our side.”

Why Everyone’s Dreams Matter

What’s at stake is not just philosophy, but public health: Nightmares are a common symptom of anxiety, PTSD, and depression, and disturbed sleep leads to attention problems, weight gain, productivity loss, and sexual dysfunction–so dream science’s impacts are material.

Sleep research has been hampered by the lab setting, though: Limited people have their brainwaves read while they sleep, and what’s worse, the dreams reported in labs tend to be different than natural ones: less emotional and narratively complex.

Sleep is big business, too: From books to relaxation tapes to pills, the market has grown to a $32.4 billion industry, increasing 8% annually. Bad dreams plague victims of abuse, grief, or divorce, children of broken homes, and the regular walking worried, ruining sleep quality.


What sets Soik’s ambitious app apart from other motion-sensitive alarms like Sleep Cycle is the team of scientists he has recruited. Data miners in fields ranging from neurobiology to clinical psychology, at universities like Harvard, MIT, and Berkeley, have joined his team, planning to harness the power of people charting their own dreams to get numeric insight into what the world is feeling–what one researcher called “a thermometer of the world’s mood.”

Shadow may be the next step of the “quantified self” movement–the data visualization of a person which emerges by recording our running or biking mileage, our diets, entertainment and information consumption, our social lives and family trees. Soon, our devices may measure our brain waves or heart rates, personalizing the process of medicine and making us aware of medical and psychological factors we’ve never known before. By giving people access to their dream patterns, and a worldwide network of scientists an unprecedentedly multidimensional dataset of dreams, Soik hopes his app will become “the human interface between the technology wearables and the platform, the dashboard of the self.”

Dream-Fishing from Kyoto to New York

I know personally that Soik is not alone, because I’ve spent years asking these same questions. I was a dream researcher in a brain-imaging lab in Japan a few years ago.

My boss, Yuki Kamitani, had a similar goal to Soik’s: gathering dreams, like a bug collector’s butterflies, and classifying them for science. But what we did with brain scanners, old-fashioned pen and paper, algorithms, and a spreadsheet in Excel–eventually yielding our dream content “neural decoding” results, published in Science–Soik is proposing to do with smartphones and the World Wide Web.

“Now we’ve got this Instagram-like experience, which is curated by Shadow,” Soik explains. “You say ‘Last night I dreamt of a mountain,’ [into your phone], and there’s a beautiful picture of a mountain. From there you can click and listen to the audio about the mountain dream, or read about it. We want to create a mechanism where if you read somebody else’s dream and you have a similar dream, you can say ‘I’ve had a dream like this,’ and then you start creating content of your own.”

The notebooks beside my futon in Kyoto were filled with stories of people’s dreams–Korean, Japanese, French, Colombian, Dutch; dreams from Alabama and California, and many from my friends and scientific colleagues in the apartment complex where I lived from September 2009 to April 2010. When I went to work each morning, I’d code these stories into a database of dream content–outdoors, indoors, fear, sadness, animal, plant, et cetera–and my labmates and I would look for statistical patterns, using machine learning algorithms: Which images occurred the most often? Which locations? Which emotions? Then we’d tie these dream traits to patterns of brain activity in the people who slept in our lab.


The YouTube visualization the Kamitani lab produced, showing images for what dreams our algorithm predicted from people’s brains, above words from their reports–is not unlike what Soik plans for Shadow: keywords, extracted from a dream report, tagged to a picture.

“Let’s say you had a dream last night about a red car,” Soik says. “We’ll say ‘Here are all 208,000 other people who had a dream about a red car, and here’s every red car dream that ever existed. We want to have a social layer where people can start talking about and interpreting their dreams, pushing up the more relevant ones or the more self-reflective ones to the top. So you can feel like when you are dreaming, other people are too, you’re part of this bigger thing.”

If You Call This An Alarm Clock, You’re Missing The Point

What’s special about the alarm clock feature of Shadow was announced last week: the “Shadow antigraphy” or “smarter smart alarm” will use the motion-sensor exploited by other sleep apps, which guesses a person’s sleep stage based on mattress movement (since the body is paralyzed during REM, tossing and turning stops when you’re in this phase); but unlike those apps, Shadow will combine such motion data with self-reported grogginess, to personalize the best wake-up time to remember dreams, for each user.

That’s great, as far as phones-as-sleep-readers go. But any mobile sleep app is limited as a detector of sleep stages, as Shadow adviser and former Zeo research scientist Stephan Fabregras concedes in a promotional video: “Sleep doesn’t happen in the mattress, it happens in the brain.” So what will make or break Shadow is not the alarm, but the analytics.

How Are You Feeling, World?

“We can measure how global events affect mankind’s unconscious,” says Shadow adviser and Spanish neuroscientist Umberto León Domínguez, PhD, a researcher in the sleep and circadian rhythms lab at the University of Madrid School of Medicine’s Psychiatry department.

The wars in Iraq and Afghanistan, natural disasters, terrorist attacks, elections, and the World Cup are examples of events Domínguez thinks impact people’s dreaming. Data collected on Shadow will show scientists how events like births, deaths, celebrity marriages, and pop cultural breakthroughs like documentaries or marketing campaigns affect global dreaming, too.


“Wouldn’t it be interesting to see how the Breaking Bad finale last Sunday night affected people’s dreams?” wondered Kelley Bulkeley, another dream researcher advising Shadow.

Harvard dream researcher Deirdre Barrett, another advisor, has done research on how dreaming was affected by the 9/11 terrorist attacks, and more recently by the Arab Spring uprisings. If the Shadow app had been available, this is the sort of study that would have benefited: pairing time-stamped events on Twitter with parallel dream content from different countries and cities where social upheavals took place. The effect of personal events like divorce could also be studied, she says, if the database queries marital status.

“Are nightmares a predictive sign for depression or suicidality?” Barrett and others have wondered. Shadow data could help answer this question, as people who are tracking their dreams can report the time when they had a breakdown, or when they started taking a drug, and see how their dreams differed before and after.

The most clinical interest in big data dream science, she suspects, will be in PTSD. Nightmares, for many patients like combat veterans, rape victims, and survivors of terrorist attacks or natural disasters, are often the strongest form of symptom of the disorder. Some interventions specifically target these nightmares, like those described by New Yorker reporter Margaret Talbot at a nightmare clinic in Arizona. The progression of nightmares in PTSD and depression is another subject scientists already know something about, but could learn much more about from Shadow’s crowdsourced data.

”We’re Making A Wikipedia For Dreams”

The dreams will be searchable in terms of key words, mined by algorithms, and once dreamers add more descriptive words, these will be clustered semantically, Soik says. “Here’s every dream about a horse,” for example, “now here’s every dream about a black horse or white horse or a flying horse.”

The categories Shadow is focusing on are positive or negative–was the dream good or bad?–plus the gender, location, and age of the dreamer. Even this basic data will allow the network to answer questions like “After Hurricane Sandy, did older New Yorkers have more or fewer nightmares than young ones? Did people who lost their homes dream differently than those who did not?”


Shadow is also translating the website’s content into 12 languages, to make their mission statement global.

“We don’t want the messaging to be an interpretation of an American idea,” explains Soik, who was born in Seoul, raised in a Wisconsin town of 9,000 people, and has spent his adult life in L.A., New York, and Germany. “We want it to be a global conversation.”

Finding Normal

With an unprecedented number of dreams from all over the world, Domínguez says, scientists can learn what are the average rates of nightmares and sex dreams, dreams about crime or sports, flying, or elephants or anything else, from different populations. Establishing “normative data” is perhaps the most important contribution of Shadow’s big data to dream science.

Kelly Bulkely, PhD, a dream researcher at the Graduate Theological Union, and advisor to Shadow, has spent much of his career developing an automated system for coding dreams reports. This effort to quantify the seemingly subjective study of dreams was spearheaded years ago by cognitive psychologist William Domhoff at the University of California, Santa Cruz, in his online database of dream reports.

In a series of studies, Bulkeley and Domhoff have shown that algorithms aimed at keywords in dream reports, based on the Hall-Van De Castle system of dream codification, can derive accurate information about a person’s waking life habits (career, sports, hobbies); relationships and sexual status (are you dating or married? Sexually active or not?); and emotional status (engaged, stressed, bored, depressed?)–from nothing but the dream series of an anonymous stranger.

“For therapeutic purposes, I think dreaming serves as an emotional mirror,” Bulkeley says. “If you give me enough dream samples, I can tell you what are the major emotional issues and relationships in your waking life.”


He envisions a future where a therapist inputs a dream series from a client, gathered on Shadow, and outputs a series of graphs of how his dream characteristics vary from the norm for his or her age, sex, and cultural background–like the cholesterol, blood pressure, and genetic readings you get from a doctor. The idea is not to automate mental health care, but rather to leave the humans free to do what humans do best, and let computers take care of the boring stuff: to visualize patterns invisible to us.

Bringing the Human Back To Psychiatry

“Dreaming as a topic in psychiatry has fallen off the map,” Bulkeley says, explaining that he dropped psychology at Stanford to study philosophy and religious studies because psych no longer addresses the unconscious mind faced by Freud and Jung.

If Shadow can validate the quantitative study of dream content, and show reliable links between dreams and mental breakdowns or recovery, it may suggest a path toward linking the “old psychiatry” of analysis with the pill-dominated mental health care of today.

“In the intellectual elite, Freud has tainted the study of dreams,” Bulkeley says. Sleep scientists in the ’50s and ’60s “felt they had to overthrow Freud and create something new–a psychobiological framework for dreaming. Now that we’ve got that, we’ve got to reconnect with meaning and the human experience of dreaming. People don’t like it when you reduce that to zeros and ones and neurotransmitters.”

“We are developing a tool that no one has seen before,” Domínguez tells me over Skype from Berlin. “This will be the first time in human history we’ve had so much data on dreams. We don’t know what will happen next.”