Brian Whitman landed at Spotify just in time. The MIT Media Lab alum and machine listening expert joined the product team at Spotify early last year when the streaming giant dropped a reported $100 million to acquire The Echo Nest, the music data company he cofounded a decade ago. Since his time at MIT, Whitman, along with cofounder and fellow PhD Tristan Jehan, has focused obsessively on the intersection of big data, artificial intelligence, and music, using that sweet spot to try and redefine music discovery in the age when songs flow freely like water and new artists pop up by the hour. Today, he’s sitting across from me in a conference room in Spotify’s New York headquarters showing me what his team has been building for the last few months.
It’s called Fresh Finds. At first glance, it’s just another playlist–it could easily be one of the collections hand-curated by Spotify’s 32 music editors, or one of its 75 million users for that matter. But Fresh Finds is different: The weekly playlist is generated using a chunk of the predictive big data technology that Whitman and his team at The Echo Nest brought with them to Spotify last March. The mission of Fresh Finds is to identify under-the-radar artists that are generating buzz online and surface the ones most likely to break out.
“Fresh Finds is a distillation of the hippest users on Spotify,” says Whitman, pulling up a list of 38 tracks projected against the conference room wall. “These are the artists that are going to break out soon because they’re being listened to by these people.”
I don’t recognize any of the artists on the list. Neither did he, Whitman admits. But now many of them have made their way into his daily rotation. “Just wait a few weeks and people will start talking more and more about them and they’ll take off.”
How does he know? The machines told him, naturally. Fresh Finds takes a central component of The Echo Nest’s original methodology–its web content crawler and natural language processing technology–to mine music blogs and reviews from sites like Pitchfork and NME and figure out which artists are starting to generate buzz, but don’t yet have the listenership to show for it. Using natural language processing, the system analyzes the text of these editorial sources to try and understand the sentiment around new artists. For instance, a blogger might write that a band’s “new EP blends an early ’90s throwback grunge sound with mid-’80s-style synthesizers and production–and it’s the best thing to come out of Detroit in years.” If this imaginary act goes on tour and writers in Brooklyn dole out praise of their own, the bots will pick up on it. It helps address an issue some people have voiced early on with Apple Music, that its selections aren’t adventurous and it tends to recommend things you already like rather than things you might like.
After generating a list of talked-about artists, the Fresh Finds algorithm checks that list against the listening habits of a subset of Spotify’s users that the system has determined are “ahead of the pack”–that is, the many thousands of very active listeners who tend to discover new artists before they break. In early, internal testing, Whitman and his team say that this approach, in addition to being clever, actually works. He shows me a collection of analytics charts that compare an artist’s “buzz” factor (as determined by the blog-reading bots) with its listenership on Spotify. Sure enough, in many cases, a burst of online chatter about a new artist is followed by a spike in listening.
“Music discovery has been sort of a wild west for me personally,” Whitman says. “It’s now a solved problem to be able to recommend Coldplay to somebody. We’ve been doing this for 15 years now and I’ve always wanted this kind of feature: Brand-new stuff nobody knows about.”
That isn’t say that the Fresh Finds is perfect. It’s most glaring blind spot is the subset of music that is new and talked about, but not yet available on Spotify. Most new, unsigned artists upload their tracks to services like SoundCloud and BandCamp before bothering to spend money on a broader distribution service like TuneCore or CDBaby, both of which push an artist’s music to dozens of streaming services. It’s a problem Whitman says they’re trying to resolve, but it may be tricky without smoothing out the onboarding process for new, unsigned artists (some of whom remain skeptical of the music subscription business model anyway).
Whitman is also eager to add some kind of explanatory text around these new tracks. As we go through the songs, he tells me about Night Court, a brilliantly named band with an album called Law & Order comprised of music inspired by 1980s cop dramas. That sort of contextual detail will be missing from Fresh Finds at the start, but once it’s added, it can only serve to tighten the bond between listeners and the new artists they discover here.
Still, for a version one product, Fresh Finds feels remarkably spot-on. This type of insight, which would be very difficult for a human being to mimic single-handedly, is the result of marrying predictive machine intelligence with data about what Spotify’s 75 million users are listening to. It was why they acquired The Echo Nest in the first place.
The timing could hardly have been better. Just two months after the Echo Nest acquisition was announced, another company in the online music space got snatched up. Apple spent $3 billion to gobble up Beats Music, whose star-studded, if not yet widely used streaming service put music discovery and curation front and center. On June 30, the company relaunched the all-you-can-stream service as Apple Music, an exceptionally well-curated service that reportedly topped 10 million subscribers in the first month of its existence (although it’s worth noting that all of those users are taking advantage of a free three-month trial, so it’s a bit early to gauge its success). However the numbers pan out, Apple Music appears to be Spotify’s most formidable competitor to date. From here on out, features like music discovery and exclusive content are what’s going to set these services apart from one another.
“I think that the music catalog is more or less commoditized at this point,” says Matthew Ogle, a senior product developer at Spotify focused on music discovery. “If you can do the deals and set the service up, you can stream the same music that we have or that Apple has to millions of people. But something we’re really keen on is establishing a relationship with our users and evolving that over time. It’s the same way you would get to know the staff at your local record store.”
Ogle, a veteran of the online music discovery space who founded the song-sharing site This Is My Jam and spent five years in web development and product at Last.fm, joined Spotify in January. Among the first projects he worked on was Discover Weekly, another semi-automated, playlist-based music discovery tool that Spotify launched two weeks ago.
Like Fresh Finds, Discover Weekly uses large-scale data analysis and machine learning to craft a weekly playlist designed to surface new music to listeners. But unlike Fresh Finds, Discovery Weekly is personalized.
“Our goal with Discover Weekly was to make something that felt like a friend or someone who knew you well was making you a mix tape and saying ‘Hey, here’s some music for you to check out. I think you’ll like it,'” Ogle explains. “But, it turns out, sitting down and making 75 million mixtapes every week is not something that humans could literally do themselves. So we thought, how do we scale this?”
Once again, it’s a combination of machine intelligence informed by human behavior on a massive scale. In this case, Spotify observes the way actual people make playlists. Since its launch, Spotify has been used to handcraft over 2 billion digital mix tapes in the form of playlists. By looking for patterns in how people piece together those lists of tracks–the system ignores playlists containing a single album or an artist’s entire discography–the technology that powers Discover Weekly can unearth new aggregate insights about the less-than-obvious relationships between various songs.
Discover Weekly analyzes your own listening history, with an emphasis on what you’ve been bingeing on recently. It then takes that knowledge and compares it to the playlisting behavior of other users in the hopes of tapping into that distinctly human, gut feeling that dictates why one song sounds good following another or why this particular collection of 35 songs feels perfect for your mood on a given afternoon. Scanning millions of playlists, the system tries to find tracks that are commonly listed alongside the music with which you’re already familiar, and then group those tracks together into a new, personalized playlist. Effectively, it takes the classic “people who like that also like this” logic of collaborative filtering and applies it to the process of making a mixtape for somebody.
Anyone who has ever thoughtfully crafted a playlist of music for a friend or been tasked with DJ’ing a summer-themed party knows how this works: You can spend hours selecting the perfect songs for the occasion and then painstakingly labor over the precise order in which the songs should play. There’s no rule book for this process; it sort of just comes from the gut as you piece the songs together. Tracks with a similar vibe go back-to-back and maybe the lower-tempo jams find their way to the end of the list. It’s this unique, barely conscious instinct that Discovery Weekly is trying to tap into.
“Man vs. machine is no longer a useful distinction in terms of how we build stuff,” says Ogle, echoing a sentiment I tend to hear coming from the discovery-focused folks at Spotify. “This is an algorithmic recommendation but it’s humans all the way down. People are inspiring what’s happening.”
The end result is a list of 30 songs by artists you may have checked out once or twice, a couple names you’ve seen before and some you’ve never heard of. On the whole, at least in my experience, it’s a pretty solid playlist that feels relevant without being too obvious in its recommendations. It’s not perfect. Indeed, Ogle and the rest of the team are already eager to make improvements like letting people train the system with Internet radio-style thumbs up/down voting. But on the whole, Discovery Weekly does feel like it could have just as easily come from a friend as from the algorithm that generated it.
Tuma Basa knows how hard this is to pull off. As part of Spotify’s curation team, he is tasked with programming and overseeing the service’s hip-hop playlists. “Rap Caviar,” a user-generated playlist that Basa took over and rebranded, now has 2.1 million followers.
“Rap Caviar” is just one of the many Spotify-branded and curated playlists that now live alongside countless user-generated playlists under the service’s “Browse” tab. These genre- and activity-based playlists are taking on an increasingly important role in Spotify’s curation strategy, especially now that Apple has hit the curation nail so squarely on the head. Meanwhile, Google’s $39 million acquisition of playlisting service Songza last July should have left little doubt that curation and discovery would become one of the next big battles in the streaming music space as the gatekeepers disappear and the world is flooded with new music. And while computers are getting way better at understanding music, they still can’t out-curate human brains like Basa’s.
“It’s not just putting together a playlist,” says Basa, whose resume includes hip-hop programming stints at MTV, BET, and Puff Daddy’s Revolt network. “It’s feeling the playlist. It’s that emotional quality control. It feels like a body of work, like there was filtering that took place. It represents what’s happening in the culture right now.”
As machines pull in billions of different signals about music from user behavior and the Internet at large, flesh-and-blood programmers like Basa still mine the music landscape the old-fashioned way: by attending shows, keeping an ear to the street, talking to people in their music industry network.
“I pay a lot of attention to the background,” says Mjeema Pickett, the head of R&B programming for Spotify. “I pay attention to who is singing in the background. A lot of artists I’ve come across that way.”
“Oh, and mixtapes too,” Basa chimes in. “Mixtapes are the NCAA of hip-hop.”
That isn’t to say that Spotify’s human curation efforts are 100% analog. These guys are tapping into things like Fresh Finds and Discovery Weekly as well, in addition to music blogs and other digital sources, both internal and external.
“We listen to data as well,” says Doug Ford, Spotify’s director of music programming. “Everything is human-picked, but it lives or dies by the stats within the company.” For instance, if a track on a workout playlist gets skipped more often than the same song it does on an R&B date-night playlist, Ford and his team know which use case better suits the song and can tweak the lists accordingly.
But with so many competitors in the music subscription space these days–most notably, Apple–Spotify has its work cut out for it. My first day testing Apple Music, I was blown away by how well the app seemed to know my tastes. Despite barely ever using iTunes for anything, Apple Music knew to deliver me a hand-curated list of Krautrock deep cuts, as well as a playlist based off Madonna’s early albums. An oddball pairing, but both things I love. Apple’s playlists can get delightfully hyper-specific, like “Indie Hits From 1994” and “Neo-Adventures In Neo-Psych,” again both things that seem hand-picked by my best friend to grab my attention.
This, combined with Apple’s massive install base and marketing might, is what Spotify is now up against. Daunting yes, but the Swedish startup is wasting no time ramping up its curation and discovery efforts, presumably among other features. Apple may have an army of expert music curators around the globe, but Spotify is building out its own curation brigade, backed by 75 million users–not to mention the team of developers, data scientists, and product gurus toiling away in Spotify’s labs.
“Our secret weapon is the fact that we have so many passionate music lovers on the service for so many years,” says Ogle. “That means that the next generation of people trying out Spotify can benefit from all of that curation and usage. I see personalization as a huge part of not just attracting users, but keeping them.”