After 70 years of publishing, Marvel Entertainment has built up an incredible universe of heroes, villains, and super teams—a sea of data that no mere wiki can organize. At long last, Marvel has embarked on a mighty quest of its own: to create an entirely new graph database and search system to conquer continuity malaise by visualizing each character across the Marvel Universe.
The time is now for a solution, and as Marvel cradles the newborn "Agents of SHIELD" and its upcoming Netflix tetrology, the company now depicts the same characters across multiple mediums, from comics to blockbuster movies. There’s a wealth of information out there, and it’s not pretty—holy massive backstory and hyperlinked contextually relevant other characters, Batman!
The problem, like with any massive chunk of data, lies in getting the right data pieces in front of users—but for Marvel, the question becomes a semantic exercise. Just who is the character Hawkeye?
Well, he’s Clint Barton, except when he’s not; erstwhile sidekick Kate Bishop and villain Bullseye have taken the Hawkeye identity. He’s a member of the Avengers, except when he’s not; he’s also been part of the Thunderbolts and West Coast Avengers. He got his skills performing trick shots in a circus, except when he didn’t: He got them as an agent redeeming a murder conviction in the Ultimate universe and as a Black Ops SHIELD agent in the Marvel films.
You can see how fans who want to dig deeper into their favorite characters could be, ahem, easily waylaid.
It’s telling that I checked the above Hawkeye information on Wikipedia and the fan-curated Marvel wikis...and not on Marvel’s own wiki, which is clumsily organized, uncertain whether it wants to be an infodump or a recommendation engine for Marvel’s Unlimited subscription service.
To be fair, no current website gives me a clear vision of a character across universes: To do so, Marvel has charged Peter Olson, the VP of Web and Application Development at Marvel Entertainment, and his team to start from the ground up. And that means figuring out a way to make all this data—every interaction between hero and villain across multiple comic titles over decades of publishing history—make sense.
Above is a powerful illustration of character connections, like a superhero social network with popular heroes exhibiting more centrality and heavily radiating connections outward. Notice Iron Man as the top-center spiral with thick connections nearly across the entire Marvel universe: Despite being surrounded by other well-known Avengers, Iron Man has appeared in many more books and interacted with many more characters.
Other popular names stand out: Spider-Man, Wolverine, and the X-Men, among others. Take note of the rogue spiral on the right: That’s the Ultimate Marvel universe, a rebooted, continuity-free version launched in 2000 with Ultimate Spider-Man serving as a heavy connection to the normal universe.
That’s just two universes colliding in a simple two-dimensional graph: Fold in the versions of characters from the Cinemaverse, video games, and television shows, and you’ll get an idea of the scale they’re trying to grasp.
"We want an uberframework—the words 'ontology' and 'taxonomy' get thrown around a lot," Olson said. "We want characters to appear as close to as possible from all their stories and iterations but, overall, we want the characters to bubble up to archetypes."
Olson believes that fiction is fluid when characters undergo change. This primarily applies to narrative arcs, but also to interpretations: from dark hero in the '40s to TV camp in the '60s to grim ‘n gritty return in the '80s, Batman retains his elemental and iconic attributes as an archetype of the DC Universe. Likewise, Captain America still has his shield, even when Steve Rogers is dead and the mantle is taken by former sidekick Bucky Barnes.
But then a Captain America movie got in the works with Rogers as the titular hero: Neither Rogers or Barnes as the star-spangled Avenger are right or wrong, Olson says, but explaining the complexity is difficult. Hence, the forthcoming website, a visualization tool for fans new and old. Hence, the rush for Marvel to show, not tell its users about its pantheon of superheroes.
Most databases are relational, most easily visualized as tables of rows and columns: When you enter a search query, like "all products that sell for more than $3 but less than $5," the search system returns absolute answers based on data properties. Marvel still has use for this kind of database that returns queries with solid, irrefutable answers, like listing all the issues they’ve sold for the above prices.
The new database, however, will run on graph theory, looking for relationships between characters, teams, and events. The graph above displays relationships between characters, which would be extremely difficult for a relational database that might look for superheroes but leave out villains instead of showing more abstract values, like how popular/visible a character is across Marvel’s comic titles.
This visualization uses the same data as the previous image, but color-codes teams as it maps character relationships: The X-Men are blue at the top, the Avengers in red at the bottom-right, Spider-Man in green at bottom-left, and Wolverine almost smack in the middle in purple. In the early '90s as Wolverine’s popularity skyrocketed, the comic community jokingly wondered how Wolverine had time to be in five or six comic titles every month.
With data, we can see how many titles nabbed Wolverine facetime. Track this across titles, across time, and patterns begin to emerge. That’s where the money is: Marvel won’t just be able to provide users with the most layered exploration of a character and his/her versions, it’ll intelligently recommend comics.
This is the much-sought key for comic universes: Instead of waiting for Amazon-style "users also bought" data suggestions, Marvel wants to track relationships within its vast library. Like the teen superhero angle of Runaways? Try Young Avengers. But wait, you just want to see more stories drawn in the style of original Runaways artist Adrian Alphona? The Marvel graph database will find an answer based not only on book similarities but nuanced metadata, like writer or artist style. Better still, it’ll do what the venerable ComicBookDatabase cannot: confidently propose a list of essential story arcs for the new fan.
And lo, Marvel’s multimedia empire strikes again: Aside from Sony’s death grip on Spider-Man, Marvel holds the rights to all its major characters, so their recommendations aren’t limited to subscriber-only comics on its Marvel Unlimited service. Let loose the hounds of suggested merchandise! Of course, this also means those ultra-streamlined character pages will become the most seamless portals to every character’s stories that the Internet has ever seen.
Olson’s vision for a graph database isn’t a niche product for one particular universe—it’s a structural framework for understanding the relationships between any universes. Like the best "who would win in a fight" cross-universe nerd debates, Marvel’s graph database thinks about abstract similarities and relationships between people and entities. Why not apply it to historical figures? Why not apply it to find unseen similarities between compatible companies?
The applications could be even more abstract, says Olson, comparing every Google Maps street intersection to a "node" of data whose traffic is chalked up to that of nearby intersections. That’s how Google Maps plots the fastest route; why is it crazy that Marvel could plot the fastest route through suggested titles to convey the essence of each hero?
Olson and his team have chatted with others working in the intersection of comics and data, like Comics.org’s attempt to create a massive pan-universe database. They’ve also talked to Schema.org, an organization dedicated to making the web more semantic so massive search engines can bring up better search results.
Since comics are basically periodicals, Olson and his team were just going to use the periodical schema for their content—but one didn’t exist, so Olson and his team wrote one. The spirit of the comics community is great, says Olson—they really want to get their hands dirty with organization and answering the question, "How do you represent comics? How do you represent material?"
As pioneers of digital comics, Olson is proud of Marvel’s commitment to harnessing graph databases to create the supreme experience for Marvel fans. It’s easy to see the dollar signs pushing Marvel’s progress, but the quest to translate Marvel’s colossal data store into a native, novel service for fan exploration is easy to rally behind.