For any streaming video provider, a big event like Game of Thrones might have been nerve-racking.
HBO, however, says it came prepared for the final season and its 4.7 million concurrent streaming viewers across both HBO Go and HBO Now. Season 8 of Game of Thrones was the first in which HBO relied entirely on its own streaming tech, having cut ties with Disney-owned BamTech a year earlier, and the network is calling it a success. HBO says video playback issues only made up about 7% of its customer support requests over the final season, and most of those came from Latin America, which has its own version of the HBO Go service.
“We’re very proud of the fact that we’ve moved all our customers from BamTech’s platform onto ours, and we’re able to handle a cultural phenomenon like this without any hiccups,” says Anderson Imes, HBO’s vice president of engineering.
The bigger test is yet to come. At the end of this year, AT&T’s WarnerMedia plans to launch HBO Max, a new streaming service, anchored by HBO and fleshed out by other content from the former Time Warner, which AT&T acquired for $85.4 billion last year. AT&T CEO Randall Stephenson has said that the service will become its “key video product” in the fight against Netflix, Disney, Amazon, and others, and Imes says his team is building the platform around HBO’s technology. While a streaming service’s underlying tech may not seem important, it becomes instantly noticeable when things go wrong. Hence the gloating about how Game of Thrones went right.
In late 2014, HBO rather infamously abandoned plans to build the stand-alone HBO Now streaming service with its own tech. Instead, the company signed on with BamTech—then called MLB Advanced Media—believing that it would deliver a more polished platform in time for HBO Now’s launch the following spring. Otto Berkes, who had been building the in-house streaming tech as HBO’s CTO since 2011, resigned when the company announced its decision.
Imes describes BamTech as a “great partner,” and has no examples of the company failing to implement any of HBO’s wishes. But by running its own streaming service, HBO can be the sole arbiter of when to add new features or how much to invest in them.
“I think the thing we had always been worried about, since the beginning, was making sure we had enough control on our platform,” Imes says. “Having that be a platform that’s first party, and we can control our destiny and invest in it, makes sense for the business.”
HBO may also have been wary of Disney’s dedication to running other companies’ streaming services with BamTech, which is now called Disney Streaming Services. Dan Rayburn, a principal analyst who covers streaming media at Frost & Sullivan, believes Disney acquired a majority stake in BamTech in 2017 primarily to power its own streaming services, such as Hulu, ESPN+, and the upcoming Disney+. Although BamTech still has outside clients, such as Twitter and Sony’s PlayStation Vue, it’s starting to shed others, including HBO and WWE.
Related: HBO Max is launched
“There’s definitely companies who are going, ‘Hey, BamTech’s now owned by Disney, Disney’s going to compete with us, and the writing’s on the wall,'” Rayburn says.
Turning to Amazon’s servers
Unlike, say, cloud computing power, streaming video tech hasn’t quite become a commodity. A company with more than 4 million concurrent streamers cannot simply call it a day after purchasing the requisite number of servers and content delivery networks. Instead, streaming video services must navigate a patchwork of video players, server setups, and testing methods to build their own tech stacks.
While developing its own service, for instance, HBO decided to stop running its own data centers and instead rely on Amazon Web Services, whose servers can automatically scale up to provide more cloud computing power as needed. Still, Imes says those systems don’t respond fast enough to handle Game of Thrones, which has 60% of its audience signing on to watch within three minutes of airing.
“There’s no auto-scaling system on earth that will add the capacity that we need,” Imes says, “so we’ve had to figure out how we pre-scale our systems for an event like this, how we ensure that no matter how many additional customers per second are being added to the system, we’re always ready.”
To test its system, HBO enlisted a firm called Apica to simulate lots of “virtual users” signing on and streaming video at once. Leading up to the Game of Thrones premiere, HBO would gradually test more demanding scenarios, and simulate potential issues such as regional service outages to make sure its system can keep up.
“When we start to we see a system that goes unhealthy in an unexpected way . . . we’ll stop the test, we’ll analyze the data that we collected from that, and we’ll put together a set of fixes that we need to put into the system,” Imes says.
Making a name
Dan Rayburn, the Frost & Sullivan analyst, says that most of what HBO is doing has become fairly standard among major streaming services. But by having its own tech, HBO and WarnerMedia can better measure and fine-tune how streaming performance is affecting the business. Doing so requires both technical expertise and a large budget, though, and Rayburn says HBO’s work is a sign that the direction of the company has changed.
“HBO’s got a lot of subscribers,” he says. “It’d be great if people think of them as one of the leaders in the market for the video stack, considering a lot of people just sort of wrote them off once they got rid of everything and put it over at BamTech.”
HBO may still have to prove itself in other areas. Unlike Netflix and Amazon, HBO still doesn’t stream any video in 4K with high dynamic range, or HDR. Imes says it hasn’t been a priority because the format requires a lot more bandwidth, and because HBO doesn’t have much 4K programming available yet anyway. But with Apple TV+ and Disney+ both promising 4K HDR for their respective streaming services, HBO’s 1080p streaming could become a liability.
It may have even cost HBO some criticism already, as one of Game of Thrones’ Season 8 episodes appeared overly dark and muddy for many viewers. Imes says 4K HDR TVs may have exacerbated the issue by trying to upscale HBO’s 1080, standard dynamic range picture.
Still, Imes is hoping HBO will at least get some recognition for smoothly handling an event that has no equal in terms of how many people are logging on and streaming at the same time. He remains a bit worried that some industry watchers believe that BamTech deserves the credit.
“I kept on seeing articles on the internet talking about how well BamTech did this year,” he says, “and it kind of made me a little sad.”
Correction: An earlier version of this story misspelled Apica, the firm that provides virtual user testing for streaming services.