For media companies, the pace of technological change can be unforgiving. The New York Times, well-staffed as it may be, is no exception. To meet rapidly changing business goals and reader expectations, the tech team there recently pulled off a redesign that required overhauling the backend of one of the biggest news sites on the planet.
As you might imagine, things got complicated.
By the time the project—code-named "NYT 5" internally—kicked off two years ago, the problems were clear. As web standards, mobile device adoption, and reader expectations all barreled forward, the NYTimes.com architecture had been gathering dust, resulting in hacked templates, messy code, and pages that didn't always play nice with the latest devices.
Even worse, the old system wasn't allowing the Times's developers and interactive journalists to iterate quickly, making it harder to deliver badly needed updates to impatient readers. These gripes, combined with business needs like increasing ad inventory, informed the first set of redesign mockups that starting make the rounds internally six months later.
"As a company we had all agreed that the fixed width layout that we were using from 2006 was no longer cutting it with plethora of different devices that are available today," says Mike Finkel, Executive Director of Web Products Technology at the New York Times.
"We were doing a lot of hacking to make our content work on our iPad app, iPhone app, and Android app. We all agreed that this was the time to do something new."
As editorial, design, and business goals all came into sharper focus, the true breadth of the project revealed itself. By the time NYT 5 launched earlier this month, it wouldn't just include the freshly polished exterior—everything under the hood would be new, too.
If you're one of the many people whose jaws hit the floor upon seeing the publication's Snow Fall story last year, consider yourself somewhat lucky. At the Times, this type of multimedia-rich layout has historically been born in spite of their publishing system, not thanks to it.
Indeed, like many of the immersive web stories that followed, Snow Fall was built alongside the Times's content management system, independent of their standard online editorial workflow.
It's not just the interactive blockbusters that demand these kinds of arduous technical backflips. Smaller multimedia projects have historically employed a variety of template hacks and workarounds too. By overhauling the site's infrastructure, the tech team took the first and biggest step toward changing all that.
At the same time, even regular news articles were published in proprietary format, static HTML documents which lived in the Times's data centers. This less-than-dynamic system made it hard to maintain a consistent look and feel across templates over time.
As development manager Reed Emmons explained in a recent post on the New York Times Open blog:
By comparison, the new infrastructure is much more dynamic. The team completely rebuilt the PHP rendering engine that pulls every page together as users navigate the site. The new framework is cleaner, more efficient, and—crucially—architected in such a way that building on top of it in the future won't create the sort of technical debt that has caused so many headaches over the years. It will also make building new features and apps easier, allowing developers to glue existing components together rather than starting from scratch.
Rather than a brand new CMS built from the ground up, the editorial side was given a host of new features and capabilities, including a new publishing template and better control over multimedia assets. Part of that newfound freedom, Emmons explains, is automation that frees them from thinking too much about how the content is formatted. Among these time-savers is "an algorithm that paints images down the page in a way that makes sense contextually."
For the editorial side, this is just phase one. Over time, publishing interactive mega-features like Snow Fall on-the-fly will become much easier.
Never in human history has a technology been as quickly adopted as the smartphone. Indeed, by the time the NYT 5 project kicked off, "responsive design" was already maturing from buzzword to best practice. The Boston Globe launched its own cross-device-friendly design in 2011, with many other major publishers leaping onto the bandwagon.
"We leveraged tools that have been very successful already and sort of glued them together," says Emmons. "There was a fair bit of tailoring to meet the needs for our organization, but all of the core principals of it are existing libraries that already exist and are well supported throughout the industry."
"There's probably a good hundred different ways that our articles can be rendered," says Emmons. "To serve all these interesting scenarios we needed a more custom solution. That's why we rolled our own responsive framework to get all these different breakpoints starting at 768 with the iPad portrait mode all the way up to 2030."
The mission of the Times's new responsive engine goes beyond the obvious need to deliver web-based news to iPads and Samsung Galaxy devices. It also allows the site to make intelligent decisions about rendering content based on the type of media elements displayed, the orientation of the device, and even the monetary value of the advertisements on the page.
Of all the tools utilized throughout the project, few were quite as handy to Emmons and Finkel as Varnish, a front-end caching layer that helps with speed and scalability.
"Varnish is a reverse proxy cache with a lot of customizability as far as like the types of things it caches, how quickly, how long it caches these things for, and whether we incrementally purge those cache entries," explains Finkel. "It's one of the biggest reasons we were able to make a lot of the more creative decisions on the backend. We knew we had this fantastic front-end cache layer to rely on."
Between all this front-end streamlining and the rebuilding of how the server-side architecture works its magic, the New York Times site should, in theory, be noticeably faster to readers. Just as important, though, is the efficiency with which developers can now code new features atop a freshly rebuilt, modern platform.
"We standardized everything so there's one way to code and it all fits into our framework," explains Emmons. "You could be a junior developer and understand the structure. It's about solving the product and business issues at hand rather than trying to figure out what data structure to use. We want to allow them to be creative with their solution without necessarily having to think about how to do it."
This is a huge deal. The reason other publications were so easily able to beat the Times to the punch with things like responsive design is because its rusting architecture made it harder to push new iterations out in response to evolving reader habits. The expectations of those readers will continue to change at an unrelenting pace. The difference is that now, the code and developers behind NYTimes.com will be in a far better position to adapt.