The Cloud Is Getting A Lot Closer To You

By now, most people have at least some vague idea of “the cloud”: A constellation of giant server farms spread across open countryside in places like rural Oregon, Nevada, or the Carolinas. From storing Facebook photos or Netflix shows to interpreting what you tell Siri or Alexa, most of the internet’s work gets done in these far-off silicon sweatshops–their products delivered to you in fractions of a second.

But even with high-speed mobile connections, that’s too slow for emerging technologies—like preventing robotic cars from running into each other, or superimposing augmented reality graphics on your view of the world in real time. So the cloud is extending out to smaller clumps of computing, to buildings in each city and even big boxes by the side of the road–the “edge” of the internet that sits within a few milliseconds of the car, phone, robot, or whatever future gadget needs instant computing beyond what it can do on its own.

Cloudflare, the San Francisco company that provides security and speed-optimizing service to websites, today announced that customers can now run basic software on a global network of over 100 data centers that, it says, are already within 10 milliseconds of 90% of the people on earth. In addition to the servers on its own content delivery network, the company plans to add many more mini datacenters, potentially thousands, even closer to people and their gadgets.

Other tech companies will likely follow—Amazon Web Services already lets customers deploy functions at edge locations on a content-delivery network—but these are still early days for edge computing.

“It really is pretty much a startup environment,” says Kelly Quinn, the research manager for datacenter trends and strategies at IDC. Whether or not the new service, called Cloudflare Workers, becomes a major player, Cloudflare’s journey to the edge is a case study of how a convergence of cheap hardware, 5G wireless, and artificial intelligence is remaking the internet.

The Networks Inside The Internet

You may know the name Cloudflare from the recent controversy over extreme right-wing, hate-filled websites; some of them, including a neo-Nazi site, used Cloudflare to block denial-of-service attacks. (Bowing to pressure externally and internally, Cloudflare dropped the site on August 16.) Those kinds of sites comprise a negligible portion of the more than 6 million customers Cloudflare’s services protect, which altogether account for 10% of global internet requests. More people pass through Cloudflare’s system by perusing OkCupid dating profiles, getting customer service through Zendesk, reading articles on The Daily Caller, or checking scores on Brazilian site Gazeta Esportiva. Many Cloudflare customers are themselves internet plumbing companies, like email service FastMail, web site host SiteGround, or cloud server provider DigitalOcean.

Cloudflare sits between clients and the open internet, absorbing the onslaught of a distributed denial of service attack, for instance, or adding security modifications to the encoding of emails and web pages (aside from a failure earlier this year when a software bug caused it to leak customer data).

Along with competitors such as Akamai, Cloudflare also “caches” copies of online content in data centers closer to the people accessing it, making it quicker to reach. Cloudflare has servers in in 118 locations around the world, with plans to reach 150 by early 2018; thus far, they’ve been dedicated to moving or protecting data, but not to running software.

“Up until now if you wanted to write code that ran across our giant edge grid network, you had to be a Cloudflare employee,” says CEO Matthew Prince. Applications for a private beta version of Cloudflare Workers open up today, and the service will be available for anyone in early 2018, says Prince. Cloudflare Workers will initially support apps written in JavaScript, with support for other programming languages later.

Related: The Debate Over Neighborhood Zoning Could Hold Up Fast 5G Wireless For Years To Come

Cloudflare is also teaming with Austin startup Vapor IO, which plans to launch thousands of mini data centers around the U.S. They’ll be at the base of or close to cell-phone towers, courtesy of Vapor IO investor and cell tower operator Crown Castle.

“The average enterprise colocation company might have, in the U.S., six to 10 locations. Crown Castle has 40,000 locations,” says Cole Crawford, Vapor IO’s founder and CEO. The company is preparing by the end of the year to launch its mini data centers–cylinders about 9.5 feet tall and wide called Vapor Chambers—in Austin, Chicago, and Pittsburgh (an autonomous car tech Mecca). Crawford says it will announce yet more planned cities in about a month.

Cloudflare will install its servers in some of these data centers, but it won’t say how many. “We’ll probably work with Vapor in certain instances,” says Prince. “We’ll work with others in other instances.” Likewise, Vapor IO has reserved space for other companies providing edge services, such as Packet, which rents out cloud server capacity to customers by the hour.

5G and Edge: Like Peanut Butter And Jelly

Cell towers are a natural place to put edge servers, for several reasons. So much of the internet’s growth will be in mobile devices: Not just smartphones and smartwatches but delivery drones, autonomous cars (and planes), augmented reality experiences way more detailed than Pokémon GO, and billions of “internet of things” devices like wireless temperature or moisture sensors.

Drones and cars need fast connections and fast processing if they are to coordinate in real time to avoid a crash. And while a lot of sensor data isn’t urgent, it’s going to come in torrents. Analyzing and condensing that data to basic insights at the edge will prevent the internet from getting swamped with mostly worthless bits and bytes.

Most of this starts to make sense with the transition over to 5G wireless, which could begin as soon as 2019 in the U.S. (which is leading the world on adoption). The 5G standard aims for delay (or latency) of just 5 milliseconds, and the ability to go as low as 1ms for mission-critical uses like drone control. That’s compared to today’s 4G networks with latencies pushing 100 milliseconds in some cases—enough lag to wipe out the benefit of fast computing at the cell tower. 5G also has a low-power, low-speed mode to handle all those trickles of data from internet of things devices.

There’s another reason why 5G and edge computing go together like peanut butter and jelly. The advent of 5G completely changes the way wireless networks are built, replacing the specialized hardware in cell towers with virtual “hardware” that runs as applications on a more versatile computer server, says Vapor IO’s Crawford. “So now what exits on the top of the tower is just a bunch of antennas. And the rest of that—the radio, the network interface card, the motherboard–is [run on generic computer] hardware. That looks more like a data center than it does telco gear.”

Related: The Wild Technology That Will Make 5G Wireless Work

In 5G, multiple cell towers will be linked over fast fiber optic cables to the servers running virtual tower equipment. Vapor IO piggybacks onto this setup, providing space for companies like Cloudflare and Packet to install their computers.

“I think it’s a really wise strategy on the part of these folks,” says Kelly Quinn. Vapor IO could even house the wireless providers’ own servers in its Vapor Chambers, the company says.

These clumps of cell towers, with their own mini cloud services, can handle time-critical operations like coordinating autonomous cars, independently, without having to touch the larger internet.

“[Autonomous cars] operate as a system, and that system has to have awareness of different cars,” says Prince. “If you have two cars that go at each other, if they both swerve in the same direction, then that doesn’t do any good. If that coordination has to backhaul from wherever you are on earth to some central datacenter, then it doesn’t work.”

Edge computing within these local nodes is the internet version of a reflex reaction. You kick instantly when someone taps your knee because the “decision” to kick comes from a few neurons in your spine, without sending a signal up the backbone to your brain. Likewise, these mission-critical decisions can happen at the edge, without sending signals up the internet backbone to some “brain” in a giant, far-off server farm.

It’s Not Too Early To Think About Robots’ Needs

Autonomous cars, let alone gaggles of them traveling together, are still several years off—and so is an edge network that could begin to coordinate them. And the biggest uses–like healthcare or financial services–currently don’t necessarily even run over wireless networks.

Many hospitals don’t have computing power to analyze radiology results such as MRIs, for instance, says Quinn of IDC. And the speed of processing data at the edge could be very lucrative for the high-speed automated financial trading operations of today’s stock market. No wonder that Don Duet, formerly Goldman Sachs’s head of technology, took the job as Vapor IO’s president and chief operating officer.

The first Cloudflare Workers apps will be nuts-and-bolts things clients develop to improve their services, such as resizing images at the edge so they can travel faster and cheaper over the internet. A customer called RankScience, for instance, makes automated tweaks to web pages, testing which modification will improve their ranking in search results. It will start doing some of this work at the edge of the network for faster results closer to web surfers.

Prince also envisions Cloudflare customers improving upon its own products. “If someone builds a better firewall, and they want to run it across our network, even though that would be competitive with a product that we have, I think that’s good for our customers,” he says.

The really interesting ideas are the ones he hasn’t thought of, Prince learned recently while interviewing a job candidate who suggested Cloudflare build a smartphone that’s as powerful as the latest models, runs five days per charge, and costs $50. The phone would be a “dumb terminal,” she said, with all the storage and processing on edge servers and just a screen with a simple wireless connection to display the results. That would leave room to pack the phone with batteries.

“Now I don’t think we’re going to build the phone, but I think it’s really neat if we’re building the network that allows someone else to build the phone,” says Prince. (He hired the candidate.)

Although Prince talks a lot about running AI apps at the edge, Cloudflare and other content delivery networks don’t even have the hardware for it yet: Its servers lack the appropriate graphics processing units or other specialized AI chips. Prince says he’s waiting to see what technology to go with. While Nvidia’s Volta chip is currently the de facto standard, Google provides AI-specialized chips called TPUs, and Intel will introduce its own AI chip, code-named Lake Crest, later this year.

But it’s not too early to think about sci-fi edge applications like AI control of drones or robot cars, says Quinn. “Having the conversations and hashing out the issues prior to any one company developing a solution is rather important,” she says. “Because those [applications] are much more sophisticated in terms of the data generated and transmitted and the importance of latency. So those are going to have to wait until all the kinks get worked out.”