How DARPA, The Secretive Agency That Invented The Internet, Is Working To Reinvent It

The agency has launched a program that takes advantage of the computing resources of smartphones, tablets, connected vehicles, IoT endpoints, and more.

How DARPA, The Secretive Agency That Invented The Internet, Is Working To Reinvent It
[Photo: Flickr user Mark Eslick]

Is the internet becoming obsolete?


The government agency that invented the network that runs the world seems to think so. So the Defense Advanced Research Projects Agency (DARPA) has launched a program aimed at “completely rethinking how to network and compute” by taking advantage of the computing resources that have begun to saturate the world around us in the form of smartphones, tablets, connected vehicles, iInternet of things endpoints, and more.

DARPA’s Dispersed Computing program (or DCOMP, as it’s known) adds another moniker to the set of emerging technologies that includes fog computing, edge computing, and distributed computing. DCOMP takes these paradigms one step further, however, and envisions a network that is able to borrow processing and communications resources from its many nodes as and when needed to accomplish whatever task its users might throw at it.

While today’s internet is fine for everything from liking pictures of your BFF’s breakfast to swaying elections, the Department of Defense, which runs DARPA, will point the DCOMP program at transforming the battlefield first, enabling warfighters to collect, transmit, and process information even in situations where access to a remote data or operations centers—or even a reliable laptop computer—cannot be counted on.

The military now relies on networked information processing more heavily than ever before. And leadership at the highest levels of the Pentagon is said to be focused on an overarching strategy that seeks to connect all the military’s devices—from the most basic sensor on the front lines to tanks, aircraft, and specialized devices in the field, to overseas operation centers and massive data stores back home—into a single networked organism in which any node is able to share, access, and act on information that is global in scope (given the right security clearance, of course).

But such a concept relies on a communications and processing network that can handle such a dynamic mission. And the internet, arguably, can’t.

[Photo: Flickr user Pierre]

A Radical Departure

“Today’s network is pretty static,” says Petros Mouchtaris, president of Vencore Labs, which is exploring programmable networks for DARPA as part of the DCOMP program. “The computing capabilities we have in our hands, at what we call the edge of the network, have really changed dramatically since the internet was invented. At same time, the core technology of the internet hasn’t really changed at all.”


Updating that core technology, according to DARPA, will require overhauling the very stuff that knits the internet together: its protocols. “Melding computing into communication is a dramatic rethink of the models and architectures we have become accustomed to,” says Jonathan Smith, DARPA’s program manager for DCOMP. The TCP/IP protocols, in use since the early 1980s, are good for getting information from one place to another, but that’s no longer enough.

“DCOMP is a radical departure from conventional network architectures,” Smith told me in an email. What DCOMP envisions is “the opportunistic use of network-embedded computing resources,” he says. Rather than thinking of the devices on the networks as nothing more than nodes that pass information from source to destination, DCOMP sees them also as distributed computing resources that can be dynamically reallocated in real time, depending on the changing needs of the current mission set. Imagine every cell phone, smart thermostat, fitness tracker, and game console in your house contributing their spare cycles to help process the video you’re trying to upload, or educate the machine learning algorithm that runs your AI personal assistant. When a dispersed computing network wants to borrow your phone, in other words, it’s going to be doing a lot more than sending a text.

On the battlefield, where soldiers may not have reliable access to the internet, being able to dynamically share computing and communications resources can be a matter of life and death. And the volume of those resources is only growing. “I think the ubiquity of devices and wearables on the battlefield really is a leading indicator of how things are going to evolve there,” says Peter Levine, a general partner at venture capital firm Andreessen Horowitz, which has invested in battlefield technology companies like Shield AI and others. “The human-to-device ratio may be one human to 100 devices, or one human to 1,000 devices. Understanding how that information is processed and utilized will be an interesting next equation that will likely need to be solved.”

A Self-Aware Network

“Today at the edge you have very capable computing devices, and there’s a lot more data you have access to, a lot more cameras, and when you talk about soldiers, there are [unmanned aerial vehicles], and a lot of other sensors,” Mouchtaris says.

To understand how and when to allocate all those devices’ computing and communications power, the network will need to be able to examine itself, to a certain extent. “We’re working on creating a network which can sense very quickly what’s going on in the network, and can react very quickly because it’s programmable,” he continues. Though the work is still in the design phase, Mouchtaris envisions a set of protocols that will change on the fly to better support current needs and conditions in the field.

“If you have video to process and send, you want the network to behave differently than for distributed computation or if you just want to send voice or big files,” he says. “Especially on the battlefield, the capacity of network links changes very quickly. Vehicles move around, soldiers move around, they get into places they can’t access the network. The protocols should automatically adapt.”


One of the challenges, Mouchtaris says, is to help the network avoid navel-gazing. “When you try to make things adaptable, there’s a lot of work you want to do to check how things are,” he says. “That generates a lot of overhead. You want to very quickly find out about what’s changing in the network and adapt quickly, but you don’t want to flood the network with requests for information about what’s going on.”

[Photo: DARPA]

Tapping Inventors For A Reinvention

While field deployment won’t happen for a couple of years, expect the technology to find its way from the military to commercial use before long—provided it works as advertised. So will we all need to switch from the internet to DCOMP once it’s in place? DARPA’s Smith says the more likely outcome is that “DCOMP will first find service as an overlay on the IP internet, and if the overlay’s advantages are compelling (as were the internet’s), the overlay’s features will gradually be added to the underlying infrastructure.”

“There have been several revolutions in communications and network topology, but in terms of mobile communications, this is the next one,” says Kevin Kelly, CEO of LGS Innovations, which is also working on networking technologies for the program.

Both LGS and Vencore have their origins in the storied Bell Laboratories, which is credited with inventing or developing everything from the laser to the Unix operating system, some of the earliest transistors, and information theory itself. Many of these developments, of course—including Unix and the transistor—were built on earlier work, and DCOMP will be no different. “It’s not practical to design a solution that requires 100% innovation,” Kelly says. “We’re challenging our folks to find practical and efficient ways to utilize as much current technology as possible, and invent in a sequential manner the technologies that are needed to enable the ultimate solutions.”

“Even though DCOMP is in its early stages, it promises to be transformational,” Smith says. Mouchtaris agrees. “What we’re talking about with dispersed computing is a completely new internet technology that’s much more suitable to today’s environment, and future environments,” he says. “It’s hard to predict what great new things will be possible, but we’re working on an infrastructure that provides a much more advanced internet than today, and which will hopefully fuel the next generation of apps that will change the world.”

At Andreessen Horowitz, Levine is looking for similar developments: “I’m waiting for the entrepreneur to come in who blows us away with the idea that there’s some next thing that needs to be done relative to [dispersed computing]. I don’t really know what it is yet, but when I see it I’ll let you know.”


Mark Wallace has written for The New York Times Magazine, Wired, Salon, and many others. He lives in San Francisco.