The world’s animals are getting their very own Facebook

The animal kingdom is getting its very own social network, but instead of vacation pics and baby announcements, it will be filled with data that could help conservationists save species from extinction.

Wildbook is open-source software built to keep track of wildlife. Researchers can upload images taken from the field, but a significant source of data comes from scraping sites like Flickr and YouTube for pictures and videos that people post from trips like whale watching tours and safaris. Using deep learning, Wildbook’s platform spots the same individual in different images, letting conservationists track creatures through their lifetime as well as gain a better idea of population sizes.

Other data can be added—like the animal’s sex, age, where it was location photographed, what other individual was nearby, or even the weather—that fills out what is known about an individual and its preferences. Each animal gets its own profile to track sightings.

The team behind Wildbook, the nonprofit Wild Me, currently builds a new site for each species but is working on the project’s next iteration—a massive database with as many different kinds of animals as possible. Individual Wildbooks have helped researchers better understand what’s going on with specific species. But give machine learning algorithms enough data and it could uncover previously unknown insights about what makes animals thrive or die. These discoveries will supercharge conservation efforts and benefit populations across the world.

“The vision is one Wildbook to rule them all,” says Tanya Berger-Wolf, a professor at the University of Illinois at Chicago and one of the cofounders of Wildbook.

From Whales To Clouds

The project was inspired by an effort in 2003 to track whale sharks. The tool, called Wild Me, used software designed by astrophysicists to recognize different pinpricks of lights in the sky. The group behind Wild Me realized that algorithm also worked great at telling apart whale sharks with their dappled skin that looked like the night sky.

But an algorithm was just the start. Since there was nothing out there like it, the team also had to build a data management system to host and share the information that conservationists, who aren’t all computer scientists, could use. This herculean effort became Wildbook.

Wildbook now relies on updated, AI-powered pattern recognition software to tell animals apart. It works a lot like the face recognition software on sites like Facebook, which has algorithms that compare points of the face to determine who it’s looking at. But since not all photos being taken of these creatures, especially marine ones, will prominently showcase a face, Wildbook applies its pattern recognition tool to other body parts. The researchers pick characteristics, like the stripes of a zebra, or the ridges on a whale’s tail, that commonly show up in images and are features that are distinct enough from animal to animal for machines to tell apart.

After years of development—and assistance from faculty and students at Princeton, Rensselaer Polytechnic Institute, and the University of Illinois-Chicago—Wildbook’s system can now be tweaked to work for a lot of different creatures. There are currently 12 Wildbooks for 30 different species—like giraffes, polar bears, and seals—with a backlog of more than 200 requests from marine biologists, ecologists, and others studying wildlife.

Being able to track individuals is essential for population counts and movement patterns, which in turn helps guide efforts like setting aside land, controlling predators, or even getting animals on the endangered species lists. Berger-Wolf says the system helps reveal social structures, breeding trends, and responses to environmental pressures like habit loss and climate change.

The biggest pressure on wildlife, of course, is us. The world is currently going through what scientists call the “sixth extinction.” While other extinctions were due to events like a major asteroid impact, this one is caused by people. Technology like AI could help make people a part of the solution, not just the problem.

An early proof of concept came from the Wildbook for Grevy’s zebras, one of the most endangered animals in Kenya. It showed that there were fewer juvenile zebras in Kenya’s Lewa Wildlife Conservancy than what was necessary to sustain a stable future population. Knowing that the nearby predators were doing well and snacking on young zebras at a high rate, the park’s managers put the lions on birth control.

Whale sharks also have had their population estimates improved. The world’s biggest fish have a migration that can go for 5,000 miles, which has made it tricky to get an accurate count. Their confirmed population went from a couple of hundred individuals to 8,500 with over 40,000 sightings. The paper that announced the new estimate had 36 authors from institutions and universities from Australia to the United Arab Emirates to Mexico.

“Not one researcher, not one institution, not one group can have all that information,” Berger-Wolf says.

While insights like those have helped, Berger-Wolf says they really need a “planetary-scale project” where conservationists can ask interdisciplinary questions like what species is where, how they change, where they travel, and how other populations are doing in comparison.

Wild Me, which is based in Portland, Oregon, has so far survived with a few developers on a small budget cobbled together with grants, taking donations and charging a fee for extra help on setting up Wildbooks. But a new, giant database will require a lot more work and computing power. That’s where AI for Earth, a project within Microsoft dedicated to solving global environmental challenges with a $50 million war chest to spend, comes in. AI for Earth is providing Microsoft cloud services, funding, and machine learning expertise for this new mega-database.

“If you’re going to scale an application like Wildbook, you need to bring it into one place,” says Lucas Joppa, Microsoft’s chief environmental scientist.

A tool that Wild Me wants to build out is an AI agent that can automatically check YouTube daily for videos titled or tagged with certain words like “Whale shark.” The agent can tell whether the video actually has relevant footage and use Microsoft Translator to read the video description for any information about the sighting. If the poster didn’t share any details, the AI can post a question asking for specifics.

Wild Me is also working on a Twitter bot called Tweet-A-Whale that could let people on whale watching tours send a photo and have the species identified, all while recording information about the sighting. Both these tools automate the task of gathering data, freeing up researchers to do other work.

Microsoft will make these and any other machine learning tools developed accessible to the broader conservation community with the goal of putting AI to work for the natural world. And the need for more tools is becoming urgent. Big data is getting bigger in conservation, with motion-sensor cameras and drones becoming cheaper and easier to deploy. Apps like iNaturalist and eBird let anyone upload images of biodiversity or birds, respectively, that scientists can use in research. It’s more data than scientists have ever had access to before, and they need new ways to parse it all.

Hurdles To Growth

Even with its new backer, Wildbook’s model has its drawbacks. The biggest? It’s reliance on the public for data. People on safari are only going to share some photos and videos, and old or mangy-looking creatures might not make the cut. Photogenic shots, like ones with the sun in the background creating striking shadows, aren’t always the easiest for the system to analyze. Some safaris might return to the same places again and again, increasing the likelihood of an unchanging cast of animals being photographed.

Bringing in other data from sources like drones could help alleviate the problem, but even as that technology is getting cheaper, it still costs money that not all groups can afford. For the time being as more people get access to smartphones with cameras and an internet connection, researchers can count on free photos and video. This situation creates a bias in the data that has to be taken into account by conservationists.

“Individual observations aren’t great but we make up for it with volume,” says Tom Dietterich, a computer science professor at Oregon State University.

There are other obstacles. The software works best on animals with patterned skin and doesn’t work at all for some creatures. Elephants, which could certainly use the help, currently can’t be identified by the algorithm with their uniformly gray skin, although Berger-Wolf says that they are looking into whether ears are unique enough for machines to differentiate. Some data might have to be kept private for animals that are at risk of poaching.

Still, the insights that could be gleaned from big data and artificial intelligence could be put to work in a way that current methods can’t match.

“All of these citizen science projects can operate at a scale beyond what professional ecologists could do,” Dietterich says.