Exercising in front of a television or an iPad should be a perfect experience. There’s an instructor right in front of you, and all you need to do is mirror their motions. But how do you know if you’re doing a squat with the right form? That question has led to new products like Mirror and Tempo, which offer feedback through 3D scanning your body—with the help of huge, dedicated pieces of camera-filled fitness equipment in our living rooms.
New research out of MIT does one better. With just $100 for materials, it has built what it calls the Intelligent Carpet. It’s a 6-foot-by-6-foot square mat that’s loaded with more than 9,000 pressure sensors. As you walk over the carpet—or even lunge or practice sit-ups—the sensors can measure the specific pressure points of your feet, hands, and butt on the mat. And by analyzing that pressure, the mat can reverse engineer your entire body’s movements.
The Intelligent Carpet can distinguish 15 different actions with 97.8% accuracy, including squats, push-ups, bends, and rolls. It can even distinguish these movements across two people who might be exercising at the same time. While it’s only a measurement device for now, it’s easy to imagine how the carpet could fit into your favorite workout software: automatically counting the reps you do as you exercise, and chiming in with tips to fix your form to help get a good workout and avoid injury.
How is it possible that a carpet can see so much? As Yiyue Luo, lead author on the paper explains, all human activity involves contact with the ground (after all, we cannot fly!). Those points of contact hold incredible amounts of information, if you can harness it. So to build the Intelligent Carpet, researchers had 10 people perform all sorts of actions on the carpet, while they were also being filmed with a 3D camera. They amassed nearly 2 million frames of this information, which allowed the researchers to train software to “see” people simply by their pressure points.
The number one use case for the Intelligent Carpet could be in monitoring older adults at home, says Luo. The carpet could blend in perfectly with someone’s living room—in theory, it could span an entire floor plan—while keeping a vigilant watch on their gait. It could see falls (perhaps even predicting them ahead of time, like the Apple Watch) and monitor the effectiveness of rehab.
Then Luo sees a lot of potential in movement-based video games and exercise. While Luo’s team has only trained 15 specific actions to date, she sees no reason that they couldn’t train the Intelligent Carpet to spot yoga poses—though she warns that the software might not be able to perfectly predict the position of someone’s hands or head in those advanced positions.
That’s because even though the Intelligent Carpet is an impressive bit of technology, it still can’t see every type of movement perfectly. The system does worse predicting the position of your upper body than your lower body. Similarly, if you tweak an exercise from the norm—like if you lift your legs during a sit-up, lifting your feet off the mat in the process—the system has less of a frame of reference to see that sit-up. Finally, the difference between bending and twisting—due to the two actions using similar pressure points in your feet—can be difficult for the system to distinguish.
As an inexpensive alternative to cameras—which are an inherent privacy issue—the Intelligent Carpet offers a lot of promise to monitor your health or track a basic workout. In the future, Luo wants to train the carpet to understand interactions between two people, such as dancing or hugging. The result would be a smart home or environment that didn’t just know you were present, but could begin to unpack the dynamics occurring between people.
Of course, the better the carpet gets at decoding human behavior through our feet, the more of a privacy risk that carpet could be, too. But if I had a choice between a hacker hijacking a camera in my home to see my family, or a carpet in my home to see their footfalls, the choice would be pretty darn obvious.