How Apple Made The Watch Work For Wheelchair Users

Apple engineer Ron Huang describes the year-long process of overhauling watchOS for people in wheelchairs.


In America alone, there are more than 2.2 million people who depend upon wheelchairs to get around every day. But most wheelchair users aren’t active. They’re more sedentary, on average, than those who have full use of their legs–and are consequently at much greater risk of heart attack, stroke, and diabetes.


Amazon is currently selling more than 1,000 different models of fitness trackers online. But how many support wheelchair users, an enormous demographic in desperate need of sophisticated tools to manage and encourage fitness?

None. Zero. Zilch. Up until now.

Two weeks ago, Apple made a seemingly small announcement at its annual Worldwide Developer’s Conference. Starting in September, the Apple Watch will support wheelchair users, allowing them to track their fitness goals the same as anyone else. But this feature is a big deal to the millions of people around the world who live their lives in wheelchairs. It was also an incredible technical challenge to pull off, requiring Apple to mount the most comprehensive study ever on fitness among wheelchair users, as well as a complete overhaul to the design of its fitness tracking algorithms.

We spoke to Apple’s Ron Huang, director of software engineering for location and motion technologies, to get an inside look at how Cupertino made the Apple Watch’s wheelchair tracking features possible.

How Fitness Trackers Work

Whether you’re talking about a Fitbit or something more sophisticated like an Apple Watch, all activity trackers work pretty much the same way. Inside is an accelerometer, a little chip that can detect and record acceleration data. In graphic form, that data looks something like a Joy Division album cover, but in the spikes and troughs of that data stream is a record of every gesture, step, jumping jack, or swimming pool cannonball you’ve made while wearing your tracker. All the device needs is the proper algorithms to decode it.


In the case of the Apple Watch, its algorithms look for two things to measure a step. Since most people who aren’t Molly Shannon on Seinfeld swing their arms when they walk, the Apple Watch tracks arm movement, with the length of the arm swing roughly corresponding to the distance of the stride. But people swing their arms a lot, even when they’re not walking, so the Apple Watch also looks for a telltale data spike of a heel striking the ground, punctuating each step.

In this way, it can tell the difference between someone jogging and someone doing a Cabbage Patch. Multiply the number of steps versus the average number of calories burned per step according to medical consensus adjusted for an individual’s height or weight, and voila! You’ve got a fitness tracker that can convert steps into calories burned.

The Science Of Wheelchair Fitness

But this algorithm breaks down for wheelchair users. Most obviously, those who get around on wheels don’t strike their heels against the ground. Even the way wheelchair users move their arms when pushing themselves is different than the way people swing their arms when they walk. Walking is a regular motion; pushing, comparatively, is irregular. Wheelchair users need to start, stop, and adjust their pushes more than walkers do. To make the Apple Watch’s fitness tracking functionality useful to wheelchair users, then, Apple needed to totally reexamine its algorithms.

First, Apple’s software engineers examined the available scientific literature on how wheelchair users burn calories. But this literature was lacking. The existing studies tended to only involve a small number of subjects, and their methodology in translating pushes to calories wasn’t applicable to the real world. For example, the studies might prevent their subjects from using their own wheelchairs, or only track how many calories a wheelchair user was burning on a treadmill, not on their home turf.

None of this was useful data for a general-audience device meant to track wheelchair users outside of a lab setting. Apple found the existing studies so lacking that it ended up conducting the most comprehensive survey of wheelchair fitness to date. They teamed up with the Lakeshore Foundation and the Challenged Athletes Foundation, two organizations dedicated to promoting fitness among people with disabilities.


Each test subject was allowed to use their own wheelchair, which they fitted with special wheel sensors. In addition, many were outfitted with server-grade geographical information systems, which collected extremely precise data on their movements through the world. The number of calories burned, meanwhile, were determined by fitting test subjects with oxygen masks, and precisely measuring their caloric expenditure as they pushed.

In the end, Apple collected more than 3,500 hours of data from more than 700 wheelchair users across all walks of life, from regular athletes to the chronically sedentary, in their natural environments: whether track or trail, carpet or asphalt. From this data, they learned how to adjust watchOS 3’s algorithms to track wheelchair users.

Tracking Pushes, Not Steps

It turns out that wheelchair users tend to push themselves in three different ways, each with its own corresponding accelerometer patterns and calorie expenditures. The first is in a semicircle, pushing from 10 o’clock to 3 o’clock. If you’ve ever pushed yourself around with a wheelchair in a hospital, this is probably the pattern your arms made. The second is called an arc push, and it’s what you do when you have to push yourself up an incline: shorter, more powerful pushes with a quick jerk to the return position to prevent yourself from rolling back. Finally, there’s the semi-loop-over: a pushing style that tends only to be done in competitive situations, like wheelchair racing, where you’re really leaning into the push.

These patterns were all identifiable in the accelerometer data, but how to reduce false positives? After all, from an accelerometer’s point of view, pushing yourself in a wheelchair and, say, turning a crank or rowing a boat can look similar, but they don’t burn the same amount of calories. That’s true for walkers swinging their arms, too, but in that case, each step is punctuated by a heel strike: If that doesn’t exist, the Apple Watch knows not to count it as a step.

Apple had to find another way. Even without a telltale heel strike, the team found it could still tell the difference between a wheelchair push and a Cabbage Patch by looking at which direction their hands are traveling: A wheelchair user pushes down, so if the wrist angle is above the horizon, Apple knows it isn’t a push.


To accommodate wheelchair users in watchOS 3, Apple also had to make some UI tweaks. For one, the Apple Watch tracks three fitness metrics through the wearable’s Activity Tracking rings: Move, Exercise, and Stand. For wheelchair users, watchOS 3 replaces the Stand ring with a Roll ring.

And since being told to stand every 60 minutes by their Apple Watch–as it does for users in watchOS 2–might be considered a little insensitive, watchOS 3 reminds wheelchair users to roll in place for a minute every hour instead. The Apple Watch Workout app has also been tweaked to include exercises appropriate for wheelchair users.

Why It Matters

Those not in a wheelchair might find this intellectually fascinating, but a little abstract. It’s incredibly important, though, to the 2-plus million people in the United States alone who depend upon wheelchairs for their day-to-day mobility, and for whom existing fitness trackers simply don’t work.

“There’s a huge disparity in the amount of exercise people with disability get, compared to those who don’t have disabilities,” says Jeff Underwood, president of the Lakewood Foundation. Underwood says a company like Apple taking an interest in fitness among wheelchair users sends a strong message to the community: “You should be exercising. Your health is important. Here’s an extra tool to motivate you and sustain a healthy lifestyle. Apple’s raising the expectations.”

Time and time again, seemingly niche accessibility features have laid the groundwork for mainstream technology improvements that benefit everyone. For instance, Siri owes a debt to a lot of earlier work done in natural speech recognition to help those without full use of their hands access computers.


So even if you don’t care about building a more inclusive world of technology–which you should, because there’s roughly a one-in-five chance you’ll have a disability in your lifetime–the work Apple is doing in making the Apple Watch accessible to wheelchair users has a good chance of benefiting you somewhere down the road.

Today, the Apple Watch can accurately count your steps, but who knows? Thanks to work done by Cupertino today to study the way wheelchair users push themselves, the Apple Watch 2 or Apple Watch 3 could get better at tracking everything else you do besides walking, from pull-ups to breast strokes to the number of times you flip people off in traffic. Opening up opportunities for millions of people with disabilities to live healthier lifestyles is a big deal no matter how you look at it. But it’s an even bigger deal when you remember that all of technology is accumulative. Even if you never know it, someday, the work Apple did making fitness trackers work for wheelchair users is likely to touch you, too.

[All Photos: courtesy Apple]