Ever wonder how you can navigate Facebook’s mobile app better today than last year? Or how that once-horrid timeline that didn’t quite flow has now become easily accessible with a single hand? Facebook engineers Ari Grant and Kang Zhang announced it’s because of its new user test they built, and it’s called Airlock.
When Facebook committed to native app development for its mobile products two years ago, it lost the ability to test its performance in an A/B setting–one that’s needed for a scale as big as Facebook’s, and is only viable in web apps. With the new Airlock testing framework, the company will now be able to experiment on 10 to 15 different variations–no easy task on iOS and Android, say the developers. “We had to relearn the rules of not letting one experiment pollute another, keeping some experiments dependent and others exclusive, and how to ensure the logging was correct in the control group,” they explain in the post.
With nearly two-thirds of Facebook users accessing via mobile devices, the company has struggled to develop a scalable testing method. Turns out leaning on the server’s categorization wasn’t cutting it. “A single bug led to a large number of people seeing a different variant than we had anticipated,” the developers wrote. “The server was insisting, ‘I told the device to show the string!’ but somewhere along the way the statement became a little fuzzy.”
The company says Airlock’s A/B test allows its developers to more seamlessly produce results from its servers to your iOS and Android apps, so devs can more easily keep control of the variables in the experiments. “The device requests the data for an experiment and the server logs the response that it sends out. When the client actually uses data, it logs it on the server,” the developers wrote. “Therefore, even if someone does not see what we want, we can still perform the correct analysis.”
The company explained the inspiration behind Airlock, which came from the design team’s attempt to simplify the app’s navigation bar. “Over the course of a few months we tested making the left-hand drawer narrower with only icons, putting a tab-bar at the bottom of the screen with your timeline in it, combining friend requests and notifications into one tab in a tab-bar, and eventually landing on the tab-bar design that is now the user interface for the Facebook for iPhone app.”
Though Airlock is solely used for Facebook developers right now, a spokesperson told Wired that the company is considering its open source value. Last week the company acquired Android app startup Little Eye Labs–another indicator the company is getting serious about its mobile experience.