Luis von Ahn is a busy guy. The founder of the highly successful CAPTCHA and reCAPTCHA technologies, von Ahn now runs language-learning startup Duolingo. His company recently received $20 million in funding for their gamified website and app, which let users learn Spanish, German, English, and other languages for free. Part of Duolingo’s success is their unorthodox approach to online learning: Much like online games and e-commerce sites, Duolingo uses A/B testing to hone their curriculum.
The small but crucial difference is that Duolingo doesn’t use A/B testing–giving different users slightly different online experiences for the purpose of testing–to sell more consumer products or in-game currency. Alongside standard web design tweaking, von Ahn and his team use A/B testing to find different ways people learn languages–and tweak Duolingo’s experiences based on that.
When I spoke with him shortly before the company’s release of an updated iPhone app this week that includes Chinese- and Japanese-language support along with multiplayer capabilities, von Ahn was less interested in discussing how English speakers learn Spanish or French than how non-English speakers learn English. There’s a reason for that: The company sees the international market as their primary moneymaker, thanks to translation deals with BuzzFeed and others. Due to English being the world’s de facto second language, Duolingo sees huge growth opportunities in offering free comprehensive English courses online. And they’re relying on A/B testing to help users, both English speaking and non-English speaking, learn.
One of Duolingo’s most important discoveries from A/B testing users, von Ahn told me, was to completely switch they way they introduced the pronoun “it” for Spanish speakers learning English. Compared to German speakers, Spanish speakers had a much worse performance when “it” was introduced early on. Although his team expected that–Spanish has no gender-neutral pronouns–they didn’t expect to see users become discouraged the way they did on Duolingo. Since “it” was a Spanish stumbling block, they decided to move lessons for the word to a much later point in their course.
Another discovery Duolingo’s engineers, linguistics, and coders found had to do with plurals. In a recent Reddit AMA, von Ahn wrote that:
We’re basically doing very large-scale data-driven education, and improving it every day. For example, if we want to know how early we should teach plurals, we simply test it. We give a group of 50,000 people plurals a little earlier in the course than everybody else, and we measure: Do these people learn faster, do they learn better, do they stick around for longer? And if it’s better, we start teaching plurals earlier to everybody. This approach improves the experience on Duolingo and also is giving us answers to those language questions we really couldn’t find answers to when we were just starting out.
Now Duolingo is trying something different: Letting their user community build their own language courses, and giving them their own A/B testing capabilities. Late in 2013, Duolingo launched a language incubator which lets active users crowdsource language courses–in effect, having the Duolingo user community build courses on the company’s behalf. Within the next few months, von Ahn says, users who are building new courses will have access to A/B tools just like the company’s own developers have. “We want to give them power to test their courses,” he added.
In the end, von Ahn told Co.Labs, Duolingo uses A/B testing for two goals: Maximizing the time users spend with the service, and maximizing what he calls their “learning outcome.” For Duolingo, they’ve found that building data-driven learning plans has been the best decision so far.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.
