How The Huffington Post Subtly Crowdsources Headlines

Crowdsourcing usually invokes an image of thousands of people contributing to a vast database--à la Wikipedia. But it needn't be so large scale: The Huffington Post uses a quick'n'dirty crowd system to test the popularity of its headlines.

The Huffington Post

In The Huffington Post's case, as Niemen Journalism Labs has found out, the main difference is that the crowd isn't being used to initially produce the headlines. Instead, it's being used to provide data on how the crowd itself prefers alternative wordings. It's a process called A/B testing, which is often used in the design of public-facing services: Two alternative versions are mocked up and shown to two halves of a focus group. Comments from both halves of the group are collated, and a final design is chosen from the two options, or a hybrid is put together.

Writers know well that headlines for a newspaper or online publication are absolutely key--they can make or break an article, almost despite the quality of its actual contents. Online, where reader clicks are the lifeblood of the business, this sensitivity is even more important: Which is why HuffPo's use of live crowd-sourced A/B testing is clever.

The HuffPo system is automated, and applied to many of the headlines on the site. Writers and editors put together two alternate versions of the headline text, and slot it into the code. For five minutes, visitors to the site get to see either one or the other version of the headline at random, embedded among the others on the front page. When the five minutes is up, the code works out which headline has garnered the most click-throughs to the full article--a perfect measure of the headline's pull, when traffic is high enough--and then the final headline is fixed. It's almost a self-optimizing system, with a successful headline basically selecting itself. It's also a neat way around one of those odd side-effects of writing: Sometimes an article just takes off in popularity unexpectedly, perhaps because its headline taps into a public mojo the writer wasn't expecting.

At this level, it's also still a human-driven experiment. But it really can't be long until someone attaches a fuzzy-logic AI into a HuffPo-like system, with auto-generated headlines being tested on the public before a final version is chosen. It seems like the sort of logical next-step (if somewhat artless) tech that someone at Google is probably already working on.

[Via NiemenLabs]

Add New Comment

5 Comments

  • Jack Butler

    Auto headline writing is just the beginning. Robotics can be deployed for content of the articles. The logical next step then would be auto-reading. It would be of great help to Chauncey Gardner.

  • Zach Seward

    Hey, Kit. Thanks! Really appreciate it, and feel better! And, yeah, I bet they are sitting on a ton of fascinating data. Too bad it's proprietary.

  • Kit Eaton

    @Zach. Hi there. That's what happens when one tries to write up a piece when suffering flu symptoms! How's it for you now?
    Given HuffPo's slightly icky public relations image, It's not surprising Berry was reticent to reveal how many stories get this treatment. Shame though--that fact would be an intriguing one to know.

  • Zach Seward

    I really do appreciate the link and the writeup, but this isn't so much "via" the Nieman Journalism Lab as according to us.