Netflix Study Says No One Knows About Streaming Video. Huh?

A new study comparing streaming video services from Amazon and Netflix found that most people surveyed didn't even know they could stream video in the first place. Wait a minute: Didn't a study we reported on yesterday say that 90% of Netflix users were aware of Watch Instantly?

Netflix

The Catalyst Group, which executed the study, told The New York Times that streaming video wasn't even on the study participants radar: "They were shocked that this is something you can do," said the group's CEO.

But the study only used 11 participants, according to its Methodology section, and only five of whom were pre-existing Netflix subscribers. The study filtered for people who had never used a Roku box—the $99 set-top box that streams both Amazon and Netflix directly to a TV—because the Roku box was the medium experimenters used to compare the services. By filtering out anyone who'd ever used a Roku from the already-tiny sample size, they ended up with a disproportionately clueless group of subjects.

streaming video

Even so, the experiment still produced a useful comparison matrix that shows how the two instant video services stack up. Participants were frustrated at Amazon VOD's poor organization, for example, and couldn't always tell if a movie they liked on Netflix had been added to the correct cue for instant viewing on the Roku box. The Catalyst Group reported—similar to the study we discussed yesterday—that users were deeply frustrated that they needed a $99 box to get streaming movies onto their TV, and that they weren't content with just watching movies on their PC.

Check out the rest of the study's findings below.

Catalyst Roku Report2

Add New Comment

5 Comments

  • gourav jain

    The Roku digital media player is very comfortable to watching movies and listening songs.people can enjoy the video or songs with using this.This appreciation are helping for creating good picture quality on the multimedia phones likenokia 5310

  • Nick Gould

    I'm CEO of Catalyst Group and I wanted to weigh in with some corrections / clarifications.

    1. The purpose of the study was qualitative not quantitative - which is clearly indicated in the report for anyone who would bother to actually read it. Our intent was to gather comparative reactions to the two download services available on the Roku - not to conduct a quantitatively significant survey. Within this limited scope, the sample size is appropriate.

    2. Calling the participants "disproportionately clueless" is absurd when the sample was specifically recruited to be unaware of the Roku experience - and we did not have to look hard to find people who didn't use a Roku. That's the vast majority of the population you're talking about. At this point, I'd say Roku users are disproportionately clued-in. If you consider the intent of the study as I've described it, I think you'll agree that this was a necessary screening qualification.

    3. One of the many observations of the study was that people were surprised that they could stream video directly to their TV without using a PC - pleasantly surprised. Our conclusion from this was not that "no one knows about streaming video" but rather that our sample had a positive reaction to the Roku experience. A result that was, incidentally, very much supported by the comments to the Times piece AND by the prior study that you referenced.

    4. Finally, you mis-characterized one of the primary observations of the study when you said: "users were deeply frustrated that they needed a $99 box to get streaming movies onto their TV, and that they weren't content with just watching movies on their PC." In fact, most people felt that $99 was quite reasonable and less than they would have expected to pay for this function. Their main issue was having to use the computer at all in the activation or movie selection process. In other words, they like the Roku concept, saw value at the price-point, and would prefer to dispense with the PC altogether as part of the experience. A finding that we thought was interesting and potentially significant in terms of the evolution of Internet-delivered video on demand services. And I think that's what Brad Stone at the Times found worth writing about as well.

    While we appreciate the coverage of the study - and some of the positive comments towards the end of your post, a simple phone call to us (or, frankly, a closer read of the study) could have cleared up some of the misconceptions in your piece. You can find us at www.catalystnyc.com

    Nick Gould
    CEO
    Catalyst Group

  • Jensen Gelfond

    an 11 person sample size? Are you kidding? You could show whatever you wanted to with a sample size that small, or even a sample size of 100. That only stats I took was intro to stats in college, but even I know that this study was not executed properly. Shame on The Catalyst Group.

  • Shevonne

    This is one of the issues I have with study. They skew the study to whatever they want.