Hallelujah and amen, Laura! Focus groups have notoriously rejected minivans, Seinfeld, and telephone answering machines. ("What do I need a machine to answer my phone for? I'm perfectly happy doing it myself...") They're statistically irrelevant -- indeed, statistically worthless -- and the participants are under huge "group think" pressure to conform. In fear of making a mistake, participants tend to favor the familiar and reject the unusual. That spells doom for innovation.

Yet, because focus groups are "scientifically" conducted by overpaid consultants in controlled environments, people take their feedback seriously. I'd rather conduct one-on-one interviews with each person, with the goal to determine what their needs and issues are.

BTW, here's the best video ever about focus groups:
http://www.youtube.com/watch?v=ku4Ugw0lQ4Q

I once participated in a focus group for a woman's clothing chain that was thinking about launching a men's division. The garments they showed us as examples of the new line were dreadful, but many of my fellow participants were gracious and tried to be flattering, out of some misguided sense of courtesy. I was appalled, and decided not only to try and help these folks avert disaster, but to try an experiment in focus group dynamics.

Instead of muttering vague reservations, I voiced my opinions in the most strident, take-no-prisoners style I could muster. It was the kind of obnoxious, domineering behavior I normally despise, but the results were eye opening. Soon the entire group was chiming in about how nasty the clothes were, what a bad idea the whole thing was, etc. Nobody wanted to be the odd person out, saying they thought the whole line was fine -- even if they authentically liked it.

Were those results then used to inform the client, or discarded as invalid because some boob (me) had swayed the jury? Who knows? All I know is that it was remarkably easy to contaminate this supposedly "objective" pool of people. I shudder to think how many costly decisions are made based on similarly suspect findings.

Focus groups can serve a purpose, but their results should be only one data point in a larger research effort. To whit: focus groups hated the Aeron Chair.

Thanks - now if you could only convince all the people who hide behind these things! The only real use I've ever found is to avert a disaster that you might otherwise have missed (Wrigley's chewing *gum*, when introduced in UK - people thought of gum as a material for the bottom of their shoes).

Focus groups are at best qualitative research, but are too often used in place of real quantitative research. The sample size is too small to make any sense, and the attendees aren't randomly picked from the population. The result? What you hear/see is just as likely to be wrong as it is to be right.

If you want to do this kind of qualitative research, why not just visit some of your customers or prospects? Oh, that would mean that you'd have to know what you're doing...

I gotta be honest - i cringed a bit when i read your article title and saw all the people giving the "finger." My time in the tech industry has led me to the conclusion that some designers may feel threatened by research at times. There are some designers who view research and customer data as rain on their parade or something. Usually these designers view reality as impinging on their fantasies and they don't like it. They would rather create something they think is cool and visually compelling (using their own internal standards) and not understand their customers' mindset and perceptions. Sure, there is bad research and if that's part of the issue, then that needs to be fixed separately. there are many problems with traditional (in person) focus groups and improper research leads to imporper conclusions. But well done research can and should lead to better design insights, happier customers and greater success in hitting the intended business goals.

Holy crap! You actively dismiss focus groups, but then claim that oh, they ARE good for understanding what people are really like, something you obviously have no experience in.

Focus groups are TERRIBLE for uncovering any needs beyond which you already know about. For all the reasons (and more) you describe, focus groups are also terrible for getting people to share deeply. They are in a group, with new and complex dynamics. They are in a group, so you can't have a followup-rich conversation with them. They are in YOUR territory, so their presentation-of-self is framed that way, and there are no cues for you to observe in order to get deeper. You want to understand a demographic (or other -graphic) group? Go to their environment and see what they have, what they do, and what they say. Don't bring them into a focus group facility with a bunch of strangers.
--
Portigal Consulting - http://www.portigal.com/
All This ChittahChattah - http://www.portigal.com/blog/

I suppose it is about the motivation.

Want to avoid failing? A focus group will make you look safe then.

Want to win? A focus group could never get you there...

On second thought... I suppose it is about using the right tool for the job - then that is half the job done for you.

Getting creativity from a focus group? Never.

But some games development studios have used very focused testing with focus groups with great success. The idea is to take what is already there, and through testing it over and over and over again with fresh eyes, you can hone it to the level of perfection, where story, gameplay elements and messaging just frictionlessly and invisibly enter the player without breaking that fragile suspense of disbelief. A quality much admired among fellow games designers. And one which apparently only can be attained through countless iterations and polishing with the help of focus groups. Because no matter how good a designer you are, you are never THAT good.

So the focus group is not relied on for creativity and designing the elements, but rather just fine-tuning what the designer has brought to the table.

I'm spoke with a group of marketing people and we are all insulted... no not really. Even as a marketer I agree completely. Leave the focus groups to find some opportunities to design for... not to evaluate the design of!

Thumbs down. Why?

I agree with the concept. Good insight.

However I had to laugh (negatively, cynically, sorry, I'm only human) that Laura's credentials are about half as long as the article itself. She would've lost me after the first sentence, but cynically I wanted to see how she waxed lyrically about all her achievements, turning her insight piece in to a sales piece. So, thumbs down... after what started as a thumbs up.

Laura, you are using your own definition to prove your own assertion and that is a weak argument (actually invalid). My own experience is that focus groups can be very useful (if... etc.), but more importantly I believe the broader underlying issue in your discussion is Design Leadership as opposed to Design Democracy.

Love him or hate him, Philippe Starck (I happen to love him, but then I am the only one in the US who admits to eating Spam occasionally) is a design leader. The Aztec, as you indicate, may be a result of design democracy. The US constitution is a result of leadership and your local congressman is a result of democracy. what is the right design approach? It depends. I would not want to pay an extra dime for a phone designed by Starck (for one, I would be concerned about usability) and I will never color my rooms based on a color mix resulted from a focus group (gray) and I much prefer my wife's leadership to get bold stimulating colors. I would probably prefer yours too.

--
Shimon Shmueli
www.touch360.com

An alternative to Focus groups is to play Innovation Games with customers of a product. These methods get customers to cooperate with each other around issues, ideas for next products etc and give the designers/product people a really different kind of input then people paid to talk over lunch do. http://www.innovationgames.com

As a marketing person I want as much data as possible. Focus groups can be useful as long as they are, well, focused, and on the right things. I've found focus groups useful for reacting to beta products and less useful for things that require imagination.

No matter what, a good marketing person has to be wary of group think, and a good moderator will work to cut off group thinking before it can take over a group. This can be a hard task, since groups are so easily dominated by strong personalities. Many people don't want to stand out or be in conflict with the rest of the group so they subordinate their true feelings to fit in.

I've found group think to be especially deadly to the creative process and it goes far beyond the focus group. When looking at creative, people naturally revert to their personal tastes and favorite colors, instead of thinking like their target audience, and when the boss asks "wouldn't it be better in blue?" suddenly everyone else thinks so too.

My best testing experience for creative comes from running small tests online. People either react to it or don't. But when you need more information than a simple "will it make them act," or your creative is tactile, my preference is for the one-on-one. It takes longer overall, but you are more likely to get a person's true reactions when he or she isn't worried about pleasing the group.

People in focus groups are paid to criticize. That's a huge problem.
You have all got to watch this brilliant video http://tinyurl.com/ltaf5d

Interesting definitions of focus groups and where to draw the line with them in this issue of Fast Co.

Basically, whole discussion I followed up till now, is not weather focus group is a valid methodology or not, but rather the discussion is about do people tell the truth in market research and especially in focus groups. If I barrow from the popular TV show, House MD, the answer is easy; “everybody lies, the only variable is about what”. Indeed, I definitely agree, it is correct that human beings are not telling the sole truth, only truth and nothing but the truth while on court, and definitely not in qualitative interviews. Actually, it would be naive or at best credulous to accept them to tell everything openly and easily. I guess there is three main reasons people are not telling the truth. These are; first, they think it is very private to share with pubic. Second, they prefer to say the cultural ideals, the desired social behavior instead of the truth. Thirdly, they do not know the real answer, and in order to not to be classified as dumb they made up rational reasons for themselves. All of these are valid for focus group, basically focus group provides an environment that mimicries the real world, people are introduced to each other with their real names and occupations, i.e. with their social status. Therefore they are unwilling to share their private life to unknown people, and they want to be liked, respected by these foreigners by playing the socially righteous, virtuous human. Therefore, focus groups are more conservative than actual audience and consumer. However, on my opinion the most important one is the last one, people tend to made up rationally looking arguments whenever they don’t know why they are doing something, that is why focus groups are not good for creative things, they would be sticking to the most boring but safe, old school argument.

Thanks Laura. Yes, not only do most of us have bad feelings and funny anecdotes about focus groups, but there is plenty of research to point out their deficiencies. Here's a good academic summary of pros and cons of both focus groups and depth interviews, courtesy of BrandTrust:

http://www.brandtrust.com/downloads/Methodolatry.PDF

Also, see the book "Truth, Lies, and Advertising" by J. Steel for more opinions and humor on the subject.

But let's not get carried away with in-home interviews, depth, one-on-one, or pseudo ethnography research the same way our "ancestors" became so enamored with quant. surveys or the focus group. For example, one could argue that in collectivist cultures - such as a few billion consumers in Asia - the group dynamics in a focus group are a better simulation of real consumer considerations/activity/environment than one-on-one interviews. In this case, Laura's comment that focus groups "can provide insight into who the participants are as a demographic GROUP" are right on.

And the in-home interview - useful in a place like the States where people are comfortable inviting guests over - is just as artificial and uncomfortable in other parts of the world as the florescent focus group facility is in the U.S.! Even a one-on-one conversation where an interviewer is asking a lone respondent for an opinion is arguably only suited for individualist cultures.

Finally to Ravi Sawhney's related focus group story, I was surprised by his practice of placing a cooking range design model next to competitive products for focus group feedback. This should only be attempted if:
--Your "design model" is only a minor styling/finish tweek
--You have magically recruited for only extreme outliers/early adopters of cooking ranges
Otherwise, respondents will choose the familiar competitive products over the design model every time. = Innovation interrupted.

--

Jeff McFarland
www.spaNconsulting.com
Singapore | Los Angeles