Concept Testing vs. Concept Development

A lot of my focus groups test concepts, either as products, services or policies. All too often clients want to use the results to "pass or fail" a concept. In fact, the Federal Government commonly uses the term "Focus Test" to describe qualitative research that involves focus groups. I have always objected to research "passing or failing" a concept. There are a few reasons. First, people tend to have negative reactions to new ideas. The actual psychological reasons are irrelevant, but people tend to feel threatened by new ideas, are concerned about change or are jealous of not having thought of the idea themselves. Second, research can never duplicate the actual product or service being tested. What we are actually testing is concept statements, or prototypes - not the actual deployment of the product, service or policy - so research is only a proxy test at best.

In my opinion, the better approach is concept development, as opposed to concept testing. That is, we should go in to the research thinking of ways to tailor the product to the hopes, dreams , opinions and practical realities of our participants. When this is done we will provide more insight and interpretive value to our clients.

Let's start by looking at the topic in a way that interests everyone - money - or value for research dollars. I don't know how many times I've been involved in research assigments that involve more than 20 focus groups, only to hear the same responses said over-and-over again. Sometimes clients have reasons for conducting this many groups, but how much value can be achieved hearing the same responses about a concept that does not change? A better approach is to take the learnings from each successive group and change the concept being tested based on the results from each of the previous groups. The value for money then becomes apparent - we begin to develop the concept based on consumer opinion so that each new session tests and refines the new idea. At the end of the assignment the client receives results that tell them what is needed to make the concept a practical reality, as opposed to passing or failing a concept. Our clients are then in a much better position to decide if they have the resources available to implement the concept based on consumer's wishes and desires. There is just not much interpretive or monetary value in hearing the same feedback again and again. If the groups are in place, it makes sense to get as much value from them as possible - and that means changing the concept with each successive group.

Based on the above, the methodology of concept development becomes quite clear - a concept becomes a variable entity that is changed and refined for each successive research session. It is no longer a static idea that gets trodded-about like some trunk-show from group to group. This form of concept development, however, requires an experienced moderator and client. The moderator, as an expert in interpreting qualitative opinion, needs to sort through what is a real opinion and what is a postured response (or defence) to the new idea. The client, on the other hand, needs to be in a position to decide whether what is heard in the groups is realistic or not. That is, it does not pay to refine a concept in such a way only to be told by the client that the refinements simply cannot be implemented due to budgets or resource or strategic constraints.

While the client's role is to ensure that successive developments are realistic, the moderator has a more difficult challenge - he must be able to interpret opinion based on one group, and this requires the ability to sort through respondent BS and real pshcyological opinion. In the groups, it requires a more aggressive pursuit of deeper opinion and reasons for it. For example, price and requests for more informtaion are ususlly NOT the main reasons for rejecting a concept. They are simply the easiest reasons participants can give to avoid sharing their true opinions.

Similarly, hearing third person comments like "society is not moving in that direction" are not useful comments, nor are comments like "other people think this" or "the target market won't go for it." Psychotherapy talks about "The Language Of Responsibility", which has as one of its tenents a first-person response. In being more aggressive about seeking individual opinions in a concept development group, I will often say in the introduction that it is important for people to talk only about themselves, and not about anything or anyone else. If someone uses a statement like this I will usually ask "What you're saying makes a lot of sense, but what does it mean for you - will you accept or reject the concept?"

Shifting the focus to "I" responses needs to be done with care and with a specific purpose. Believe it or not, the purpose of the question is not to get someone to say whether or not they will actually accept or reject the concept - the actual purpose is simply to get the participant to start expressing their own inner thoughts and feelings. If a participant says "The market is not going in that direction so the concept will not work" they are rejecting the concept - and they are doing so in a manner that does not put responsibility on them, or in other words, they are taking the easy way out. If they are asked to phrase the statement in terms of "I" language, they may realize that they, themselves, are rejecting the concept - and they may not want to take responsibility for it for fear of offending the moderator, or people behind the mirror. More likely, they predisposed to being perceived as agreeable. As such, they may say something like "Oh, if it were me, sure I'd try it - I was just commenting on what I think the market would do."

What happens next is critical - the moderator cannot accept that response on face value or ask the participant why they are now accept the concept. To ask why a respondent accepts a concept at this stage will likely make them dig deeper into more inaccurate justifications (and in analysis, these responses should not be used as key findings, also the goal is concept development and not testing). Rather, the moderator should just say "OK, let's keep going - tell me what you think about..." and then continue to ask questions that elicit a first-person response. The goal of the intervention is to get the respondent to simply talk in terms of themself and not in the third person. Also, by not acknowledging the response, we will gain credibility in the eyes of the participant - if they are lying, and we did not acknowledge or fawn over the response, they will likely understand that we can detect their BS.

When a moderator gets the participant to internalize the concept, the more accurate the responses will be. At the end of the discussion, if the focus has been on "I" responses, the modeartor will have a very good set of qualitative data based on the participant's true thoughts and feelings as opposed to an assemblage of views that are really stated simply to avoid any real judgement of the concept to begin with.

There are other important tricks up my sleeve that I like to use when concept testing, and that is when I show a concept statement, I will often start with asking people what they dislike about the concept first. This gets the negative stuff out of the way. People tend to want to criticize a concept more than they want to praise it, so it is a good idea to let these thoughts out into the open, so that participants are not hanging on to them when it comes time for positive comments. Also, since there will likely be more negative comments, this tends to get the creative juices flowing in the group and encourages participants to open-up more. It is important to realize that if the goal is concept development, and not concept testing, clients worried about too much emphasis on the negative turing people off the product, should relalize that the goal of the resaerch is not to determine whether people will actually use the product or not.

Another method I often use is projective techniques at the beginning of the group. I will spend maybe 5-10 minutes lip-servicing the general likes and dislikes of the product category so that the client gets a sense of the "head-responses" people have towards the category. To me, the more valuable exercise is a projective technique where participants are split into two groups. One group imagines the most positive experience they can with the product category, and the other group immages the worst possible debacle. This kind of exercise will produce more relevant information about the product category than a simple "what do you like or dislike" about a product. These techniques encourage clients to tell a story - we are more likely to understand their entire situation and gain a more holistic view of the overall data.

While group process can play an important part in concept development (i.e. as new developments are described, the group can react), I believe that there is also a strong case to be made for one-on-one interviews for concept development. The main reason is that one-on-ones tend to avoid posturing by participants. In the case of concept development, participants are more likely to give more direct feedback on a concept without being intimidated by anyone in the group. They are not so concerned, for example, as coming off as someone who wants to please others, and therefore never criticizes in public. While a good moderator can find ways around posturing, I believe that one-on-ones are certainly a more efficient method.

Finally, in terms of concept statements, it is important to work with clients to emphasize benefits as well as features. In terms of a concept statement, benefits are what will get people interested in the product or service, not a listing of feautres. The features may be important, but it it the benefits that sell the product. I always encourage a client that for every feature they list, they must list either a direct corresponding benefit or have the same number of features and benefits in the concept statement. I also avoid statements that contain jargon like "class-leading horsepower among mid-sized sedans" or "high-end apparel and accessories for the fashion-forward and trendy career woman." Bottom line - that is not the way consumers talk.

All of these techniques - projectives, focusing on benefits, changing the concept statement and ensuring that participants evaluate the product from their point of view are all designed to lead to a single two-second moment in the qualitative research. It is when participants go "A-Ha", or "That's what I mean" and a light goes off in their head. This signifies that the product actually provides a solution to them. The "A-Ha" is truly the way of "testing" whether a product will work or not in the marketplace. In my opinion, the only way to reach that moment is to "develop the concept" within the group. The chances of any one concept statement eliciting that response, without any creative intervention or continual development is slim to none. And in fact, I believe, that without taking the steps above to properly research a concept, those are the exact chances that a product will have in the marketplace.

Ultimately, good marketers are looking to give their concepts the best chance of success possible in the marketplace. They are not looking to see simply whether a concept will fly or not - and that is why I practice concept development, which by its very definition, gives marketers exactly what they are looking for.