This is a quote from Malcolm Gladwell's book "Blink". The central premise of the book is that humans make snap judgments within a blink of an eye - BUT that this mechanism works at an unconscious or "unaware" level, and attempts to have people explain WHY or HOW they reach these quick decisions often ends in failure. In fact, asking people to explain what the actual snap judgment was is often an exercise in futility.
Gladwell makes the quote because the exploration of the why and how we make decisions is the purview of psychoanalysts and psychotherapists. In my own psychotherapy training, every time I asked "why" to a mock client - my instructors (and the entire class, if they were watching, would cringe at the question). When I was reviewing other students who asked questions like "HOW does that make you feel", I'd call "Dr. Phil" on them.
So how do psychotherapists get at hearing these snap decisions, and at asking the hows and whys? Typically, we look at two things - inconsistencies and explanations. Inconsistencies are often what academic psychologists spend a lot of time studying. They study physiological responses when people lie, they map the human face or they grade how people interact with one another in order to
get deeper understandings of the relationships between individuals. Psychotherapists, however, do not have access to the monitors or methods these academics use - so we end up using what happens in the "here and now" as a way of digging deeper.
We are trained to look for inconsistencies and "explanations". There's a saying that goes something like "I can't hear what you're saying, because what you're not saying is deafening." It's what's not being said that is the useful stuff. For example, if we ask someone to comment on whether they like the look of a new product and they say "Ummmm... well... it's kind of ugly." What is of interest to us is the hesitation the participant is presenting us with, and not what they are saying. There is a reason for the hesitation. What the moderator needs to do is get to the bottom of it as it relates to the client's product (as opposed to relating to their psychological issues).
What I would probably do is just repeat the words "kind of..." back to the participant and see where it goes from there. They may say "Well, I don't know... the colour is a bit off and the sides are too angular." At that point I have all I need - the participant has said "Well, I don't know..." At that point, I know that the transaction is less about the ugliness of the product, and more about the participant trying to come up with words to describe something that is new to them. The door is wide open for me to get deeper meaning words than "ugly" to describe the product. I may say "I see - the colour and the sides. Let's not focus on those. You said it was ugly, yet I get a sense that that word was just your initial reaction. What else is going on?"
Besides having a hesitant participant, an exact opposite situation may occur. A person may say "Oh my God!! Look at the thing. It's a design disaster. My two year old could take Lego and do a better version of it than that!!" From there I may say, "Well, tell me all about it... let it rip." What I will do is let the person explain away, and exhaust themselves, their need to be superior and their need to explain - what this person wants is an audience, and a chance to be heard - whether or not what they are saying is useful or true. I will generally not be roped-in by any of the explanation given (unless something is really striking). Once they are done, I will say something like "Now that you're done... look at it again - and this time, tell me what you really think." In this situation, it is critical to ensure that the participant does not think that that I am trying to bias or change his answer in any way. What I am trying to do is get past the initial BS, and get to opinions that are not based on a person's neurotic desire to criticize.
In closing - Malcolm Gladwell states that when people evaluate new products, they often do not have the language to describe them. This is true - and another axiom of new product development is that people tend to criticize more than they compliment - people don't like change. There is no doubt that initial reactions play a very strong role in marketing - competition is fierce, and many companies can't take the "time" to see if something will succeed or not (which to me is a very sad state of affairs). However, what is initially "figural" for someone about a new product WILL eventually change or shift. It is one of the central beliefs of psychotherapy - people do have the ability to change. The examples above attempt to show how someone can go from an initial impression of a product to the next stage. A good marketing campaign will use this information to get people there faster, and with a higher rate of success.
Tuesday, July 31, 2007
Anyone wonder why we are called moderators
When I started moderating about 10 years ago I wondered why that was the title given to us. Why aren't we called "interviewers", "facilitators" or "leaders"?
The answer suddenly hit me as I completed my training in psychotherapy. In Gestalt, one of the models of a healthy person is one who can deal with circumstances in an "even-keeled way" and take responsibility for his or her actions. In other words, they can moderate their behaviour.
Looking further at the definition of the verb "to moderate", we see it means "to reduce the excessiveness of; make less violent, severe, intense, or rigorous: to moderate the sharpness of one's words."
To me this gives the perfect definition of a focus group moderator - they are there to make sure that everyone's opinions, thoughts and emotions get equal time - no one person or thought should dominate. What this means to me, however, is that I am moderating the whole person within the qualitative research. My goal is not to just let a person's head or rational thoughts dominate the discussion. My goal is to view participants as a whole and moderate between their head, heart, emotions, dreams and fears. I know I have done a good job as a moderator when I have moderated the "sharpness of one's words" and brought out "the richness of one's emotions".
The answer suddenly hit me as I completed my training in psychotherapy. In Gestalt, one of the models of a healthy person is one who can deal with circumstances in an "even-keeled way" and take responsibility for his or her actions. In other words, they can moderate their behaviour.
Looking further at the definition of the verb "to moderate", we see it means "to reduce the excessiveness of; make less violent, severe, intense, or rigorous: to moderate the sharpness of one's words."
To me this gives the perfect definition of a focus group moderator - they are there to make sure that everyone's opinions, thoughts and emotions get equal time - no one person or thought should dominate. What this means to me, however, is that I am moderating the whole person within the qualitative research. My goal is not to just let a person's head or rational thoughts dominate the discussion. My goal is to view participants as a whole and moderate between their head, heart, emotions, dreams and fears. I know I have done a good job as a moderator when I have moderated the "sharpness of one's words" and brought out "the richness of one's emotions".
Communicating Research To Dumb-Dumbs
Bet the title caught your eye. For any clients or potential clients, the title is not what you think at all!! I actually had a client that I just finished an assignment for a few weeks ago. The main objective was to determine how they could build better relationships with one of the constituencies they serve. A specific focus was to determine how my client could present scientific research to their constituents in a way that would convince them of its validity.
In my focus groups, when I asked about their use of scientific research, my BS detector went off when I heard their responses. Most of the groups were comprised of individuals who did not have a research or scientific background - so to them research was "complicated" and "biased". Moreover, participants doubted the methods and conclusions drawn, saying they just did not know how the research could be so precise as to isolate specific effects of certain substances. I had no reason to doubt these stated responses, but the BS detector told me that these participants were actually scared of research and embarrassed about not knowing how to use or interpret it. This was the unsaid truth in the groups.
Knowing this, my recommendation to the client was not modify how research was presented - rather I simply told them to present the research as is, but to LISTEN and actually AGREE with most everything their constituents had to say about the research, even though, in my clients eyes what was being said was not true. I said in my presentation "What you need to realize is that these people don't know how to use research and are scared of it. If you actually agree with them in general, and then softly point-out point out how the research you are discussing is different, they will be much more likely to listen to you. You can teach them without triggering their concerns over not knowing too much about scientific research."
The recommendation is based on a number of theories in Gestalt Psychotherapy:
- In every transaction, the other person has "the power", not you. If you can yield to this, your mind begins to open-up and listen to the other person. In this example, my client's constituents are actually obliterating the research with a simple thought - how's that for power. They don't have to take any action - all the constituents have to do is say "I don't believe what you're putting in front of me." When this happens, you're dead in the water, and there is no way that you will be able to out maneuver this state of mind. There's a wonderful saying I use - "Never argue with an idiot - they just drag you down to their level and beat you with experience." Now I'm not calling anyone involved in this research an idiot, but the saying holds. If someone is not going to see your point of view, no matter how "right" you are, you're the one that walks away frustrated and angry. The other person walks away "victorious", because they could care less about increasing their knowledge - they care more about winning and have in fact done so.
- There is a wonderful model called the "Co-dependency triangle". In it, people involved in any transaction can be described as "Persecutors", "Rescuers" and "Victims." In my client's case, the constituents were the Persecutors, and my clients were the Victims (i.e. "How could you not accept this research... it's completely valid"). The way to deal with a Persecutor is not to react to what they are saying, but to simply listen to it, keep listening and then show that you have listened. The persecutor simply wants to be heard. When that happens, the transaction opens to more possibilities.
- The Gestalt Cycle is a model that explains how a person makes contact with his or her environment. The very first part of the cycle assumes that a person is able to recognize that his environment is saying something to him (ever day-dream at the office and have someone call your name for 5 seconds - you're missing the environment), and that what the environment says is valid (ever feel slightly warm, but wait too long before you blast your A/C - you're dismissing yourself and your hot environment). My client was not recognizing any of their constituent's objections as valid. When this happens, my client is in a better position to respond.
In closing, I have taken on as my position to bring "humanity" into market research. While I have used three very "highfalutin" theories to describe why my client should engage with their constituent's objections, the bottom line is that I am simply recommending that my client make some human contact and show empathy to someone who is a bit scared and does not know how to express it. When looked at this way, who needs the theory, eh?
In my focus groups, when I asked about their use of scientific research, my BS detector went off when I heard their responses. Most of the groups were comprised of individuals who did not have a research or scientific background - so to them research was "complicated" and "biased". Moreover, participants doubted the methods and conclusions drawn, saying they just did not know how the research could be so precise as to isolate specific effects of certain substances. I had no reason to doubt these stated responses, but the BS detector told me that these participants were actually scared of research and embarrassed about not knowing how to use or interpret it. This was the unsaid truth in the groups.
Knowing this, my recommendation to the client was not modify how research was presented - rather I simply told them to present the research as is, but to LISTEN and actually AGREE with most everything their constituents had to say about the research, even though, in my clients eyes what was being said was not true. I said in my presentation "What you need to realize is that these people don't know how to use research and are scared of it. If you actually agree with them in general, and then softly point-out point out how the research you are discussing is different, they will be much more likely to listen to you. You can teach them without triggering their concerns over not knowing too much about scientific research."
The recommendation is based on a number of theories in Gestalt Psychotherapy:
- In every transaction, the other person has "the power", not you. If you can yield to this, your mind begins to open-up and listen to the other person. In this example, my client's constituents are actually obliterating the research with a simple thought - how's that for power. They don't have to take any action - all the constituents have to do is say "I don't believe what you're putting in front of me." When this happens, you're dead in the water, and there is no way that you will be able to out maneuver this state of mind. There's a wonderful saying I use - "Never argue with an idiot - they just drag you down to their level and beat you with experience." Now I'm not calling anyone involved in this research an idiot, but the saying holds. If someone is not going to see your point of view, no matter how "right" you are, you're the one that walks away frustrated and angry. The other person walks away "victorious", because they could care less about increasing their knowledge - they care more about winning and have in fact done so.
- There is a wonderful model called the "Co-dependency triangle". In it, people involved in any transaction can be described as "Persecutors", "Rescuers" and "Victims." In my client's case, the constituents were the Persecutors, and my clients were the Victims (i.e. "How could you not accept this research... it's completely valid"). The way to deal with a Persecutor is not to react to what they are saying, but to simply listen to it, keep listening and then show that you have listened. The persecutor simply wants to be heard. When that happens, the transaction opens to more possibilities.
- The Gestalt Cycle is a model that explains how a person makes contact with his or her environment. The very first part of the cycle assumes that a person is able to recognize that his environment is saying something to him (ever day-dream at the office and have someone call your name for 5 seconds - you're missing the environment), and that what the environment says is valid (ever feel slightly warm, but wait too long before you blast your A/C - you're dismissing yourself and your hot environment). My client was not recognizing any of their constituent's objections as valid. When this happens, my client is in a better position to respond.
In closing, I have taken on as my position to bring "humanity" into market research. While I have used three very "highfalutin" theories to describe why my client should engage with their constituent's objections, the bottom line is that I am simply recommending that my client make some human contact and show empathy to someone who is a bit scared and does not know how to express it. When looked at this way, who needs the theory, eh?
It's Time To Stop Using Qualitative Research As A "Pass/Fail" System
Qualitative Research, and in particular, Focus Groups, are the target of a lot of criticism because of their inability to accurately determine the success or failure of what is being tested. If I hear one more New Coke story, or Malcolm Gladwell's Herman Miller example in "Blink", I think I'll puke. However, despite my gastrointestinal reflux reflex, the fact is these stories ring true, and qualitative research (and I would argue Quant as well, but I don't do either Windows or Quant so I won't speak to it) should not be used to pass or fail a product.
A good psychotherapist will know within a second what someone's reaction is to a new product, regardless of what they state. A good researcher will know how to use that reaction to get relevant data for their clients. The main way to do this is to take the "focus" off the "evaluation" and put it more into a moderated discussion, or into a realm where people do not have to justify anything. "Evaluations" and "justifications" (e.g. Why did you react this way? What do you like... what don't you like...) have a very technical term in Gestalt Psychotherapy - they are called Bullshit. What ends up happening is that even though the researcher/therapist can see how a person reacts to a new product within a second, often the person doing the reacting is completely unaware of their reaction. As such, when they are asked to justify or explain, they end up being confused and spewing-out answers in order to please themselves or the moderator. This also has the effect of "fixing" a person onto a specified answer. Participants will not want to be seen as "flip-flopping" in front of a group, so they tend to keep justifying a position that may have changed.
There are very few reactions someone can have when seeing something new - positive, negative or neutral (neutral could also be called confused). What needs to happen in a product evaluation is that even though the moderator knows initial reaction, the moderator must let participants sit with their thoughts and feelings for 30-60 seconds before people have a chance to speak.
From there, the moderator simply probes with a "Well..." and lets the discussion flow from there. What begins to happen in the discussion is that people are much more in-tune with their initial impressions (they've had a chance to sort through their own BS so their clarity of thinking is better), and will often what will happen is that people will begin to refer to their initial reaction, and discuss how it changes as they have had time to sit with their impression, or to hear others in a group.
So, here's where the pass/fail concept gets thrown out the window. Someone could have had a negative initial split second view of the product, but when they sit with their thoughts and feelings, the negative impression can be melted away - and this is what the product test measures - the change in opinion (if any) from initial reaction through to a final opinion. The change can go from negative to positive, positive to negative or from any change in between. The only thing that a moderator needs to be on the lookout for is whether the change in views someone has is a real phenomenon or whether it a result of "group-think", an attempt to please, or an attempt to "kybosh" a good idea. These three behaviours are considered neurotic and unproductive, and a good moderator will know how to get around them, and know the extent of legitimate influence or neurotic influence.
So what we wind-up getting from a concept test is not a pass/fail result at all. Instead what we get is a dynamic result (which the last time I checked is the way a market actually operates). We can measure initial opinions (which, to a good marketer or advertiser should mean very little - their job is to change opinions after-all), but more importantly we can measure how those opinions can be changed and influenced. The group discussion will illuminate what factors changed their initial impressions of the product, or what factors keep people stuck in their initial impressions. The goal for the moderator is to ensure that the conversation is kept free of that very technical Gestalt term - bullshit.
And speaking of that, it is worthwhile to return to the initial split-second reaction observed by the moderator when the product is first exposed - he needs to be keenly aware of it as people are speaking. It is possible that someone could "fudge" their explanations, or change their story based on what they hear in the group. It is important for the moderator to check-out what a participant is saying versus the initial observed reaction. There is nothing wrong with a moderator calling someone and saying "I hear that you initially said you liked Product X, but I happened to see you out of the corner of my eye and I would have made a bet that you didn't like it - just let me know how far off I am." All this does is re-frame the participant back to their initial thoughts so that the data is more accurate, and we can more accurately measure the progression of thoughts and opinions.
A good psychotherapist will know within a second what someone's reaction is to a new product, regardless of what they state. A good researcher will know how to use that reaction to get relevant data for their clients. The main way to do this is to take the "focus" off the "evaluation" and put it more into a moderated discussion, or into a realm where people do not have to justify anything. "Evaluations" and "justifications" (e.g. Why did you react this way? What do you like... what don't you like...) have a very technical term in Gestalt Psychotherapy - they are called Bullshit. What ends up happening is that even though the researcher/therapist can see how a person reacts to a new product within a second, often the person doing the reacting is completely unaware of their reaction. As such, when they are asked to justify or explain, they end up being confused and spewing-out answers in order to please themselves or the moderator. This also has the effect of "fixing" a person onto a specified answer. Participants will not want to be seen as "flip-flopping" in front of a group, so they tend to keep justifying a position that may have changed.
There are very few reactions someone can have when seeing something new - positive, negative or neutral (neutral could also be called confused). What needs to happen in a product evaluation is that even though the moderator knows initial reaction, the moderator must let participants sit with their thoughts and feelings for 30-60 seconds before people have a chance to speak.
From there, the moderator simply probes with a "Well..." and lets the discussion flow from there. What begins to happen in the discussion is that people are much more in-tune with their initial impressions (they've had a chance to sort through their own BS so their clarity of thinking is better), and will often what will happen is that people will begin to refer to their initial reaction, and discuss how it changes as they have had time to sit with their impression, or to hear others in a group.
So, here's where the pass/fail concept gets thrown out the window. Someone could have had a negative initial split second view of the product, but when they sit with their thoughts and feelings, the negative impression can be melted away - and this is what the product test measures - the change in opinion (if any) from initial reaction through to a final opinion. The change can go from negative to positive, positive to negative or from any change in between. The only thing that a moderator needs to be on the lookout for is whether the change in views someone has is a real phenomenon or whether it a result of "group-think", an attempt to please, or an attempt to "kybosh" a good idea. These three behaviours are considered neurotic and unproductive, and a good moderator will know how to get around them, and know the extent of legitimate influence or neurotic influence.
So what we wind-up getting from a concept test is not a pass/fail result at all. Instead what we get is a dynamic result (which the last time I checked is the way a market actually operates). We can measure initial opinions (which, to a good marketer or advertiser should mean very little - their job is to change opinions after-all), but more importantly we can measure how those opinions can be changed and influenced. The group discussion will illuminate what factors changed their initial impressions of the product, or what factors keep people stuck in their initial impressions. The goal for the moderator is to ensure that the conversation is kept free of that very technical Gestalt term - bullshit.
And speaking of that, it is worthwhile to return to the initial split-second reaction observed by the moderator when the product is first exposed - he needs to be keenly aware of it as people are speaking. It is possible that someone could "fudge" their explanations, or change their story based on what they hear in the group. It is important for the moderator to check-out what a participant is saying versus the initial observed reaction. There is nothing wrong with a moderator calling someone and saying "I hear that you initially said you liked Product X, but I happened to see you out of the corner of my eye and I would have made a bet that you didn't like it - just let me know how far off I am." All this does is re-frame the participant back to their initial thoughts so that the data is more accurate, and we can more accurately measure the progression of thoughts and opinions.
Monday, July 30, 2007
If You Want "Real Usage" Figures Go Elsewhere!
Most reported "usage" data from qualitative research should be treated as suspect, and I would go so far as to recommend that my clients NOT make decisions based on usage data reported in qualitative research.
Before I go into details, let me provide a key reason for my recommendation. I have a summer cottage, and a few times a year we have myriad visitors, like cousins and friends of cousins who go to the local casino. The funny thing is that most every visitor to this casino comes back "a winner" - they doubled their money, beat the bank, killed at Blackjack and knew just how to bet at Roulette. My question to them is simply this - "What farking casino are you people going to, because every time I go, I lose money." When I challenge them with this, what invariably happens is that these "killer winnings" get whittled down to statements like "well, we won enough to pay for dinner" or "I did well at roulette, but I lost a lot at blackjack."
Now, most people know exactly what's going on. A bunch of beer-drinkin, nature-luvin, Muskoka-bound, middle-aged Canadian "dudes" want to "play up" their winnings so that they can puff-out their chests bigger than the next guy. Of course, we know full-well that the odds of so many people winning so much money are slim to none.
So, how does this relate to reported usage in qualitative research? There are three ways. The first is rather intuitive - many people in a group will "fudge" just a bit to blend-in. The fudging may be to ensure that they are equal with the group, or ensure that what they say corresponds to social norms. Some may just be scared of telling the truth. This forms the basis for why we can't rely on qualitative research to provide us with numbers of any sort of reliability.
However, the next two reasons are where a deeper understanding of psychology comes into play. The second reason my story relates to qualitative research is because it is not the reported numbers or usage that is important to us. What is important to us is the deeper feelings and thoughts that underly the answer. If a moderator's BS detector is going-off full-tilt like mine does when I hear these "casino stories", I know to probe further. What is it about this reporting that is common to most of these people? There is a hope that they will win, and a fear of looking foolish in front of others.
The third way this relates to qualitative research is what I do after the fact. In my casino example, I actually stood quite firm. The way I questioned people about the casino they went to was such that they knew I would likely follow-up with "Exactly how much did you win?" or "Tell me the method you used that night?" When people have to compound their "exaggeration", they usually back-down. Now, in a focus group - it is highly unlikely that I would ask "what casino are you going to..." - mainly because I mentioned that it is not important for me to get the truth in this instance. Instead, what I would likely do is say "Wow - there's a lot of excitement going on in here... Can someone show me what it is like to win a lot of money at a Casino?" Engaging them in their fantasy will show the client how to enhance a brand, experience or communication to play to the perception of excitement that these players have. Attempting to find the "truth" of the matter would serve to ground participants in a reality that they would rather not face, and one that would not be appropriate for a casino message.
Before I go into details, let me provide a key reason for my recommendation. I have a summer cottage, and a few times a year we have myriad visitors, like cousins and friends of cousins who go to the local casino. The funny thing is that most every visitor to this casino comes back "a winner" - they doubled their money, beat the bank, killed at Blackjack and knew just how to bet at Roulette. My question to them is simply this - "What farking casino are you people going to, because every time I go, I lose money." When I challenge them with this, what invariably happens is that these "killer winnings" get whittled down to statements like "well, we won enough to pay for dinner" or "I did well at roulette, but I lost a lot at blackjack."
Now, most people know exactly what's going on. A bunch of beer-drinkin, nature-luvin, Muskoka-bound, middle-aged Canadian "dudes" want to "play up" their winnings so that they can puff-out their chests bigger than the next guy. Of course, we know full-well that the odds of so many people winning so much money are slim to none.
So, how does this relate to reported usage in qualitative research? There are three ways. The first is rather intuitive - many people in a group will "fudge" just a bit to blend-in. The fudging may be to ensure that they are equal with the group, or ensure that what they say corresponds to social norms. Some may just be scared of telling the truth. This forms the basis for why we can't rely on qualitative research to provide us with numbers of any sort of reliability.
However, the next two reasons are where a deeper understanding of psychology comes into play. The second reason my story relates to qualitative research is because it is not the reported numbers or usage that is important to us. What is important to us is the deeper feelings and thoughts that underly the answer. If a moderator's BS detector is going-off full-tilt like mine does when I hear these "casino stories", I know to probe further. What is it about this reporting that is common to most of these people? There is a hope that they will win, and a fear of looking foolish in front of others.
The third way this relates to qualitative research is what I do after the fact. In my casino example, I actually stood quite firm. The way I questioned people about the casino they went to was such that they knew I would likely follow-up with "Exactly how much did you win?" or "Tell me the method you used that night?" When people have to compound their "exaggeration", they usually back-down. Now, in a focus group - it is highly unlikely that I would ask "what casino are you going to..." - mainly because I mentioned that it is not important for me to get the truth in this instance. Instead, what I would likely do is say "Wow - there's a lot of excitement going on in here... Can someone show me what it is like to win a lot of money at a Casino?" Engaging them in their fantasy will show the client how to enhance a brand, experience or communication to play to the perception of excitement that these players have. Attempting to find the "truth" of the matter would serve to ground participants in a reality that they would rather not face, and one that would not be appropriate for a casino message.
Subscribe to:
Posts (Atom)
