Quick Quip About Bias

Just a quick entry today. When I started out in the industry and used to tell other people who were just starting out what I did, they would be quick to say "I hope you don't bias your results and your data." I quickly learned to say "Of course I bias my results and data - I just know what I'm doing, that's all."

I firmly believe that any research, regardless of how careful the designers are, will bias its participants in one way or another. I believe that a researcher proves his or her mettle when they are able to analyze bias better than they are able to analyze the raw data itself.

I did 25-focus group study and found that participants would be comfortable if a particulary contentious government policy were impelmented on a voluntary basis, instead of a mandatory basis. Thought the client trusted me, they were very surprised to hear the result, and I don't blame them. Here was my explanation - I asked participants about whether it should be voluntary or mandatory after nearly two hours of previous discussion with it. As such, they were comfortable with certain aspects of the policy and other factors such that they felt a voluntary implementation would be just fine.

However, if you asked the same question on a quant study, most likely you would get a majority of people wanting a mandatory implementation. There are two reasons - first, a quant study does not allow (in my opinion) to get intimately comfortable with a topic, so they tend to answer from a place of alienation and fear especially if they do not know all the details of a policy - and mandatory sounds like it will provide more protection than voluntary. The second reason is why would someone answer that they want less of something (i.e. a voluntary implementation), when we present them with the option to get more of something (i.e. a mandatory implementation) at no additional cost or burden to them?

Here is how I interpreted the results:

- The participants got comfortable with the topic enough so that they felt a voluntary standard would suffice. If the client wants the Canadian public to accept a voluntary standard, they would need to spend significant time and effort to educate and communicate with them the same way that a 2-hour focus group got them comfortable with the topic. If the client has the resources to do this, then a voluntary implementation would work.

- If, however, the client did not have the resources available to intimately educate the public about the policy, then they would most likely want a mandatory implementation of it.

The analysis above was more based on an analysis of bias of the survey instrument and method than it was on any of the particular results that were contained in the study.

As a final strategic note, if the client did not have the resources or cash to communicate intimately with the Canadian public to try and create acceptance of a voluntary standard, they would still have to pay-out significant sums of money if policy became mandatory. Specifically, they would need to set-up monitoring and enforcement functions among the industry they regulate. As such, the client would need to investigate which option would cost them less to implement, but realizing that both options would take a significant amount of money to implement.