Marketing Research Blog

At conferences over the years, I’ve encountered lots of in-house marketing research folks who were disappointed that few of their recommendations were put into action. I told them that my experience was quite the opposite–that most recommendations that I had made were acted upon. When they asked why, I simply told them that “I was so convinced that what I was recommending was the right thing to do that I spent the necessary time selling them on the correctness of the conclusions and recommendations.”

In retrospect, I believe that there were two main reasons that most of the research that I conducted, as an in-house research, was acted upon:

1. I saw myself as a team member, not simply a supplier to in-house clients–as some folks do. So it was always “us” not “we” versus them. In my opinion, the in-house supplier-client conception that many folks seem to promote is far off the mark; and
2. I was proactive, not reactive. For the vast majority of the research that I conducted as an in-house research type, I saw the need and implemented the research with out being asked.

It probably boils down to the fact that there are 3 types of people (and organizations):

• those who make things happen;
• those who watch things happening; and
• those who wonder what happened.

As for me, I’m one who makes things happen!

 In the OJ Simpson Trial, the DNA evidence was so incredibly strong that some pundits suggested that the qustion “Is OJ guilty?” constituted an intelligence test. Though not politically correct, the “yes” or “no” clearly indicated one’s reasoning ability since the likelihood that the DNA came from anyone other than OJ was less than 1 in a billion.

I’ve long used an equally simple way to gauge the IQ of folks who consider themselves to be marketing researchers. Let me explain!

Suppose you have created two versions of an ad designed to increase your brand’s unaided awareness. In a marketing research study, unaided awareness for Ad “A” is found to be 21% and 18% for Ad “B.” But the difference is not statistically significant (p > .05). Assuming that there are no other differences between the two ads and that you must make the decision right now, the question that you should ask the researcher is “Which ad would you recommend be run in the media?”If the so-called “marketing researcher” responds with something like “It doesn’t matter or flip a coin,” I’d humbly suggest that you fire him or her and that you run Ad “A.” That is, go with the higher number. The best information that you have is that Ad “A” performed better on unaided awareness than did Ad “B.” The fact that there was no significance difference only means that the observed 3-point difference could have occurred more than 5 times out of 100–not that no difference was present.

I’ve posed this problem to numerous folks who consider themselves “marketing researchers” including some Ph.D. types. Sadly many of these supposedly learned folks said, “Flip a coin; it makes no difference.” Unfortunately, after statistics 101, they never got their brains out of low gear; clearly they flunked this simple intelligence test.

Suppose that, in a Fox News Poll, it is reported that likely voters prefer Obama over McCain by 2% but that this difference in preference is within the margin of error. This would mean two things:

1. the difference in preference is not significant at the at the .05 level of statistical significance; and
2. That a difference in preference of the reported magnitude (2%) would be expected to occur by chance 1 time out of 20 (that is, 5% of the time that such a survey with whatever sample size they used was conducted).

Had I conducted that survey, I would have reported the results to Fox News as:

• In a poll of likely voters, 48% preferred Obama and 46% preferred McCain a difference that was not statistically significant (P < .05).

So the way that I portrayed Fox News as having reported the results would have been true and accurate.If Fox had happen to mention that the margin of error was +/- 3%, that statement would have meant that the sample size (the number of people surveyed) was approximately 1100. The magnitude of the margin of error is always a reflection of sample size. A sample size smaller than 1100 would have produced a margin of error larger than 3%. Likewise, a larger sample size than 1100 would have resulted in a margin of error less than 3%.

Welcome world! I’m regularly going to be blogging about marketing research:

• How to use it;
• How not to abuse it;
• How to get the most out of it;
• I will make many of the technical aspects of marketing research understandable; and
• I will encourage you to ask questions

The first topic that I plan to blog about will be “margin of error,” a term that we often hear on news programs and that we will be hearing more and more as the Presidential Election draws nearer.