INSIGHT on INSIGHT: 15 Principles for qualifying the right respondents
In an overly simplified sense, shopper insight is all about asking the right questions to get the right answers from the right people.
It isn’t very useful to know what single males or retired couples think about baby diapers.
Or what people making minimum wage think about Cadillac.
The data used to find new insight must be based on the right respondents. In most cases, this involves identifying or recruiting a small sample of respondents that are representative of a larger population. The goal is to identify insights among the sample that can be accurately extrapolated to apply to the larger population.
Not understanding or not selecting the right sample can compromise the data and produce misleading results when that sample does not reflect the larger population.
Clients can make this mistake either by overthinking the criteria and creating too narrow of a respondent profile or by overlooking important qualifiers that create too broad of a sample.
However, getting the right profile is not that difficult if the following principles are considered:
PRINCIPLES FOR QUALIFYING RESEARCH RESPONDENTS
Balance the distribution of the sample to reflect the broader population: Surveys typically structure the outgoing sample to reflect core demographics (perhaps primarily because they can be easily recognized and measured). It is typically smart to at least balance sample for age, income, ethnicity and geographic region.
Don’t just focus on females: The perception that 80% of all purchase decision are made (or influenced) by females caused a lot of companies to ignore male shoppers. In some situations, this still makes sense, but more and more categories have significant emerging opportunities by better appealing to male shoppers.
Make sure the respondent is actually making the decisions you’re studying: This can be confirmed at a category level (when was the last time they bought the category?) or a more general confirmation that the respondent is doing the majority of the shopping for their household. In practice, the vast majority of survey panelists are also the primary shopper for the household so this typically does not screen out a significant number.
Set quotas when necessary: When specific comparisons are needed, put controls into the data collection to make sure a sufficient number of respondents are captured for each variable. However, be aware that quotas can alter the data set so it is no longer representative of the larger population.
Realize that samples are rarely random, meaning there is always some bias: If it hasn’t crossed you mind yet, let’s just acknowledge that a certain personality profile is more likely to sign up to participate in online surveys or respond to an invitation for qualitative research. Most notably, these people tend to be more opinionated, find more satisfaction in sharing their opinion, be more likely to value the compensation offered by panels (such as cash, gift cards or frequent flyer miles), and generally be less busy at the time of participation. This means some panels have a higher penetration of groups like amazon.com shoppers (because they were attracted by the offer of an amazon.com gift card) and other groups can just be more difficult to reach (like small business owners, doctors or busy working moms with multiple kids).
Consider including marginal shopper groups: While the product you’re researching may have a clearly defined prime prospect, it can be very insightful to include a slightly larger group (such as slightly lower or higher income or slightly younger or older ages than the specific definition of your prospect) to both validate the fit of your prime prospect and potentially identify incremental opportunities where different or new products could broaden the buyers you are able to attract.
Include groups that represent incremental volume potential: Only studying current category buyers or current store shoppers eliminates the ability to identify, quantify and figure out what it would take to attract 100% incremental volume through new category buyers or new store shoppers.
Set a time frame for how recent qualifying behaviors had to happen: As a rule of thumb, past 12 month behavior (abbreviated P12M) is the most common way to qualify respondents (i.e. P12M Walmart shoppers, P12M dog food buyers). However, it may be more appropriate for some categories with particularly long or short purchase cycle to only qualify people that have bought in the past 3 months (i.e. milk) or the past 2 years (i.e. cars or TVs).
Distinguish between the shopper/buyer of a product and the consumer/user of that product: Much of the time, the shopper/buyer/consumer/user is all the same person. However, many households and many categories have a shopper that buys items that someone else uses (i.e. mom buying razors for the husband). In these situations, insight can potentially be gained from both the shopper and the user depending on the objectives. However, the shopper shouldn’t be answering questions about use and the user shouldn’t be answering questions about shopping.
Consider including lapsed or prospective buyers in addition to current buyers: For many categories, lapsed users represent both an important source of information about what causes people to stop buying a category and potentially a high concentration of future buyers. Prospective buyers may not be able to discuss the non-existent behaviors they have related to the category they do not buy, but they can provide a wealth of information about perceptions, information-seeking and motivation to make a future purchase. This is particularly important for items with a longer purchase cycle (like a car).
Include additional respondent groups to allow for benchmarking or comparison: Making sense of the buyers of a particular brand is often only possible by comparing them to buyers of other brands (or non-buyers of the client’s brand). Interest in one new product only makes sense when it can be compared to interest in other products. Make sure your sample is not so focused that the results can only be evaluated in isolation.
Screen for psychographic or behavioral barriers: When trying to identify barriers or find ways to change existing habits, make sure to include understanding of factors that may be dictating existing behavior and be beyond your control to change. This could include food allergies, lack of access to certain products or stores (i.e. no car, no Costco in the area) or lack of need (i.e. no driver’s license to need a car or no lawn to need a mower).
Screen for bias: While less common now, recruiters used to ruthlessly screen out people that worked for or had relatives that worked for retailers or manufacturers or agencies related to the topic being studied. Nowadays, the consolidation and growth of many companies along with broader social networks has reduced this concern.
Screen for professional respondents: Yes, there are definitely a few people who make their living filling out surveys and participating in interviews. While some may view this as a concern, these individuals can actually be better “trained” as a respondent and actually provide better, more thorough answers compared to someone taking their first survey. Participating in a lot of research does not make a respondent bad. It is only the possible bias related to that participation (accurate awareness of uncommon information or mindless clicks to get a survey complete) that can be a concern.
In qualitative research, be sure to include articulation criteria: There is nothing more painful than interviewing someone that answers every question in five words or less. Simple techniques like asking potential respondents what they did last weekend or what they have planned for this weekend will be a clear indicator of how well they can answer an open-ended question.
No matter how hard you try, any set of data will have some amount of noise due to poor respondents. However, better screening can dramatically reduce the issues related to this.
Take time to think through (but don’t over-think) who needs to be answering the questions you have.