INSIGHT on INSIGHT: The Components of Cost for Insight

Some people view research as an expense.  Others view it as an investment.

In the first case, the goal is to minimize the cost.  In the second, it is to maximize the return-on-investment.

To accomplish either end, one must have a basic understanding of the four components that both drive the majority of cost for research and determine how much value it delivers.

Those four components are the number of questions asked (scope), the number of respondents involved (scale), how hard those people are to find (incidence) and the amount of analysis done (depth).

Put in a formula, it would look something like this:

Cost = (Scope * Scale * Depth) / Incidence 

Of course, accelerated or fast timelines and introduce additional costs.

While available budget is often the ultimate limiting factor, smart design decisions can dramatically expand what is accomplished with any amount of funding.

SCOPE: 

The project’s core objectives should define the scope of any project.  However, the objectives do not always dictate what questions or how many questions need to (or should) be asked.  The scope will determine how much time participants are asked to give in exchange for compensation, how much behind-the-scenes time is spent with design and how much data will be produced for possible analysis.  Some factors to consider include:

What questions are necessary to qualify people for the research?

Can questions be asked (and answered) directly, or are they more accurate if answered indirectly or derived from other questions?  Should both approaches be incorporated for comparison?

What groups need to be identified in the analysis?  Demographics are easy to ask, but what questions are right to identify other groups based on psychographics or habits?

What are the topics or themes needing thorough exploration?

What questions will be important to provide more context around how the core questions are answered?

What questions are needed to provide a complete picture of those topics or themes, and prevent the risk of making inaccurate conclusions due to incompleteness?

Are there economies of scale, making it worthwhile to incorporate a few more questions within this project versus leaving them unanswered or creating the need for an additional project in the future?

Is there data that would be valuable for future reference even if it is not needed immediately?

SCALE: 

Determining the number of research subjects (or sample size) is based on needing confidence that the data is accurate and representative of the larger population.  It is theoretically possible to interview or survey a single person who perfectly reflects the ‘average’ in all dimensions.  However, most people have unique bias or preferences that are only useful when aggregated with and understood relative to others. 

At the same time, relying on too small of a sample limits the ability to identify unique dimensions or opportunities among smaller sub-segments.  Each additional participant will demand incremental recruitment and compensation costs, and in some cases will increase the time spent on execution or analysis (such as conducting more one-on-one interviews).  While qualitative research is concerned with sample size in a slightly different way than quantitative research is, the principles are still largely the same.  When deciding scale for your project, consider the following:

What unique sub-groups do you need to read results individually or be able to make comparisons across?

What is the smallest subgroup of respondents you need to understand?

What specific reports will be delivered from the results?  Do you need sufficient sample to create reports for certain retailers or to compare results versus competitive brands?

Are you testing different variations of a stimulus (a product, package, branding component, etc.), and need to measure each in a monadic (isolated) context?

What expectations will others have regarding the necessary sample size to consider the results reliable?

How much statistical accuracy do you care about?  Will you really interpret the data differently if the margin of error is +/-2% v. +/-5%?

INCIDENCE: 

Incidence refers to how much of the given population will qualify for the research.  In other words, if one hundred people were contacted to participate, how many would be able to?  High-incidence research is far less expensive than low incidence-research.  In fact, recruitment costs can increase exponentially as incidence drops due to the large amount of resources consumed while filtering through all potential respondents to find the few that qualify.  

Consider the following factors that will influence the incidence of your project:

Do you want a sample that is nationally representative (based on factors like gender, age, income, ethnicity, population density or geographic region) or are you looking for a particular segment that is not representative?

What factors determine the basic profile of the population you are trying to learn from?

Is it necessary to cut off certain groups?  For example, is there a good reason to exclude people below a certain income or above a certain age?

Do you only want to learn from people currently using the category (or buying your brand) or could the research reveal opportunities to attract new users or buyers?

What marginal groups could provide valuable information beyond core respondents (perhaps your prime prospects), such as those that are slightly younger or older, or lower or higher income than the specific profile you’re focused on?

Do respondents need to shop at certain stores?  If so, do they have to buy your particular category at that store (hint:  Retailers have a lot of interest in converting store shoppers into buyers of more categories at their store.  They’re not just interested in current buyers)?

DEPTH: 

As the number of variables in a data set grows, the possible ways to analyze that data grows exponentially.  It doesn’t make sense to spend money collecting data that never gets analyzed.  However, there is also a point of diminishing returns where further analysis does not produce further insight.  Our goal is to always do just a little more analysis than is necessary.  We like to see the point of diminishing returns to know we’ve gotten the majority of value out of a project.  As you plan your analysis, consider the following:

What core groups need to be looked at just to answer the project’s objectives?

Do you want a systematic approach to analysis to uncover unexpected insight or should the analysis solely focus on looking right where answers are expected?

Are their certain demographics worth analyzing that could reveal unique habits or attitudes or opportunities?

Is the research trying to create principles or guidelines or models that might require iterative solutions that need to be re-evaluated with the data?

What types of questions have been asked?  The data produced by ranking or rating can require much more effort to completely analyze compared to multiple choice ‘select one’ or ‘select all that apply’ questions.

Are there particular types of more complex analysis expected or desired?

For qualitative research, is video necessary or can audio files serve the same purpose? 

Will highlight videos or audio sound bites be an important part of bringing the results to life and selling the story?

By thinking through the above components, more educated and better tradeoffs can be made that help keep the cost of the research down (by eliminating unnecessary elements) while increasing the ROI (by getting the most out of the data that is collected).