INSIGHT on INSIGHT: Utilizing Scale Questions
WHAT AND WHEN: Proper usage
Many people are familiar with the use of multi-point scales to gain more detailed understanding of preference for certain variables. These may be Likert scales with 5, 7 or 10 points, asking respondents to pick the point that best applies to them.
Instead of having binary knowledge (i.e. Do you like me? Yes/No), scales help understand different degrees of answers (i.e. On the following scale from totally hate to absolutely love, please indicate how you feel about me.)
Multi-point scales acknowledge the fact that many questions cannot be accurately or completely answered with just one positive and one negative option. They acknowledge a spectrum or increments of agreement exist.
WHY: The benefit and value
Questions using scales are more demanding on respondents, but provide more detailed information that can create the option to do more flexible or detailed analysis. This also allows for clearer interpretation of results that might be more ambiguous if captured through other techniques.
A variety of design approaches can be used depending on what information is desired for analysis.
However, overuse of scale questions can exhaust respondents (who will start to gravitate to consistently picking certain scale points) and burden analysts with data that requires a significant amount of extra processing.
HOW: Tips to guide a basic approach
The increased fidelity of multi-point scales comes with additional effort and responsibility. This, in turn, creates more opportunity for mistakes to be unintentionally made.
To minimize the risk of mistakes, follow these guidelines:
Realize if you are using a balanced or skewed scale: Scales are generally designed to have a neutral center point and an equal number of equally spaced positive and negative points. Make sure your scale doesn't bias the data.
Decide if individual scale labels are needed or end-points are enough: Some scales may label every point while others may just label the end points (and possibly the middle point), leaving more room for respondents to interpret the definition of each point between these.
Determine if a neutral center point should be offered: Odd-numbered scales create a center point while even-numbered scales force respondents to lean at least slightly toward one of the end points. While neutral center points are most common, forcing preference could reveal bias or create an artificial sense of preference.
Have an analysis plan: Multi-point scales require significantly more effort to analyze and interpret, often causing many people to simply do a Top 2 Box v. Bottom 2 Box (T2B v. B2B) comparison, which isn’t that different than a binary yes-no question. Data that goes unanalyzed wasn’t worth getting in the first place. Consider how different weighting or assessments of polarization can further clarify what story the results are telling.
Be aware of the risk of response patterns that concentrate on a limited range in the scale or only use end points: Some respondents naturally gravitate to certain parts of a scale (i.e. all answers are near the middle of the scale or the top of the scale). This bias can confuse results when two respondents each give the same attribute their lowest overall score, but that score is a 7 for one of them and a 1 for the other respondent.
Be aware of including items that do not fit the scale points: Longer lists can often incorporate questions or statements that are not consistent with the formatting of the scale. These can cause confusion and dirty data when respondents are forced to answer a question that can’t be easily answered.
APPLICATION: What to do with the results
Multi-point questions are just a tactic to control data capture. They allow for the analysis of results in a variety of ways to provide multiple approaches to a question or problem. These different views could zero in on a single, consistent answer or reveal that the solution is less clear, leaving the interpreters with more responsibility to arrive at the right answer.
In general, surveys should limit the use of scale questions and only use them when other techniques will not get adequate data to derive the desired insight from.