I spend a lot (perhaps an unreasonable amount) of time planning, editing, and shipping surveys. My product team at Vox Media uses surveys to gain rapid feedback about the products we build or changes we make. For my editorial and leadership colleagues, a survey helps us spot audience trends and adjust our publishing and revenue strategies accordingly.
Among my go-to research methods, I find surveys to be the best way to get some early signals around experience issues and opportunities. On its own, a good survey allows me to segment user feedback by audience type. Coupled with a follow-up interview or usability test, the survey serves as something akin to lead generation—a means to getting in touch with the right people for a given topic.
Because I encourage everyone I work with to conduct research, other teams often ask me for help in drafting surveys. One question I get relates to answer scales: what’s the right scale to use? For example:
- On a scale from 1 (highly dissatisfied) to 9 (extremely satisfied), how satisfied are you with this newsletter?
- How would you rate this podcast on a scale from 1 (bad) to 5 (great)?
- How did you like this article—thumbs up or thumbs down?
My answer, as with almost every answer any researcher will ever offer until the end of time, is “it depends.” A wide scale (from 1 to 9) introduces a lot of noise. If your goal is to gather feedback quickly, the wide scale won’t give you clarity. After all, how does a 6 differ from a 7? Or a 2 from a 3? When you want a strong signal, narrow your scale. Some of the narrow scales I use in user or audience surveys run from 1 (extremely frustrated) to 5 (delighted), or 1 (dislike) to 3 (like).
If you’re researching something longitudinally—that is, tracking feedback over time—the wide scale is beneficial. For example, as your product onboarding flow evolves, you’ll want to track user satisfaction and peg it to design changes. An onboarding experience that once elicited a 4 might become a 6 a few months later, and work its way up to a 7 after that. When you’re studying something over time, widen your scale.
Just remember that the beauty of surveys is that they’re iterative. You can test your surveys with a small group of respondents to dial in the questions and scales before sharing with the world. Or you can experiment from survey to survey if you need to adjust the strength of the signal you want to see in the responses.
