Writing statistically insightful survey questions is hard — there is a whole science behind it! To get you started on CivicBell, we recommend avoiding the following four pitfalls:
❌ Avoid leading questions
- A leading question assigns positive or negative context, which may influence respondent answers.
- Leading: How strongly do you approve of the dutiful mayor?
- Non-leading: How would you rate your approval of the incumbent mayor?
- This leading question implies strong approval and depicts the mayor as dutiful. Instead, replace these leading elements with neutral language.
❌ Avoid loaded questions
- A loaded question assumes the answer to an underlying question.
- Loaded: Which candidate will you vote for?
- Non-loaded: Which candidate do you prefer?
- Non-loaded: If you plan on voting, which candidate will you vote for?
- This loaded question presumes that the respondent will be voting. Instead, reframe the question to explicitly target your desired respondent or generalize the question to better incorporate respondents.
❌ Avoid multiple questions at a time
- Separate several topics into multiple questions: refrain from asking multiple questions in one.
- Multiple: Would you support an increase in school funding and a higher public works budget?
❌ Avoid binary questions
- Yes/No questions are a great way to receive sentiment, but they don’t allow participants to be “indifferent” or “undecided”.
- Always have, e.g. an “undecided” or “indifferent” option available when asking Yes/No questions to truly understand which of the participants are for or against something.