Your data will only be as good as the questions you ask. A poorly written questionnaire will lead to poor data.
This blog will help you cut through the rubbish and put together a questionnaire which will ensure the data you gather is going to give you the answers you need.
The methodology you use will depend on budget, timing and your audience. However, the points below apply whether you are conducting your research over the telephone, online, face to face, or even by post (yes! some research still happens by snail mail).
BE TRANSPARENT
We are all concerned about data falling into the wrong hands, so it’s prudent to include an introduction outlining why you are conducting the research, what you will do with the answers provided, how they will be stored, and who will have access to your data.
Of course, you must ensure that you are storing the data in a GDPR-compliant way, and you must give your respondents the opportunity to opt out of the process.
The Market Research Society (or MRS, the UK’s industry body) has published a series of guidelines to which all members must adhere. You can read the full document here but one major point is that MRS members will never provide clients with details which enable respondents to be identified – unless respondents have given their permission.
BE WELL ORDERED
No-one wants to open a questionnaire and immediately be asked to give their gender, age, income, and postcode. You need to build a rapport with respondents first. Do this by asking non-intrusive questions to begin with. If you need to know age, income or other sensitive information, then ask these questions towards the end of the questionnaire when respondents will be feeling more at ease with the process.
Make sure that your questionnaire flows well. Questions about similar things should be grouped together. Think about those questions which might affect how respondents answer other questions. For instance, if there is a section which only applies to people who have visited your website, then ask about website use first, so that those who haven’t visited it can skip any irrelevant questions.
BE APPROPRIATE
Make sure that the language you use is appropriate to your audience. Remember that your respondents won’t be as close to your business as you are, so avoid using overly technical terms or jargon. Instead think about how respondents might refer to your products or services, rather than how you describe them.
Think about the age of your respondents, and make sure you use language that will be understood by everyone.
If you need to provide your questionnaire in other languages, then please, please, please use a translation agency! Online translation tools are amazing if you’re on holiday and want to order a coffee, but if you use these tools for your questionnaire then you’re running the risk that your results will be a jumbled mess.
LIMIT FORCED ANSWERS
A great benefit of using an online survey tool is that you can force respondents to answer each question before they move on to the next. This avoids a situation where you end up with lots of partially completed questionnaires and therefore you’ll have full data sets for everyone who has answered.
BUT please don’t force people to answer questions about income, gender, relationship status – you WILL annoy them. If you insist on making questions mandatory then you should include an “I prefer not to answer” option.
Also make sure that any mandatory question has a full set of answer options available. For instance, don’t ask “Which days of the week do you work?” listing the options as Monday, Tuesday, Wednesday, Thursday, Friday – you need to include Saturday and Sunday too. You also need an option for “I don’t work”.
ARE ALL YOUR QUESTIONS NECESSARY?
Be careful about how many personal questions you include. Do you really need to know their gender? Or their age? Is that relevant to your research? Could you ask respondents to give you their customer number instead? This could link back to other information you already hold about them, saving the need to fill in their age, income, gender, marital status. Think about what you are asking and why you need that information.
And again, be clear about what you are going to do with this data. You must ensure you are GDPR-compliant.
CHECK FOR INCOMPLETE ANSWER OPTIONS
Time and time again I’ve abandoned a survey part-way through because a question I’m being asked doesn’t have an option that applies to me. Look at this example –
Q Which is your favourite boy band?
Westlife
Boyzone
JLS
The Wanted
This question forces people to choose from a list of four options.
Where do I tick to show my preference for Take That?
Where do I tick to show that I don’t have a favourite boy band, because I love thrash metal?
The results won’t be a true representation of opinions on favourite boy bands. This is because the question actually being asked is, “From this list of four, which is your favourite boy band?”
At the very least you need to allow an “Other” option. The ideal version would show an alphabetical list of 20 or so boy bands plus an “Other – please specify” option and an “I don’t like boy bands” option.
BEWARE BIASED SCALES
I see this ALL.THE.TIME. The example below is about gathering satisfaction with customer service, but four of the five options indicate a positive response. The results will therefore be skewed towards a positive response. So, yes, you are likely to be able to say that most people are satisfied, but because of the poor question structure those results will be meaningless.
Q How satisfied are you with the level of customer service we provide?
Extremely satisfied
Very satisfied
Quite satisfied
Satisfied
Not satisfied
The example below has a balance scale, with an equal number of options available for positive and negative replies, along with a middle ground option for those who don’t have a strong opinion either way.
Q How satisfied are you with the level of customer service we provide?
Very satisfied
Quite satisfied
Neither satisfied nor unsatisfied
Quite unsatisfied
Very unsatisfied
AVOID VAGUE QUESTIONS
Be direct – if you want to know about customer service then ask about customer service. Don’t be vague.
Q Do you like our company?
A Well, I quite like the name, and I love Jane because she always answers the phone really quickly. But last time I ordered it took four weeks to arrive, and then my order was wrong.
Which bits of that answer are relevant for you?
KEEP QUESTIONS NEUTRAL
Don’t assume anything. Don’t ask a question which, by its nature, sways respondents to answer in a particular way. You will alienate people if you assume that they have certain behavioural traits. And, again, your data won’t be meaningful because you’ve led respondents to answer in a particular way. These examples show you what I mean –
Example 1
Q Please tell us what you love most about our fabulous new and exciting website?
You are leading people with the words “fabulous”, “new” and “exciting”. Much better to ask..
Q Please tell us what you like best about our website?
Example 2
Q Where do you enjoy drinking tea?
You are assuming people enjoy drinking tea. Much better to ask..
Q Do you drink tea? Yes/ No
Q If yes, where do you enjoy drinking tea?
TEST, TEST AND TEST AGAIN
I’ve lost count of the number of times when I’ve started a questionnaire but then had to abort because it was taking too long, or I got stuck in a routing loop and couldn’t get to the end.
I’ve lost count of the number of times when I’ve started a questionnaire but then had to abort because it was taking too long, or I got stuck in a routing loop and couldn’t get to the end.
Once you’ve finished the questionnaire, test it.
Ask a colleague to complete the questionnaire, timing how long it takes them to do it.
Then ask another colleague to complete it.
Did it take each of them the same amount of time?
I’ve lost count of the number of times when I’ve started a questionnaire but then had to abort because it was taking too long, or I got stuck in a routing loop and couldn’t get to the end.
I hope that has been useful. Hopefully it’s given you ideas and tips to help with your own questionnaire design process.
Get in touch to find out how market research can help you make better business decisions.
