In the previous article, I looked at the importance of setting a clear and focused objective for research. That is always the starting point. What does the organisation need to know, and what may need to be done as a result of the findings?
Once that objective is clear, attention naturally turns to the questions themselves. What needs to be asked in order to meet that objective? That is a good question, but before writing the questions, it is worth stepping back and considering something equally important: how those questions should be asked.
This is where the quality of research can begin to strengthen or weaken. A question may appear relevant on the surface, but if it is structured poorly, it can confuse the respondent, limit the response, introduce bias, or make the findings less reliable than they first appear. In other words, it is not just what you ask that matters. It is how you ask it.
Good question design is not about making research more complicated. It is about making sure the evidence gathered is relevant, robust and useful in supporting a decision. Let us look at some of the key distinctions that help shape effective questions.
Why question design matters in market research
Question design matters because research is only as useful as the evidence it produces.
If the objective is clear but the questions are weak, the findings may be difficult to trust or limited in their value. If the questions are carefully designed, the evidence is far more likely to support a sound interpretation and a practical next step.
This is why I see research as more than a process of collecting information. It is a structured way of helping organisations understand where they are now, what they need to know, and what action may be appropriate as a result. For that to happen, the method and the questions both need to be relevant to the business, the people being asked, and the decision that needs to be made.
A good questionnaire does not happen by accident. It requires thought, care and a clear connection to purpose.
Open and closed questions in market research
One of the first decisions in question design is whether a question should be open or closed.
A closed question provides the respondent with a defined set of response options. These may allow for a single answer or multiple answers, depending on the purpose of the question. Closed questions are particularly useful where the likely range of responses is already understood and where the aim is to produce clear, structured data that can be analysed consistently.
For example, if an organisation wants to know which customer support services have been used in the last month, it may be more effective to provide a list of service options than to ask respondents to recall and write each one from scratch. This makes the question easier to answer and the findings easier to interpret. It can also be helpful to include options such as Other, Don’t know, or Not applicable where these are relevant to the respondents’ circumstances.
That said, closed questions still need care. The response options need to be complete, relevant and mutually exclusive. If important options are missing, or categories overlap, the usefulness of the data is reduced.
An open question works differently. Rather than asking the respondent to choose from a list, it gives them space to answer in their own words. This can be especially valuable when the organisation needs explanation, detail or insight that has not already been anticipated. Open questions can help uncover the reasons behind a response, the language respondents naturally use, or issues the question designer may not have considered.
For that reason, open questions can be particularly effective when depth matters more than structure. They are often less useful where the purpose is simple measurement, but much more useful where understanding is needed.
Neither open nor closed questions are inherently better. The important point is that they do different jobs. The format should be chosen because it suits the objective, not because it is familiar or convenient.
Rating and ranking questions: what is the difference?
Another distinction that is easy to overlook is the difference between rating and ranking.
A rating question asks respondents to assess each item independently, usually on a scale. This is useful when the organisation wants to understand how people feel about each area in its own right. For example, a respondent might be asked to rate politeness, response time, listening and knowledge on a scale from very bad to very good. This helps show the strength of feeling about each individual area.
A ranking question asks something else. It requires respondents to compare items and place them in order. This can be useful where the organisation needs to understand priorities. For example, it may want to know which aspect of customer service matters most in comparison with others.
The distinction matters because these question types do not produce the same evidence. Rating can show how well each item performs in the eyes of the respondent, but it may result in several items receiving similar scores. Ranking forces respondents to make choices between items, but it does not tell us how strongly they feel about those differences.
That means the choice between rating and ranking should be driven by what the organisation actually needs to know. If the need is to assess performance or satisfaction across several areas, rating may be more appropriate. If the need is to understand relative importance or order of preference, ranking may be more useful.
As with all question design, the method needs to support the purpose.
Fact based and thought based questions
It is also important to consider whether the research is asking for fact or thought.
A fact based question asks for information about something that has happened, is happening, or is definitely planned. These questions help establish actual behaviour, circumstance or activity. They are valuable when the organisation needs clarity on the reality of a situation. Questions about who lives in a household, what someone ate for breakfast, where they work, or how they usually travel are all factual in nature.
A thought based question explores something different. It may ask about opinions, beliefs, expectations, preferences or intentions. These questions are valuable when the organisation needs to understand attitudes, perceptions or what may influence future decisions. Questions about an ideal breakfast, preferred travel methods, or where someone would most like to work are not factual in the same sense. They reflect views rather than concrete activity.
Both have value, but they are not interchangeable. Facts tell us what is. Thoughts help us understand what may be driving behaviour, how something is perceived, or what may happen next. If the distinction is not clear, there is a risk that opinion is interpreted as fact, or that important context is missed altogether.
Again, this comes back to the original objective. What does the organisation need to know, and what kind of answer is required to support that understanding?
Common questionnaire design mistakes to avoid
Some damaging weaknesses in research can come from basic question design errors that are easily avoidable.
One common issue is missing options. If age bands leave out a group altogether, some respondents will not know how to answer accurately. Another is overlapping options, where a respondent could reasonably fit into more than one category. In both cases, the problem begins before analysis has even started.
Coding also needs care. If one option is a subset of another, such as listing sage separately from green, the categories are no longer operating at the same level. That weakens the clarity of the response structure.
Scales can also become imbalanced. If the positive end of a scale is more detailed or more generous than the negative end, respondents may be nudged towards a favourable answer. The same applies to wording that is leading or biased. A question such as “How good was the useful aftercare service you received?” is already suggesting the answer before the respondent has given it.
Another common issue is asking two questions in one. For example, asking whether a product is cost effective and reliable assumes both ideas can be answered together. They may not be judged in the same way by the respondent, which makes the answer harder to interpret.
Questions can also become confusing through assumptions, ambiguity or double negatives. If a question assumes awareness, experience or preference that the respondent may not have, it risks producing misleading data. If wording such as regularly is left undefined, different respondents may interpret it differently. If the sentence structure is confusing, the quality of the answer suffers.
These are not minor technical details. They go directly to the quality of the evidence. A report may still look polished at the end, but if the questions were poorly designed, the findings are built on weaker foundations.
How better research questions lead to better decisions
If the objective has been set clearly, the next step is to make sure the questions are capable of supporting that objective.
That means thinking carefully about whether questions should be open or closed, whether they should ask respondents to rate or rank, whether they are seeking facts or thoughts, and whether the wording is clear, balanced and appropriate.
Each of these choices influences the response received. Each affects the quality of the evidence. And each plays a part in whether the research can genuinely support a decision.
In the first article, I focused on the why behind research by looking at objective setting. Here, the focus has been the how in question design. The next step is to look at the what in the questionnaire tool itself, and how that tool can be shaped to support effective engagement and robust findings.
If you are considering a piece of research, the starting point is always the same. What is the objective, what does the organisation need to know, and how can the research be designed in a way that produces evidence that is genuinely useful?
