Skip to content Skip to search

Insight Center Resources

Industry knowledge to help you grow your business

Survey Question Design Part II – The Good, Bad and Ugly

As a continuation of the previous article, “Survey Question Design: The Good, Bad and Ugly,” this article addresses the pros and cons of utilizing open-ended questions in a survey. An open-ended question, also called an “infinite response question,” is an unstructured question for which predefined options or response choices are not offered. Rather, the respondent answers the question in his or her own words. This type of question is exploratory in nature and designed to collect narrative data. Open-ended questions usually begin with the words, “what,” “why,” or “how.“ An example of a typical open-ended question is, “What factors would lead you to recommending your bank to a family member or friend?”

The good aspects of including open-ended questions in a survey are:

  • They capture the respondents’ responses in their own words
  • They obtain more in-depth information since there are no limits to the possibilities of responses
  • They can provide unanticipated findings
  • They tend to provide greater insight

The bad aspects are:

  • Open-ended questions tend to take up more room on a survey
  • They most often require more time to answer
  • The responses may not be relevant
  • The data they provide is complex and highly time-consuming to analyze

The type of narrative response data collected from open-ended questions is called “qualitative data,” since the information cannot be measured. Rather, each response must be independently read to gather the information being provided by the respondent. Depending on the sample size, this endeavor may take a significant amount of time to complete. However, the investment is worthwhile, since the information provided is likely to offer unique and valuable information directly from the minds of your customers.

Another way to analyze the data that is typically used is to transform the collected narrative qualitative data into quantitative form that can be measured and compared. In order to do this, the analyst will separate the narrative responses into defined groups.

Possible responses to the open-ended question example provided above might include:

  • Reputation of the bank
  • Great rates
  • Excellent customer service
  • Convenient bank locations
  • Online banking capabilities
  • Mobile banking capabilities
  • Fraud protection
  • Does not make recommendations to family members or friends

The analyst would read through all of the responses and separate the response into the appropriate group. Each response would count as one point and each group would eventually produce a count based on the number of responses separated into each group. Once this is accomplished, the responses from the open-ended question can be measured, compared and analyzed further.

However, this approach threatens to lose the richness of the information the open-ended question was designed to capture. Therefore, the data collected from open-ended questions is most effective when analyzed from both a qualitative and a quantitative perspective.

The next article will investigate another approach by which to capture in-depth, insightful, unanticipated and informative data using only open-ended questions — the Focus Group.