The actual survey format has the potential to impact response rate. However, it is important to understand that seeing the survey is one of the last steps that individuals take to complete the survey. Focusing attention at the earlier steps may result in a bigger impact on response rate because by the time the respondent gets to the survey, they have usually decided whether or not they are going to complete it. It is important to take steps to motivate the respondent to actually get to the opening page of the survey.
Once a respondent has accessed the survey, it is important to make sure they can complete the survey easily. Studies that have asked participants for reasons that they did not respond to the survey have found that there are many features of the survey that can cause frustration for participants, and lead to terminating the survey before it is completed. Here we will focus on research surrounding three main areas: presentation type, survey length, and number of pages.
Presentation of the survey (elements such as graphics, formatting, use of colors, etc.)
o Simple surveys (e.g., plain, textual, black/white, minimum graphics) have a higher response rate than complex surveys (e.g., fancy, visual cues, graphics, colors). These findings are attributed to the increased downloading times associated with complex surveys
o Consider the respondents’ web capabilities and be cautious of increasing download times
o Non-respondents frequently cited survey presentation factors as reason for frustration and termination of a web survey
Survey length (number of questions asked in the survey)
o Some studies have found that shorter versions of surveys are associated with higher response rates
o However, other studies have had mixed findings concerning the impact of survey length on response rate, some showing no impact on response rate or data quality
o Other factors, such as topic salience, etc, most likely could help to explain these findings.
Number of pages
o Studies have found no significant difference in response rate between one and multiple pages
o Increased pages may increase download times
o There are some advantages to having multiple pages in a web survey:
o Allows the researcher to capture partial information because the respondent’s data is saved after each page
o If branching and skips are used in the survey, using multiple pages allows the researcher to have control over the questions the respondent sees
While survey format is generally thought of more in an aesthetic manner, it is important to understand that the choices can influence a person’s willingness to response as well as their actual responses. It is important to make the survey as easy as possible for the respondent in order to decrease early termination of the survey. However, more and more research has found that changing these elements of the survey can impact question/scale meaning for respondents and score distributions. Response rate, item non-response, and dropouts can definitely be impacted by survey design, but focusing on the earlier steps, such as invite, may have a greater impact on your response rate.
Stat Analyst, Marketing Sciences