For the last few weeks, we have been focusing on whether customer perceptions can be trended when methodological changes are implemented. The answer that we have suggested is yes. This week we are going to consider whether that answer holds true when the mode of the survey has changed.
There are many different modes of data collection – in-person, phone, web, IVR or multi-mode interviews. Because of the decreased cost and time spent in data collection, more and more researchers are moving their surveys to the web for at least part, if not all, of their data collection. However, many of these changes are taking place without any thought given to how or if this mode change will impact scores, even though research indicates that the mode can affect the way a respondent arrives at an answer.
Research indicates that survey mode can lead to different responses if the mode affects the way a respondent arrives at an answer. There are two main factors which produce different responses between modes: normative or sociological factors and cognitive or psychological factors. The normative factor refers to how cultural norms are invoked differently depending on the mode (ex. acquiescence bias, social desirability, etc.). The cognitive factor refers to how aural and visual stimuli can influence the processing of information and how that can vary by mode (ex. interviewer characteristics, visual mid-points, etc).
So does this mean that I can never change the current mode I’m using to collect data? No, changing the mode can sometimes be very beneficial; however, as with most design changes, it is important to consider the advantages and disadvantages to the choice that you make.
Some points to consider:
· Your mode needs to fit the audience that you are targeting.
· Consider a multi-mode approach, at least for a transition period, in order to test and/or quantify the impact of the mode on responses.
- Make sure you are able to identify which mode respondents used in your data.
· Understand the research on various modes and their impact on response rate, response quality, and potential biases/error.
· Consider how your data is being used.
So while customer perceptions can still be trended if there is a mode change, it is important to weigh the pros/cons to the switch, ensure the new mode fits your audience, look at the research for what shifts to expect in your data, and if possible, plan for testing to see how much of the change is simply due to the mode of data collection.Becca Lewis
Stat Analyst, Marketing Sciences