Walker Information
Helping you put the customer at the heart of every decision.

Does Your Panel Data Have Bad Respondents?

Online panels are becoming a popular sample source in marketing research because they readily provide a pool of respondents who are willing to participate in surveys. However, recent research suggests that when these panels are used, it is likely that some fraudulent data will be collected because of the presence of ‘professional survey takers’ who take surveys merely to get incentives. These panelists are likely to answer differently from the engaged respondent, so their responses can potentially impact the findings of the research. 

Before beginning a study based on panel data, the panel vendor should be interviewed to determine what controls they have in place to prevent “bad” panel responses. Even if there are good controls in place, it is probably also a good idea to examine the data collected via a panel and consider some respondents for deletion from the dataset. Some things to look for when trying to identify these fraudulent respondents are:

·         Speeding: These respondents take the survey at a much faster pace than normal, suggesting that perhaps they are just trying to get through the survey and are not supplying thoughtful responses to the questions asked. 

o    Recommendation: Respondents identified as speeders should immediately be removed from the dataset. 

·         Illogical Responses: These respondents type nonsense or gibberish into the open-ended questions of the survey, or provide an answer that is clearly not a proper response to the question. 

o    Recommendation: Answers such as these show that the respondent was not giving a lot of thought to the survey. Respondents who answered gibberish should immediately be deleted from the dataset. Others should be considered for removal, with the final decision resting on whether or not they were deemed fraudulent in other areas.

·         Trap Failure: These respondents fail to correctly answer a “trick” question that has been placed in the survey (ex: “Please answer ‘Very Satisfied’ to this question). 

o    Recommendation:  Consider removing these respondents from the dataset if they are also identified as fraudulent in other areas.

·         Straight-lining: These respondents give the same response to each question throughout the survey. This might be an indication of thoughtlessness during the survey, but it may also truly be how the respondent feels. 

o    Recommendation: This is a more subjective indication of a bogus respondent. Therefore, it is recommended to leave these respondents in the dataset unless their lack of variation is causing problems in the analysis.  

In conclusion, online panels provide many benefits, but if you’re using them, be sure you’re doing some checks on the back-end to ensure you have only the highest quality respondents!

Jessica Gregory,

Marketing Sciences

Share your thoughts

Your email address will not be published. Required fields are marked *