Over the past six months several Walker colleagues and I have been working on an experiment. The experiment focuses on using an alternative analytical approach to achieving insights from customer feedback. And, in order to apply a different analytical approach, we had to redesign the survey instrument.Three Goals
1 - Test something new on behalf of Walker clients and other customer strategists.
2 - Provide our clients with something they can use to improve their customer feedback program.
3 - Generate richer insights that will help Walker bring increased value to all of our client relationships.
Walker recently completed its annual relationship survey, which served as the testing ground for this new concept. And, while the jury is still out on whether or not our experiment accomplished its goals, I thought it might be helpful to document the concept and the design implications.
The Previous Design
In the past, our relationship assessment included a series of questions, where respondents were asked to evaluate our performance on a scale of Excellent-to-Poor. With the feedback we collected, we used multiple regression models to identify the areas that have significant impact on our customer's relationship with Walker. This information was used to prioritize initiatives at both the customer level and the corporate level.
The Experiment Design Concept
The survey design for this experiment includes a rather different approach. Instead of asking customers to evaluate Walker's performance, we asked them to evaluate particular aspects of the solution using the same Excellent-to-Poor scale (e.g., Communications with customers). We followed up with a series of questions where the respondents were asked to pick the characteristics that describe that area (e.g., frequency of communications, method they use to communicate, who sends the communication, etc.).
To be clear, these follow-up questions were not an evaluation of the area (as they have been in the past). Instead they were pick lists that included characteristics of that specific solution component.
Looking Forward
With this new approach, we plan to use an analytical technique that will profile various aspects of a customer feedback program. This approach will provide our clients and Walker a view of what makes up an excellent customer feedback solution.
Although we are still in the process of analyzing the results, the team has already learned quite a bit. For example, while this design doesn't include more questions, the types of questions require more thought and time to complete. Stay tuned for an upcoming blog about lessons learned from a long survey.
While the experiment is not perfect, I remain excited about the insights it will bring to our clients. I look forward to sharing more with you too.
Photo credit: The Lab Depot, Inc.
Note: This post was originally published in Customer Connection on 12/10/2009.


Comments for An experiment in survey design