Customer Feedback Analysis

1. Involve respected, trustworthy people from every key function and department in the company. These people will find it much easier to sell their colleagues on the importance of your customer intelligence. And you will also get some key feedback to make the information you gather more applicable to their departments.
2. Show people proof that customer intelligence can help them achieve their own goals.
3. Treat them like partners. You can learn as much from them as they can from you.
4. Communicate clearly and effectively. There are a lot of good ideas on our blog related to this topic.
5. Allow field tests. If someone doubts the usefulness or impact of the information, devise a simple test or pilot program to help prove it. This is not always possible, but it can often work if you think creatively about it.

I'm sorry, but I can't hear a phrase ending in "the rest of us" without thinking of Festivus...oh, how Seinfeld has changed our lives! But that's a whole different post.

Today I am writing about Customer Experience Competency Centers. That's right, the CECC's you've all been hearing so much about. What? You haven't heard of this? Well, that's probably because the term has not (yet!) taken the business world by storm.

I was invited to present a session at Clarabridge's user conference, C3, this week. As a side note, if you are serious about mining unstructured text for customer insights, you have to talk to Clarabridge. The topic for my session was "Creating a Customer Experience Competency Center within an Organization." I have to admit, I was a bit stumped at first. I had heard the term "competency center" a couple times but had no idea what it meant, and I definitely had never heard of one related to customer experience. To my relief, it was not a commonly understood term in the CEM world, which would have been embarrassing for me not to know.

I'm hesitant to enter another buzzword into an already crowded canon of business buzzwords and fads. However, the brain trust behind the C3 program was definitely on to something with this topic. As I researched and thought about the concept, I realized the customer experience management field can use this idea.

So, here's my working definition of a Customer Experience Competency Center:

A cross-functional unit within an organization that drives and supports its customer experience management efforts through the collection, integration, analysis, communication, and delivery of customer information.

Still a little vague? Well, it is a working definition. In essence, it's the entity responsible for pushing your organization towards ever-increasing customer-centricity by ensuring that every decision is informed by the customer perspective.

Granted, this is not a totally new concept, and many great, customer-centric companies already have cross-functional teams that do some of this. However, I think there is a place for something a little bigger, grander, and more impactful than we've traditionally seen. If customer-centricity is truly a key competitive differentiator, then our organizations and their stakeholders deserve a formalized structure dedicated to ensuring that all our customer information is turned into the insights and intelligence necessary to create and sustain that competitive advantage.

In future postings I'll talk more about the details of this concept, but I'm interested in any feedback on the idea, good or bad. In the meantime, if you want to learn more about competency centers in general, follow these links to see how IBM and Oracle, SAP, and SAS are implementing them to ensure customers get the full value from their products and solutions.

Online panels are becoming a popular sample source in marketing research because they readily provide a pool of respondents who are willing to participate in surveys. However, recent research suggests that when these panels are used, it is likely that some fraudulent data will be collected because of the presence of ‘professional survey takers’ who take surveys merely to get incentives. These panelists are likely to answer differently from the engaged respondent, so their responses can potentially impact the findings of the research. 

Before beginning a study based on panel data, the panel vendor should be interviewed to determine what controls they have in place to prevent “bad” panel responses. Even if there are good controls in place, it is probably also a good idea to examine the data collected via a panel and consider some respondents for deletion from the dataset. Some things to look for when trying to identify these fraudulent respondents are:

·         Speeding: These respondents take the survey at a much faster pace than normal, suggesting that perhaps they are just trying to get through the survey and are not supplying thoughtful responses to the questions asked. 

o    Recommendation: Respondents identified as speeders should immediately be removed from the dataset. 

·         Illogical Responses: These respondents type nonsense or gibberish into the open-ended questions of the survey, or provide an answer that is clearly not a proper response to the question. 

o    Recommendation: Answers such as these show that the respondent was not giving a lot of thought to the survey. Respondents who answered gibberish should immediately be deleted from the dataset. Others should be considered for removal, with the final decision resting on whether or not they were deemed fraudulent in other areas.

·         Trap Failure: These respondents fail to correctly answer a “trick” question that has been placed in the survey (ex: “Please answer ‘Very Satisfied’ to this question). 

o    Recommendation:  Consider removing these respondents from the dataset if they are also identified as fraudulent in other areas.

·         Straight-lining: These respondents give the same response to each question throughout the survey. This might be an indication of thoughtlessness during the survey, but it may also truly be how the respondent feels. 

o    Recommendation: This is a more subjective indication of a bogus respondent. Therefore, it is recommended to leave these respondents in the dataset unless their lack of variation is causing problems in the analysis.  

In conclusion, online panels provide many benefits, but if you’re using them, be sure you’re doing some checks on the back-end to ensure you have only the highest quality respondents!

Jessica Gregory,

Marketing Sciences

Does your company rely on membership-driven business?

Findings from recent Walker programs find common factors which distinguish Truly Loyal Members from those that are Not Truly Loyal. Specifically, in contrast to those respondents who are Not Loyal, Truly Loyal respondents…

- have positive perceptions of the balance between their membership fees and what they get.

- believe that the organization cares about their success.

-perceive that membership has a positive impact on their business performance.

- have positive interactions on key experiences, like- the membership renewal process,  being given opportunities for networking and connecting with peers, and being made aware of events and offerings.

So what does this mean for your membership-driven company?

-          Communicate to members all of the options that are available with their membership.
 

-          Provide avenues for members to connect with each other. One study found that the odds of renewing membership increase by 171% for each increase in rating of how the membership facilitates connection between its members. 
 

-          Ensure that you highlight how the membership can have a positive impact on your member’s business. The more members perceive a positive impact of membership on their business, the more likely they are to renew their membership. 

By focusing on the concepts mentioned above, you can work to increase member loyalty. What steps are you taking to meet the needs of your membership-driven customer base?

Amy Heleine and Becca Lewis

Marketing Sciences

There are certain Dilbert strips that seem perfectly appropriate to the way people and companies can behave, even though they are never quite as blatant as the characters in the comic.

At this time of year, when we are more focused than usual in our personal lives on giving and making others happy, I think we can also take a moment to reflect on the focus of our professional lives. How often do we take something that is supposed to be about others - customers, employees, colleagues - and turn it into something that serves our happiness?

In the realm of voice of the customer programs, this usually means an over-emphasis on the "score" we get from customers and the method used to get it instead of focusing on the customer problems that are driving the score and how to fix them. This is very likely to occur when you tie compensation goals to customer survey scores (see my previous post on this topic).

So, as the new year begins, take some time to look at the metrics you use and the goals you pursue in your day-to-day job. Are they focused on the right objective? The only way to ensure long-term success in your job or your life is to ensure your actions towards those who matter the most to you will make them happier and more successful.

Happy Holidays!

Troy Powell, Ph.D.
Vice President
Walker Information

As Christmas approaches many people begin to reminisce on what has happened over the past year as well as past holiday seasons. I am no different and recently have gotten into multiple conversations about what Christmas Day was like growing up and how that is different for the routines for this year. As kids, one of the most important things about Christmas was the gifts – we just couldn’t wait to see what new toy was under the tree. But as you grow up, the focus shifts away from the toys – even though there is still an element of excitement for what new gadget might be under the tree – and onto other things, such as time with family, a few days off of work, or just a little relaxation.

After dinner with a friend where we talked about our Christmas routines as kids and then for next Friday, I went home and was doing some analysis on customer loyalty. So that got me thinking about how in any relationship that lasts over time, there are going to be shifts in what is most important, even business relationships. So what does that mean for companies that collect feedback from customers?

A lot of companies solicit feedback in order to understand how to make their customers more loyal. After they have looked at the results they come up with a way to implement the drivers of loyalty in the customer experience, but they don’t factor in the fact that there are shifts in what is important to a customer. In fact, a lot of companies treat customers the same regardless of how long they have been working with the company.

Not surprisingly, research supports the idea that the drivers of customer loyalty differ when you segment the data by customer tenure. In the beginning of a relationship, customers tend to focus on satisfaction and price; however, as time goes on, trust as well as the relationship aspects (such as account teams, etc) become more important and have a greater impact on customer loyalty. This implies that depending on how new a customer is to your organization there may be different things that the account managers/contacts should be focusing on.

While this does not mean that you should never look at customer data in total, it does suggest that organizations should have a way to identify customer tenure and should use some time analyzing if there are differences between various customer tenures. The results may shed light on ways to improve relationships for different tenure classifications or help explain why changes impact customers differently.

Becca Lewis

Statistical Analyst

So far, we have explored how customer loyalty data can be connected statistically to financial performance at both an internal/micro level as well as an external/macro perspective. To summarize the key findings:

From an internal/micro perspective (i.e., at the customer/account level), we can use the linkage of loyalty and financial performance to:

·         Identify where revenue is at risk (and, conversely, more secure);

·         Evaluate to what extent customers act on their intentions;

·         Articulate the value of improving customer loyalty;

·         Build tailored customer-level strategies to build on existing loyalty or address the impediments to customer loyalty;

These tools are used in conjunction with strategic account planning to facilitate the growth and profit objectives of the firm.

At the external/macro perspective (i.e., at the market performance level), the literature to date suggests that customer satisfaction/loyalty metrics…

·         can be used as a leading indicator of future stock price trends,

·         can be used as a leading indicator of stock returns risk, and

·         have utility in financial markets and should be disclosed in the ongoing filings public companies are required to file with the SEC.

In other words, we have tangible evidence that intangibles such as customer loyalty can add to (or detract from) the value of a firm, and therefore have a place on the balance sheet.[1]

It is interesting to note that each approach, while different in its focus (external vs. internal), complements each other and creates a virtuous cycle of value creation. In other words, it makes sense that a firm that focuses on tailored customer-level strategies would have a more customer base, which means a more stable revenue base. A more stable revenue base means that there is likely less volatility in the firm's earnings, which attracts investors. When a firm attracts more investors (and the current investors are more interested in holding the stock vs. selling it), the laws of supply and demand tell us that the stock price will appreciate. Strong stock returns attract attention (generally positive), which serves to create awareness and demand for the firm’s products and services. And so the cycle continues.

Collectively, these findings reinforce the strategic value of being a customer-focused organization, and the implications are broad-reaching; consider the following scenarios:

1)      A mutual fund manager is interested in investing in a given sector and has narrowed his focus to three firms – each has a strong balance sheet, a solid growth record, and reputable management. With all the basic criteria so evenly aligned, the “tie-breaker” is the customer loyalty metrics each firm publishes.
 

2)      A firm is interested in acquiring one of its competitors; in the due diligence phase, the acquiring firm decides to conduct an assessment of the customers of the target firm. The results suggest that the customer base is tenuous at best. The acquiring firm may determine to back out of the deal altogether (or, at a minimum, substantially reduce its offer price).
 

3)      A management team decides that it wants to “walk the talk” by making certain its managers and leaders are personally invested in the strategy of being a customer-focused organization. To do this, performance targets on customer loyalty levels are set; to further reinforce the level of “skin in the game” that managers and team leaders have, the incentive is stock-based.
 

4)      An account manager wants to ensure that she is allocating her time focusing on the customers with the greatest value to her organization. Rather than focus solely on total spend/revenue, she employs the Value Mapping discipline to evaluate which clients hold the greatest strategic value to the firm. Using the output from the Value Mapping process, she is able to construct tailored, specific action plans for each customer.

These are just a few ways in which customer loyalty can be leveraged to increase the value of the firm. What they all have in common is that they are focused on strategic business questions – in other words, tracking loyalty for the sake of scorekeeping holds absolutely no value to the firm. To create strategic value, the data have to be leveraged to address substantive, strategic business questions.

Mark Ratekin
Sr. Vice President, Consulting Services & Resource Management

 

In my last entry, I discussed ways that we conduct analysis that links customer loyalty to firm financial performance at a customer/account level. In this entry, I will discuss the linkage between customer loyalty and market performance.

A growing base of research has quantified the linkage between stock price, stock returns and customer loyalty (see, for example, Aksoy, Cooil, Groening, Keiningham & Yalcin (2008) and Fornell, Johnson, Mithas, & Krishnan (2006)). Much of the literature has focused on the connection between customer loyalty and stock price – that is, is there a link between customer sentiment (as measured by traditional customer satisfaction/loyalty metrics) and how much value a share of the company’s stock carries? As it turns out, there is a connection, and it follows intuitive reasoning – the more satisfied a company’s customer base, the more favorable the stock price.

But stock price is only one factor to consider – the other factor to consider is volatility. Volatility refers to the ups and downs a stock price experiences, and it is a measure of risk – the more risk in a stock, the greater the volatility. When we refer to volatility in a stock, we are generally referring to the composite of two broad types of risk – first, there is systematic risk – this is the risk that is associated with the market at large, best characterized by the John F. Kennedy quote that “a rising tide lifts all boats.” The other type of risk is idiosyncratic risk – this is the risk associated with actions of the firm. For example, management decisions regarding products, pricing, customer segments, etc. have an impact on idiosyncratic risk.[1]

To date, the literature on the connection between stock risk and customer loyalty has been pretty sparse. Moreover, the available literature has focused exclusively on the topic of systematic risk – in other words, they have focused on how stock prices move relative to the total market, not the company-based idiosyncratic risk.

In the November, 2009 Journal of Marketing, Kapil Tuli and Sundar Bharadwaj add significantly to the literature in their paper “Customer Satisfaction and Stock Returns Risk” by focusing on both sources of risk – systematic as well as idiosyncratic. They find that customer satisfaction scores “insulate a firm’s stock returns from market movements (overall and downside systematic risk) and lower the volatility of its stock returns (overall and downside idiosyncratic risk).”[2] That is, the greater the satisfaction/loyalty of a customer base, the less volatility that is exhibited by the stock.

So, the bottom line is this - companies with strong customer loyalty enjoy not only better stock returns, but they are also less susceptible to volatility in their stock price. We have independently corroborated these findings with our Walker Index - a composite stock index  of Walker clients that has outperformed the broader market indices by a factor of 5 to 6 times since its inception. The Walker Index also has less volatility than the broader market indices, as measured by beta as well as upside and downside capture ratios.

In these first two entries, we have discussed how loyalty metrics and financial performance are linked from an internal/micro perspective (i.e., at the customer/account level) as well as at an external/macro perspective (i.e., at the market performance level). What are the implications of these findings? I’ll address that in the third (and final) entry of this series.

In the meantime, what do you think? How can we leverage this information to more effectively run our businesses?

Mark Ratekin
Sr. Vice President, Consulting Services & Resource Management

Congratulations! You have developed a classification model that correctly identifies promising pharmaceutical compounds 10% of the time! But isn’t that just a 10% success rate? Why spend the time and money developing that model when you could get a 50% success rate simply by flipping a coin? The usefulness of a classification model is sometimes underestimated based on a false assumption that if there are two possible outcomes, then there is a 50% chance that either will occur. In most environments in which two possible outcomes exist, there is NOT an equal probability that each will occur. 

Consider the pharmaceutical example presented recently by Graettinger. A company has determined that 99.9% of potential compounds do not yield promising drugs. They would like to determine a statistical rule that will increase chances of identifying potentially promising compounds.  In his article, Graettinger presents two models:

·         Model 1 is a classification model (resulting from Graettinger’s data mining techniques) that correctly identifies promising compounds 10% of the time.  

·         Model 2 is a simple rule that labels ALL potential compounds as “unpromising.”

It would appear that Model 1 has a 10% success rate, while Model 2 has a 99.9% success rate. But what is the definition of success? It is NOT simply the correct identification of a compound as “promising” or “unpromising.” Rather, it is the correct identification of “promising” compounds alone. Thus Model 1 boasts a 10% success rate compared to 0% for Model 2.

We can all agree that 10% success is better than 0% success, but is it really something to be excited about? Consider the business application for the pharmaceutical company. Without the classification model, the company would have to test each potential compound. The knowledge that 99.9% of the potential compounds do not yield promising drugs indicates that 1 of every 1000 tests will identify a promising compound. If the classification model is applied and only those labeled as “promising” are tested, the 10% model success rate indicates that 1 of every 10 tests will yield a promising compound. The model allows the company to achieve the same result with only 1% of the testing resources that would be required without the model.

Cortney Lantry

Director, Marketing Sciences

Resource: "Data Mining Misconceptions", Graettinger, 2008-2009, www.discoverycorpsinc.com

Walker clients have heard us talk about the various techniques my colleagues and I have developed to demonstrate the tangible value of customer loyalty. When working with clients, we tend to look at the linkage between customer loyalty and financial performance relative to either individual customer relationships, by product segments, or some other variable of choice. These techniques allow us to discover a number of valuable insights that poise the client to take meaningful action on the customer results:

1)      We can see how much revenue and/or profitability risk exists within the customer base;

2)      We can determine the reality of the linkage between what customers say they will do and their subsequent behavior (and the time lag between the two);

3)      We can calculate the return on customer loyalty – in other words, when we take action on the customer feedback and create change that generates gains in customer loyalty, how much gain do we see in revenue and/or profitability?

4)      Finally, we can use Value Mapping to use Loyalty and the Three P’s – Profitability, Potential and Partnership – as a segmentation method to build differential strategies for taking action.

To accomplish these analyses, we must have not only the customer perspective (gathered, generally, through a quantitative research study), but we also must have financial data at a granular (account, product, geography, etc.) level. This, of course, requires that the client firm tracks customer data at this level. Moreover, it requires a collaborative effort between the Walker team and the client’s finance professionals. For the analysis to be credible, the data have to be capable of “talking to each other” – and this can often be a challenge.

Marrying these data enables us to tell a compelling reason for change – in dollars and cents. Beyond this, though, the process serves an extremely strategic purpose by taking “soft” metrics like customer loyalty and creating a valuation of the customer asset – a concept that has been espoused by Baruch Lev and others who have called for fundamental changes in our accounting methods to adequately reflect the shift of firm wealth metrics from hard assets (factories, inventories, etc.) to more intangible assets (customers, intellectual property, etc.).

What if we don’t have financial data at a granular level? Fortunately, we can still conduct linkage analysis; however, the utility from an actionability perspective within the company is highly constrained. From an external perspective, however, this information can be quite valuable. I will cover that in my next entry.

Mark Ratekin
Sr. Vice President, Consulting Services & Resource Management

Since the behaviorists, psychological theory has taken a more humanistic approach to understanding human motivation, moving beyond the tactic of presenting/removing a stimulus in order to achieve a desired response. One of the early psychologists to probe deeper into human motivation was Maslow. The research field has used Maslow’s Hierarchy of Needs as a theoretical basis for how to motivate respondents to complete a survey. Here is some background on Maslow’s theory-

Summary of Maslow’s Theory:

There are six levels to Maslow’s hierarchy. The hierarchy proceeds from the lowest, base needs, to the highest, self-fulfilling needs.

Level 1: Physiological Needs- Food, water, oxygen, etc.

Level 2: Safety and Security- Structure, order and predictability.

Level 3: Love and Belonging- Having family, friends, and group identification.

Level 4: Esteem- Recognition, esteem, status, feeling adequate and competent.

Level 5: Self-Actualization- The need for personal growth and fulfillment.

Level 6: Knowledge and Understanding and Aesthetic Needs.

Translating Maslow’s Theory to Collecting Customer Feedback:

Within the arena of collecting customer feedback, researchers have tended to focus on the need for esteem as a way to encourage survey participation. More specifically, Maslow’s theory has been translated in the following ways-

-          Interviewers ask for help in completing an important survey.

-          Survey invitees are told that they have been chosen to participate in the survey.

-          Survey invitees are told that their opinion counts.

Shortcomings of Applying Maslow’s Theory to Response Rate:

The above words have been chosen to try and create feelings of esteem in potential responders, attempting to make the solicited customer feel special. While these words may have motivated people to participate in surveys in years past, given the prevalence of surveys, and the knowledge that most people have about this prevalence, these words alone are not likely to solve the response rate issue. This is not to say that potential responders should not be made to feel special, rather this should just be one aspect of the overall approach to increasing response rate.  A sustained increase in response rate is likely to involve consideration of the entire customer feedback process, not just one element. Theoretical findings from fields like psychology can provide guidance in addressing specific aspects of the feedback process that can be improved upon in order to achieve improved response rates.

Next week, we will apply a more recent theory on human motivation to response rate- Self Determination Theory.

Amy Heleine 

Director, Marketing Sciences 

Reference

Cape, P., “Understanding Respondent Motivation,” Survey Sampling International White Paper, 1-17.

In previous blogs, we have reviewed findings-to-date, regarding what specific elements of the survey process have been shown to be influential on response rate.  If we take a step back from the specifics, at the heart of the response rate issue, is respondent motivation. How do we get respondents to want to take a survey? In a recent article by Pete Cape, “Understanding Respondent Motivation,” he provides a historical review of how psychological theory on human motivation can help us tap-into motivating survey respondents.  This week, I will start with his review of the Behaviorists, relating how Behaviorist theory can provide insight to the response rate issue.

Behaviorist Theory in a Nutshell

·         The behaviorists provided the concepts of reinforcement, punishment and extinction.

o   Reinforcement concerns strengthening a desired behavior as the result of either experiencing a positive condition (Positive Reinforcement) or eliminating a negative condition (Negative Reinforcement).

o   Punishment concerns weakening behavior as a result of experiencing an aversive condition (Positive Punishment), or removing a positive condition (Negative Punishment).  

o   Extinction occurs when the reinforced behavior is no longer effective.

·         A primary criticism of behaviorist theory is that it provides a mechanistic view of human behavior, which is limited, as the motivation behind human behavior is complex.

How does Behaviorist Theory Aid Understanding of Response Rate?

Sometimes you can learn the most from the criticisms of a theory. Respondents are unlikely to be sustainably motivated to complete a survey by making any single improvement, or removing any single negative component, at one point-in-time. For example, providing a deadline by which to complete the survey has been shown to significantly impact response rate; however, a deadline alone is not likely to sustain response rate, rather there are many other influential factors. Instead of focusing on any one factor, consider the entire survey process-

·         Remember that the process starts before the respondent chooses/declines to take the survey- creating a survey that is easy for the respondent to take, pre-communication, personalizing communication, scheduling reminders, etc.

·         And, the process continues after the respondent completes the survey- communicating and acting upon findings, showing and proving that the feedback matters.

Next week, we will start to examine psychological theories that assume a more humanized-approach, exploring how understanding human needs can help us tackle declining response rates.

Amy Heleine

Director, Marketing Sciences

Reference

Cape, P., “Understanding Respondent Motivation,” Survey Sampling International White Paper, 1-17.

Science has advanced mainly through failed hypotheses. New theories, ideas, or inventions are based on the invalidation of a previous hypothesis. This type of advancement is also true in other arenas, including business. However, most people focus on confirming hypotheses and spend little, if any, time looking for contrary evidence. I think there are a number of reasons for this:

• We have an emotional investment in the hypothesis and want to be right. This is especially true when we created the hypothesis or when our job is seemingly dependent on the hypothesis being true (which is hardly ever true, even when it seems to be).
• Our boss or executive leader is a big proponent of the hypothesis, and we don't want to contradict them.
• We created a bad hypothesis that really can't be invalidated.
• We don't even consider that our assumption or belief should be tested and skip creating a true hypothesis. We treat our assumption as a statement of fact and then go about putting together all the evidence that supports it without even considering contradictory evidence.

1. Create a culture of testing and inquiry. It is essential to moving your company forward.
2. Develop good hypotheses based on your assumptions. 
3. Know what it would require to disprove the hypothesis.
4. Try to find good, solid evidence against your hypothesis. The supporting evidence is much easier to find. 
5. Be willing to challenge the common wisdom of your company or leadership when the hypothesis is shown to be wrong. This can be uncomfortable, but if you do it respectfully and with the support of strong evidence, the message should be accepted. It also helps to have a few other colleagues in your corner! 

The October issue of Quirk's published the last of a three-part series on how satisfaction measurement has gone awry and ways to fix it. I thoroughly enjoyed this series and would encourage anyone who administers or uses data from a survey-based customer feedback program to read it.

Of all the problems with traditional customer listening programs, there is only one thing the authors claim will thoroughly corrupt a program - tying feedback scores to corporate incentive plans or compensation. This may seem counter-intuitive. Including customer scores in an incentive program helps focus the organization on customers, right? Ideally, yes. Practically, not so much. It focuses the organization on the score, but not usually on the customer.

The authors provide three key reasons why tying incentives to customer feedback scores is dangerous:

1. The goal of the survey shifts from pleasing the customer to pleasing the organization. If you currently use survey scores in your incentive plan, consider how much time you spend setting, explaining and defending the goals. And this time increases exponentially if you miss a target, which is exactly when you should be talking to customers to see why scores are down instead of defending the numbers internally.
2. Those responsible for the survey become the "police" instead of the "partners." The time you spend with internal stakeholders is more often focused on defending the metrics and methodology than on helping the organization internalize and act upon the customer feedback.
3. You actually give executives an incentive to question the research and results. Making it part of the compensation plan brings high-level attention to the customer survey, but actually encourages executives to question and nit-pick the details instead of focusing on the big picture.

• We will understand where we have financial security (and risk) in our customer base.
• We will understand who our most valuable customers are.
• We will understand what potential revenue gains exist among our current customers.

• Changing economic conditions;
• Regulatory changes in your market;
• Competitive forces – new entrants, exiting competitors, supplier influences, resource constraints, etc.
• Changing consumer tastes; We recommend taking a fresh look at your program every couple of years to see how things are changing in your environment (and we need to revamp in the program to keep it fresh, relevant, and impactful).

• How do we define an account – one person or many people? One customer organization or entities within that organization?
• Should our sample mirror our revenue composition? If so, do we weight results, control the sample, or both?
• Do we have access to (and can identify) all relevant customer contacts? Should we differentiate by decision role, for example?

As the debate around healthcare reform in the U.S. rages on, we are inundated with data from either side of the political spectrum that strive to reinforce the major points of the plans under proposal. The media is even doing comparative analysis; on CNN, Lou Dobbs has been running series of segments comparing the current U.S. healthcare system to other systems around the world. These segments (one of which can be viewed here), are full of stats and metrics, including patient satisfaction levels.

One of the common practices in these comparison is to compare spending levels (in absolute spend or percentage of country GDP) with average life expectancy. The theory is that greater spend levels should yield longer life expectancy.  Consider the following data:

 

These data do not bear out that theory – the country in these data with the longest life expectancy, Japan, spends the least on healthcare. What’s going on?

Before we jump to the conclusion that healthcare is too expensive relative to the outcome, I would suggest that we take a critical look at the data to see what, beyond public policy, could lead us to the data we see here.

What do you think? If you have thoughts, please leave a comment. One caveat, though – it is not my goal to open up a political discussion. Please consider only the data and possible sources of bias or error that could impact our interpretation.

In my next blog, I’ll look at five factors that may impact the interpretation.

Mark Ratekin
Sr. Vice President, Consulting Services and Resource Management

There are more ways than ever before to efficiently analyze information. All of this readily available information can be overwhelming, but using it wisely can provide robust, actionable guidance. The following paragraphs attempt to provide some guidance on navigating all of this available information.

·         First and foremost, know the question(s) you are trying to answer. Sample design, survey design and how the data is analyzed, all should tie-back to the question.
 

·         Understand your sample plan- know to whom your findings do and do not generalize.
 

·         Do your homework before designing the survey. This will help to ensure that important measures are not excluded.

o   There are a myriad of available statistics to help assess the goodness of the model and variables measured. The findings from these techniques can be used to guide survey design moving forward.
 

·         Use all pieces of available information to answer the question- survey results- both closed and open-ended measures, behavioral data, voice of the customer, competitive data and information, historical data, news and industry information, etc.

o   Using all available information helps to provide a robust answer to the question, but keep this in-mind- some findings may appear to be inconsistent. These apparent inconsistencies can be another valuable source of information (assuming no errors have been made).
 

·         Understand the level of data being used- is it being analyzed at an individual customer level, or is the data being used at an aggregate level (e.g., by account, by call center, by region, etc.). 

o   Post-data collection is also the time to re-evaluate sample representativeness- are there customer non-response issues, how do the sample distributions align with the population distributions, etc.
 

·         Use the most appropriate statistics to effectively and efficiently answer questions.

o   Rather than just comparing performance scores, statistical testing can be used to narrow down where statistically meaningful differences exist between scores.

o   Multivariate analytic techniques can further help to direct focus on the measures that most effectively distinguish or help in understanding customer attitudes and behaviors.

o   Do not assume that if you do not see patterns in the data, that none exist. Sometimes the failure to see patterns in the data is a result of not appropriately segmenting the data. As a follow-up to customer segmentation, profiling techniques can be applied for further understanding and description of customer segments.
 

·         Answering the question often leads to new questions being asked. A single program cannot/should not be designed to answer all questions.

o   Tracking the questions and answers over time, and analyzing how these things have evolved, can again, lead to further insights. Review this process periodically to adapt to the ever-changing business world.