In my last blog, I reviewed some data from the World Health Organization on the amount spent on healthcare in the U.S., Canada, Sweden and Japan. The results of the data were counterintuitive – the higher the spending on healthcare, the lower the life expectancy.
As good analysts, we should scrutinize any data to ensure that it is not only appropriately used (and that any limitations to its use are noted and considered when making recommendations), but also that it adequately addresses the core business question. Of course, for this to occur, we must have a well-defined, properly framed business question.
I set out a challenge for our readers to consider what limitations existed in the data and how this might constrain what conclusions we could make about the linkage between healthcare spending and life expectancy. In this blog, I’ll review five factors that could impact how we interpret the data.
1. What is the underlying question we are trying to answer?
The first issue that creates confusion (and, potentially, misinterpretation of the data) relates to the underlying question at hand – is the goal in the healthcare debate to increase life expectancy overall, to minimize the expense of healthcare, to get coverage to more people, or something else? Of course, like the business questions we face daily, there is not a single answer; I would suggest, however, that pundits are trying to answer too many questions with a single data set.
2. Is the outcome metric the right metric?
The second issue relates to the core outcome metric being employed – mean life expectancy. While longer life expectancy is good, it does not tell us the whole story; to know that, we must understand the underlying distribution of the data. Consider the following data distributions:
Graph 1 Graph 2
Graph 3 Graph 4
The shapes of these distributions clearly suggest differences at play in the data; they do, however, have one thing in common – their mean value (78 in this case).
Let’s explore what these might mean from the perspective of life expectancy – how would we interpret each of these?
Graph 1: The variation around the mean is fairly constrained; this suggests that most people will live within a fairly tight range around 78 years.
Graph 2: The data skew toward a lower life expectancy (roughly 58 years). What this means is while the average life expectancy is 78 years, many people (about one-third) will not live past 63 years; on the other hand, there are a great volume of people that live much longer lives, which effectively increases the average.
Graph 3: This is a normally-distributed set of data with wide variation around the mean; this tells us that while life expectancy is 78 years, there is a high likelihood that one could die much earlier or much later.
Graph 4: This is a bimodal distribution – it tells us that we have higher mortality at earlier years (approximately 58 years) and later years (approximately 98 years). While the mean of this distribution is 78 years, most people, in raw numbers do not perish at 78 – they either die much earlier or much later.
For us to effectively compare means, we have to make an assumption that the underlying distributions are normally distributed – i.e., graphs 1 and 3 above. Moreover, we also must know the standard deviation of the distribution to make effective comparisons.
Incidentally, if we suspect the impact of getting healthcare to more people is a healthier population, I would suggest that simply looking at the distributions would be sufficient – in this case, we would hypothesize that countries with uniform, available healthcare might look like Graph 1, while countries with less available healthcare options might have distributions like those in graphs 2-4. In this case, the variation is the more important metric, not the mean.
3. Exchange rate and geographic pricing difference
The data shown have been converted to U.S. dollars for the sake of comparability. This could say more about the valuation of the U.S. dollar than the state of healthcare; moreover, it is not uncommon for pricing practices among healthcare organizations to differ geographically based on market pressures, governmental controls, and other factors.
4. Selection bias
The World Health Organization has data on many countries throughout the world; the example I have shown focuses on four countries. The extent to which these are not representative could have a dramatic impact on the confidence we place not only in the analysis, but the strength of our recommendations moving forward. Worse, if the examples have been hand-picked to reinforce our preferred argument, then we cannot make a scientific conclusion from the data provided.
5. Overly simplistic models
Even if we assume that the core business question is clearly articulated, the model shown is too simple to capture the nuances of such a complex issue. For example, the data do not allow us to control for any variables that may have an adverse impact on per capita spend levels. Several have been articulated in the debate – for example, the extent to which limits are placed on lawsuits, which impacts the cost of business as it relates to necessary tests and physician compensation (for malpractice insurance premiums) as well as the nature of the competitive environment (which places downward pressure on prices).
Simple models, while attractive, usually have limited utility; in addition, the simpler the model, the more defined and specific the underlying business question needs to be.
These are just five things to consider (I’m sure there are more – please let me know if you have other ideas). In my next blog (the last in this series), we will explore what we can take away from this example as we focus on the topic of customer loyalty management analysis.
Sr. Vice President, Consulting Services & Resource Management