‘If it matters, measure it’

There are many variations in that old saying, but it holds true. How therefore do we end up with hundreds of measures that seem not to matter?

Fear.

Fear of missing a measure that does matter, so we create metrics for every-bloody-thing to ensure that we do not miss one.

That is crap management.

Let’s think about measuring stuff that does matter, and then measuring it at the point where the decisions and actions that influence the outcome are made. This is tying cause and effect together at the point where they intersect, not looking at a range of data and wondering what happened to cause that!

How do we define what matters?

To me it is simple, if it moves the performance indicator, it matters. Clearly, the converse is also true.

Ask yourself, does the number of Facebook likes you have impact your profitability? If it does not, and I would contend it never does, so why use it as a KPI? It is simply a readily available metric that has no relevance to performance. It is what those ‘likers’ do with your information that counts, much harder to define and measure, but if you understand that, and the cause/effect chains, it just might move the performance needle and become a KPI worth measuring.

In short, behaviour determines the outcomes, so set out to measure the behaviours you need to deliver the performance you are looking for, not the other way around.

How do we measure what matters?

A measure without a target is not of much value, as we cannot see if any movement is relevant to performance. A measure should articulate the performance against which we need to move the performance needle in a strategically significant manner. This setting of targets is challenging if we do it properly. Applying a 3% increase in last year’s performance is not doing it properly, it is just extrapolating, accepting that history will repeat itself.

To measure properly, we need to consider the factors at work that will influence performance, seeking the causes, and measuring them, not just glancing at the metrics and having no idea of whether or not any movement is significant. Holy cow Batman, we just got another 5,000 likes on the Facebook page. Wow! But so what?

A further caution. ‘Sandbagging’ so called KPI’s is common in situations where there is little strategic linkage, and analysis of flow on impact. Two examples. Sales people when incentivised only by a target will be tempted to keep the targets as low as possible in order to achieve their bonuses.  Who has not seen that? Purchasing people incentivised only by purchase price will not care too much about the performance of the cheaper version they opt for, which in the factory, may corrupt the efficiency numbers, and have a far greater financial impact than the saving of a few bob on the initial purchase price.

Do not focus on averages.

Too many times I see piles of measures, taken at a high level, so that they reflect the average of a whole lot of other factors. If I have one foot in an ice bucket, and the other in the fire, on average the temperature of my feet is about right.

Nonsense.

Measure the outliers, the things that are unseen in averages in order to better manage them. For a KPI to be meaningful, it has to influence the outcome. Removing one foot from the fire will influence the average, but if I have not realised that the effect is caused by the removal of the foot in fire, I will at some point put my foot back in the fire.

I do not remember much from the statistics I did 45 years ago at university, but one of the ideas I do remember is that of standard deviation.  I recall little of the mathematical gobbledy Gook and probably do not need to any longer, as the formula is in Excel, just fill in the boxes, but I do remember what it means. (Forgive the pun).

In the normal distribution curve we are all familiar with, 68% of outcomes are within one standard deviation of the mean. These can reasonably be classified as an ‘expected’ result, given that forecasting is not an exact science, it is just a best informed guess, and the level of ‘informed’ varies hugely, depending on who has their mouth open at any one time.  95% of outcomes fall in the range of 2 standard deviations, and 99.7% fall in the range of three standard deviations. This is commonly called the ‘Rule of 68’

A focus on the unexpected, the outliers, will give you far greater leverage on the outcomes than a focus on the averages, or expected. It might lead to taking one foot out of the fire, and understanding that this is what has caused the increase in the comfort level.

Defining the outliers, like most things in life, can be made easier by imagery. A core piece of process improvement is defining the levels of variability, and then seeking to understand the causes of that variability. A visual way of communicating this is a performance graph that includes what you define as the limits of the variability you would consider to be ‘normal’. Commonly this is called a ‘statistical control chart’, and includes the upper and lower limits of what can be expected. Anything outside these limits needs to be investigated.

Anything inside the control limits is by definition, ‘normal’ and therefore not necessary to spend a lot of time considering. What however is worth great consideration is determining what the control limits are, where the normal becomes abnormal, which is where action must be taken. Over time, in an improvement process, the control limits will be progressively tightened as the outliers are progressively understood, so they become part of the normal, or eliminated.

 Cascade the KPI responsibility

Having any more than 6 or 7 KPI’s to manage creates a situation where we skate over the top, not able to devote the time and energy to improving the things that matter, that move the performance needle. The things that really matter will be different at each level, and in each part of the enterprise.  Therefore, constructing KPI’s relevant to each role should be a core part of the process of managing the resources of the enterprise, and especially in encouraging the behaviour we want  that will collectively, move the performance needle. Within each functional area, there will be a cascade of KPI’s that together add up to the 6 or 7 KPI’s to which the functional manager is held accountable. This is not to forget that the processes we are measuring are very often cross functional, and ignoring those cause and effect chains leads to sub optimal performance as in the purchasing/operations example noted earlier. This can be addressed by ensuring that the purchasing manager has a KPI that involves operational efficiency in the measurement.

Use the narrative in reporting.

A dashboard of a few easily understood performance indicators is terrific, it tells you what has happened, but lacks two vital pieces of information: Why it is happening in this way, and what should be done about it.

Narrative is the best way to communicate these vital factors, the core of great management, indeed, leadership. Knowing clearly what is happening is step 1, steps 2 and 3 are what make the difference between the companies that struggle to survive and those that prosper and grow. Illustrating these narratives with graphical KPI movements over time is a powerful way to illustrate the impact of performance at any level.

Credit Wikipedia: Rule of 68-95-99.7.

Header credit: Hugh McLeod Gaping void