In a home kitchen in the early 1830’s a, then unknown, inventor saw potential in the milky sap from the Indian rubber tree. Surely, he thought, there must be a way to make the sticky substance more useful.

Through many years of experiments he mixed the raw rubber with a variety of different substances and tested the outcome. Did this mixture improve the properties of the raw material? No; onto the next trial. None were successful until he hit upon the idea of also heating one of the mixtures.

With that, Charles Goodyear had invented Vulcanisation and the rest is, as they say, history.

But think about it, how successful do you think Charles would have been had he run his experiments and simply put the resulting substances in his bottom draw, never checking to see what he had created? Yet this is what happens with so much of the customer feedback collected everyday by organisations around the world.

If you want to make change in your organisation you need to do more than collect data, you must analyse the customer feedback to inform change and drive new approaches.

Here are five elements of analysis that you should consider.

Read every comment

So this may sound simple but you’d be surprised at how many organisations don’t do this simple step. It’s like running an experiment and never looking at the results.

If you need more reasons to read those comment then consider these statistics that one of our clients uncovered.

1. 1-in-1,000 comments is a discontinuous innovation idea: Normally we think of customer feedback as an incremental innovation data source through which we can gradually improve the business. This organisation identified that in addition to that, a small number of pieces of feedback provide an incredibly important seed for a major breakthrough in the business. Do you want to miss those ideas?
2. 3-4% are directly actionable: These are the comments that you can take direct action on to improve that customer’s perception of your business. If you do nothing with this feedback you are leaving disgruntled customers to become ex-customers.

Make Root Cause Analysis a priority

You must understand what is driving customer loyalty if you want to improve it. So you need to analyse your customer feedback to determine the management levers you have at your disposal. If you don’t know what drives change, you can’t drive change.

In this area, we find organisations that have some sort of quality management system are much more effective than organisations that don’t.

Typically it doesn’t matter what type of quality system it is, as they all have their basis in the same ideas; 6-Sigma, Lean, TQM, etc.

If you have never heard of the 5 Whys, Fishbone analysis, KJ analysis then you need to obtain some training for your organisation in quality system basics. Spend some time to get this skill into your business. This is a necessary but not sufficient condition for success in customer feedback.

Know the value of a customer

If you do not know the value of a customer and the value of moving them along the customer feedback scale, be that Net Promoter, Customer Effort Score, or others, then you have no way to prioritise your change management efforts.

For example nib health funds know the impact of changing a customer feedback score by one point. They know for instance that “A one point increase in “Would Recommend” score results in a decreased of risk of termination by 7.8%”

This information is the basis for your business cases: if you want to invest x thousand dollars in a change to your business because it will drive up customer loyalty, the first question management will ask is how much is that worth to the business.

So you need to have an answer for both (I)nvestment and (R)eturn in the ROI calculation. Knowing the value of a customer will give you that (R).

The Trend is Your Friend

One of the things we need to be very careful about when we do any time based analysis of customer feedback, including Net Promoter Score, is determining what is a real change and what is probably just a statistical aberration. This is not as simple as it might seem.

One approach that can help is to look at the longer term trend. If the score is repeatedly going up, or down, that will increase the likelihood that the change is real.

There are some rules of thumb you can use based on the statistics of control charts. For example: if the score rises or decreases seven times in a row then you have a pretty good indication that the change is real and not statistical chance. Note that this is more than most people would guess. It takes longer to be certain than you might expect.

If you can’t wait for a row of seven rising or falling you can do the statistics on your data but that is often complex and for NPS even more challenging.

You can download this free tool where we have done all of the difficult maths for NPS. Just enter the number of Promoters, Neutrals and Detractors for each sample and it will tell you if the change is probably real. Download the  Net Promoter® Comparison Tester.

Don’t Compare what is not Comparable

The urge to compare data from different parts of the business or geographies or surveys is very hard to resist but resist you should.

The chart below shows the difference between Australian and European Net Promoter scores for a couple of different industries. If you were to compare your NPS in Australia against another business unit in Europe you can see you’d be at a disadvantage; and for no good reason.

So it is clear that you need to be very careful when comparing data. You may think that the “would recommend” question doesn’t change, so surely you can just compare the scores; actually it’s not that easy.

There are many factors that may affect the score. For instance, I can tell you that:

1. NPS scores from telephone surveys are lower than email surveys
2. NPS scores from surveys taken a week after a transaction are lower than scores taken the next day.
3. NPS scores are lower where the “would recommend” question is last in survey and higher where it is first in the survey.

This doesn’t mean the NPS is not useful. It does mean that relative change is much more important than absolute numbers.

It’s much better, although still not perfect, to give everyone the goal of improving NPS by, say, five points during the next six months, rather than giving everybody the goal of +20.

To be useful you need to have a consistent routine for your data collection and then keep with that process. Only then can you compare changes between different time periods.