The great debate about the prediction ability of Net Promoter Score (NPS®) continues apace.  With any change or new idea, there are those that embrace the idea and there are those that reject the idea.

While I consider myself in the embrace camp, I believe that I am so from an empirical, “we need to get things done”, stand point.  I am happy for NPS to be replaced when someone comes up with an empirically better approach.

In fact, I’ll be the first to use it.  However, those in the camp looking for NPS deficiencies are not helping themselves with this sort of research: Measurement Deficiencies in the Net Promoter Score.

Apart from some just plain bad logic this research piece goes off on a tangent [1] of trying to understand why NPS works.

In my view, it is a mistake to equate not understanding the reasons that a process works, with the fact that it does.  We don’t know how gravity works (well at least very few of us do). And we do not doubt that it does work and it can be relied upon to give us a good bump on head if we ignore it.

So let’s explore some of the issues in this paper.

Faults of logic

At one point the authors note that a similar measure does not perform as well as Net Promoter

“Morgan and Rego (2006) constructed a measure that was similar to the NPS [1] and found that this was not as effective as other measures, constructed a measure that was similar to the NPS1 and found that this was not as effective as other measures…”

“[1] Their measure was based on past recommendation given rather than the likelihood that customers would give a Recommendation now”

The actual measure was based on “Have you discussed your experiences” subtract “Have you formally or informally complained…”  This question is hardly even similar to the Net Promoter question.  You can read the whole referenced paper here: http://www.telesight.com/morgan-rego.pdf

Another example is the statement that following some analysis (which I do not suggest is flawed) that

“This evidence indicates that NPS is poorly related to the production of PWOM [positive word of mouth] and NWOM [negative word of mouth].”

This may be true but it is irrelevant.  If the point of the paper is to suggest that NPS can be directly correlated to the generation of PWOM and NWOM then it is a fair comparison to make.  However, the underlying goodness of the Net Promoter Score measurement approach is that it is correlated to company revenue.

(See our Introduction to Net Promoter Score for several examples from different organizations on this effect).

The problem here is between cause and effect: NPS is correlated to revenue.  Whether NPS is correlated to PWOM or NWOM is not relevant to the argument: “is NPS a good loyalty indicator”.

Actually this logic flaw so early in the paper makes much of the rest of the document interesting but not related to it’s title: Measurement Deficiencies in the Net Promoter Score.  Perhaps it should be retitled?

Faults of Market Research

“We ask the receiver to report the impact and frequency of WOM received on any brand in the category.”

Asking the receiver to make this type of report is simply not reliable.  What’s more, respondents to these types of surveys are unlikely to be able to accurately determine the objective impact of the WOM.

While not a very scientific approach to proving this point, a quick Google search on “advertising does not work” provides 293 million hits; yet clearly it does.

Why I think NPS works

There are other issues with this paper but I will not labor them further.

I don’t know why NPS correlates well to future company revenue and I’m not sure anyone else does either.  My own belief is that it has nothing to do with generating actual word of mouth; positive or negative.  It is all about how the respondent feels about the brand and whether they might, hypothetically, distribute some of their personal credibility to the brand under consideration.  A bit like Google weighs the credibility of inbound links to a website when it decides where a web page will appear in search rankings.

Indeed I may be right, or wrong, but it doesn’t really matter.  All that does matter is that NPS correlates to company revenue at quite a high level, i.e. it works.

[1] This is a tangent for those of us trying to provide our clients with practical advice on how to improve their business revenue and profit.