Students as customers

The Net Promoter Score as a measure of satisfaction and loyalty in higher education

Paper presented in track 2 at the

EAIR 37th Annual Forum in Krems, Austria

30 August till 2 September 2015

Name of Author(s)
Raphael Schmatz
Gerald Wolf
Mareike Landmann

Contact Details
Raphael Schmatz
University of Cologne
Albertus-Magnus-Platz
50923 Cologne
Germany
E-mail: raphael.schmatz@uni-koeln.de

Key words

Higher education research design, Institutional performance measures, Quality, Research

design and methodologies, Student experience

1 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
Abstract

Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in
higher education

In an ever more competitive sector of higher education, universities are often regarded as providers
of educational services for their students, who, in turn, assume a role as their customers. We first
discuss the applicability of this provider-customer-concept to the higher education sector. We then
present our experience with the use of the Net Promoter Score (NPS) as a measure for students
satisfaction with and loyalty to their university, in particular the University of Cologne, Germany. We
finally contrast our results with data from other higher education institutions and develop ideas on
how NPS could be increased.

2 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
Presentation

Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in
higher education

Introduction: A changing sector of higher education

The European higher education system is passing through a far-reaching process of transformation
initiated by the Bologna reform (cf. Sursock & Smidt 2010). One of the most evident signs of change
is the assimilation of the European degree structure, which is said to make it easier for students to
choose and switch between higher education institutions. This greater freedom of choice has
implications for both students and universities. In an ever more competitive sector of higher
education, the former resemble customers on some kind of educational market whereas the latter
assume the role of service providers competing with other tertiary institutions for these customers.

The ongoing harmonisation of the European Higher Education Area (EHEA) has hence forced
universities into a new way of strategic thinking that focuses more on competition not only on a
national but also on an international level (cf. Schwaiger 2003, p. 33; Katsaraova 2015, p. 23f.). This
paradigm shift (Hrsch & Rudinger 2009, p. 76) makes it necessary to learn more about the provider-
customer-relation between universities and their students. Two important questions arise: Why is
customer satisfaction so important, even in higher education, and how can it be measured?

Since customer orientation has always played a crucial role for commercial enterprises, it seems
reasonable to draw on their experience and try to apply it to the higher education context. This is
why we seek to answer the questions posed by employing the concept of the so called Net Promoter
Score (NPS) (cf. Reichheld 2003) a measure of customer satisfaction and loyalty that has become
quite popular in management within the last years (cf. Keiningham et al. 2008, p. 53f.) but has
seldom been used in higher education so far. Before we put this idea to the test, however, some
general remarks on customer satisfaction and the NPS are necessary.

Background: The Net Promoter Score

Most companies primary goal is growth. In order to achieve it, they need to build high-quality
relationships (Reichheld 2006, p. 44) with their customers. Yet, how can a company tell the quality?

In an attempt to find an answer, Fred Reichheld (2003) first introduced the Net Promoter Score in
2003. Using a pool of about 20 different questions that were commonly asked in customer
satisfaction surveys, he hoped to discover the Ultimate Question (Reichheld 2006) for gauging the
quality of customer relations. To that end, he linked peoples responses to the satisfaction survey
questions with data about their individual purchase and referral behaviours after buying a certain
product or dealing with a certain company. It turned out that answers to the question How likely is it
that you would recommend Company X to a friend or colleague? were most effective in predicting
customers actual behaviours for most industries included in the study (Reichheld 2003, p. 51).
Interestingly, the wording avoided terms like satisfaction or quality altogether. It rather seemed
to address something that goes even further than customer satisfaction: loyalty (for some general
thoughts on customer loyalty cf. Reichheld 2001).

Loyalty in general can be defined as the willingness [] to make an investment or personal sacrifice
in order to strengthen a relationship (Reichheld 2003, p. 48). With regard to customers, loyalty
manifests in taking the risk of recommending something beyond ones control, i.e. a company or its
products. Only very satisfied customers are likely to take that risk. A referral could thus be
interpreted as the strongest sign of customer loyalty (Reichheld 2003, p. 48).

3 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
This interpretation received support by a more detailed analysis of customers buying, referral and
answer behaviours according to which three groups of customers can be distinguished (cf. figure 1).
Customers in the first group show the highest repurchase and referral rates and are consequently
called promoters. They typically express their satisfaction by a rating of nine or ten on the zero-to-
ten scale used to answer the ultimate question. A second group consists of customers who are
somewhat satisfied but not enough to spread the word. The so-called passives answer the question
with a rating of seven or eight. Finally, there are the most unsatisfied customers as indicated by a
rating below seven. They might even engage in negative word-of-mouth and are hence called
detractors. The positive effects of the promoters actions are presumably diminished by the
detractors derogatory behaviours. Therefore, the NPS is calculated by subtracting the percentage of
detractors from the percentage of promoters while the percentage of passives is not used for the
calculation (cf. Hrsch & Rudinger 2009: 75f.). The NPS thus ranges between +100 and -100, with a
positive value representing a desirable goal as further analyses suggest: In line with his initial
considerations, Reichheld was able to reveal strong positive correlations between companies Net
Promoter Scores and their respective growth rates across several industries (cf. Reichheld 2006, p.
41ff.). These findings seemed to support his confident notion, that he might actually have discovered
the ultimate question for connecting customer satisfaction to growth.

Figure 1: Groups of customers according to their likeliness of recommendation (cf. Reichheld 2006, p.
30)

Labelling something as ultimate will quite likely arouse two things: attention and criticism. And this
is exactly what happened. On the one hand, many well-known companies such as General Motors, T-
Mobile or American Express started to use the NPS for monitoring their costumers satisfaction (cf.
Keiningham et al. 2007, p. 39, 2008, p. 53f.). On the other hand, Reichhelds metric yielded a
considerable number of critical responses that need to be considered carefully when working with
the NPS. Besides authors who criticize the NPS concept as a whole and doubt its efficiency (cf. Sharp
2008), two main points of criticism can be found in the literature.

Firstly, the way in which Reichheld promotes his ideas can be regarded as somewhat misleading.
Titles like The Ultimate Question (Reichheld 2006) suggest that the NPS is a universally applicable
metric. Yet, other authors find no support for [Reichhelds] claim that Net Promoter is the single
most reliable indicator of a companys ability to grow or that it is superior to other metrics
(Keiningham et al. 2007, p. 45; cf. also Pingitore et al. 2007, p. 12f.). Secondly, there is the problem
with the replicability of Reichhelds empirical findings. Keiningham et al. (2008) for instance, failed in
replicating the strong link between a companys NPS and its growth rate that Reichheld reported.

Despite the criticism, the Net Promoter Scores three major assets remain. It is simple, easy to collect
and easy to understand (cf. Reichheld 2006, pp. 18, 29) three qualities that instruments used in
empirical higher education research are sometimes lacking. Therefore, using the NPS for evaluation
and quality assurance in a university seems compelling. But is that even possible, given the fact that
the metric was obviously developed from a corporate management perspective? We will now discuss
the issue of transferability.

4 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
Transferability: Satisfaction and loyalty in higher education
When adapting the idea of the Net Promoter Score to the higher education context, two difficulties
need to be taken into account. First: Unlike commercial companies, universities do not primarily try
to achieve growth and profit. Second: Students are not customers in a classical sense. Repurchase
decisions are irrelevant for the greater part of them because, typically, their buying decisions (i.e.
their decisions to begin studies at a certain university) result from a one-time process. Still, we argue
that a transfer of the NPS concept is possible. Our line of reasoning can be spelled out as follows.

A universitys most important competitive factor and thus main goal is a positive reputation (cf.
Theus 1993, p. 277). Following a definition of corporate reputation that Gotsi and Wilson (2001, p.
29) suggested, it can be understood as a stakeholders overall evaluation that originates in direct
experiences [], any other form of communication and symbolism that provides information []
and/or a comparison with the actions of other leading rivals.

What can be derived from this definition are ideas for a course of action an academic institution
might want to take to increase its reputation. To begin with, it should invest in students direct
experiences, e.g. by providing good teaching (cf. Langer et al. 2001, p. 5) and student services
aiming at facilitating studies (Lenecke 2005, p. 65). As repurchase rates are less relevant in higher
education (although admittedly, a Bachelor graduates decision to continue studying in a masters
course at the same university can be regarded as a repurchase decision), this is first and foremost an
investment in students satisfaction and long term loyalty. On the one hand, this will presumably
have a positive influence on what students communicate about their alma mater and what might
consequently serve as an indirect experience for third parties, most importantly for prospective
students. On the other hand, only very satisfied students will stay in touch after graduation, ideally
expressing their loyalty as sponsors and supporters (cf. Hrsch & Rudinger 2009: 80). In addition to
that, a university should take care of communication itself, e.g. by offering comprehensive
information, and make efforts to position itself as a unique provider of higher education among its
competitors.

While the last aspect mentioned is rather an issue of strategic direction that is beyond the scope of
this paper, it becomes apparent that in many ways, a good relation with students is a vital condition
for universities to achieve their goals just as a good relation with customers is crucial for companies
to achieve theirs. The Net Promoter Score thus appears to be an appropriate measure of satisfaction
and loyalty in higher education to us.

Approach: The NPS in the First-Year Student Survey at the University of Cologne
Knowing a universitys NPS alone will not do much good if there is no other data that makes it
possible to tell the differences between promoters, passives and detractors. This is why we decided
to implement the would recommend question in our multi-theme First-Year Student Survey. It is
tailored to the situation of newly arrived students and designed to learn more about this specific
group. Among others, the online survey contains questions about students fields of study,
motivation, social, educational and vocational backgrounds and evaluation of certain student
services.

We included two questions that specifically deal with the NPS. Based on the assumption that there
might be considerable differences in the willingness to recommend the university as a whole and
ones course of studies, we settled for starting with a two-tier question that distinguishes between
these two aggregates. The question was: How likely is it that you would recommend the University
of Cologne and your course of studies to a friend or an acquaintance? For technical reasons, our
eleven-point scales provided for the answers ranged from one (very unlikely) to eleven (very
likely) as opposed to the original range from zero to ten.

5 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
This first question was followed by an open-ended second question aiming at a deeper understanding
for the likeliness of a referral. Respondents were asked to briefly describe the reasons for their
ratings given. Figure 2 illustrates how the questions appeared in the online survey.

Figure 2: The NPS questions in the First-Year Student Survey

The First-Year Student Survey is conducted every semester. The latest round started in June 2015
when an e-mail was sent to 1,981 students inviting them to take part in the entirely revised survey.
For the following analyses, we used a preliminary dataset as of mid-July containing a total number of
309 respondents (response rate 15.6%) of whom 208 completed the entire survey.

Analyses & Results

When trying to assess the usefulness of an instrument in a certain context for the first time, such as
the Net Promoters Score in our case, it seems reasonable to start by comparing it to an established
metric that can serve as a reference point. What comes to mind is a question that has been used to
capture student satisfaction at the University of Cologne for several years now and that is included in
our First-Year Student Survey, too: When you think about your experience so far, how satisfied are
you all in all with your first semester at the University of Cologne? Students are invited to give their
answers using a five-point scale ranging from 1 = very unsatisfied to 5 = very satisfied.

A correlation analysis revealed a highly significant and rather strong positive relation between
students satisfaction measured as previously described and their willingness to recommend the
University of Cologne (Pearsons r = .51, p < .001, n = 219). Although the object of referral differs,
students overall satisfaction and willingness to recommend their courses of studies show a highly
significant yet somewhat weaker correlation as well (Pearsons r = .32, p < .001, n = 217).

This first bivariate analysis conveys the impression that the NPS questions might indeed be quite
suitable measures of satisfaction and loyalty in higher education. Therefore, let us now take a more
detailed look at the NPS itself. To that end, students first have to be classified as detractors, passives
and promoters according to their answers to the referral questions. The Net Promoter Scores can
then be calculated as described above (NPS = % of promoters - % of detractors). Figure 3 shows the
results.

6 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
Figure 3: Detractors, passives and promoters among first-year students

The largest share of students belongs to the group of the passively satisfied (36.1 %) with regard to
the University of Cologne. 34.2% and 29.7% of students are promoters and detractors, respectively,
which results in a Net Promoter Score of 4.5 for the university as a whole. As has been suspected
above, this score differs from the NPS for the courses of studies, which amounts to 8.2. However, in
the following, we will disregard this difference and focus on the university as a whole. We do so
because, on the one hand, Net Promoter Scores vary considerably between different courses of
studies. This needs to be examined more thoroughly than would be possible in this paper. On the
other hand, the number of respondents in some courses of studies is yet too small to make such an
examination possible. Hence, this will be subject to future research.

Now that we know the universitys NPS, the question of how to productively work with this measure
(i.e. how to increase it) has to be dealt with. The logic of calculation implies that there are only two
ways to increase the NPS: decreasing the percentage of detractors by turning them into passives
and/or increasing the percentage of promoters by turning the passives into promoters. Thus, one
needs to learn more about why detractors and passives are not willing to give a recommendation.

More than half of the students who gave a rating on the NPS scales also described the reasons
afterwards (n = 119). Their free text answers to our second question (Please briefly describe the
reasons for your ratings given above) enable us to analyse their assessments. Since our analyses are
based on a preliminary dataset, we restrict ourselves to presenting only some typical statements
from each group of students, starting with the group of detractors. A systematic categorisation of the
answers will be developed with the final dataset.

Detractors mostly complain about a lack of appropriate information, especially in the beginning of
their studies. Some also perceive the organisation of the university as chaotic, e.g. the admission
procedure with the universitys campus management tool KLIPS:
I rated the university rather low because it is unorganised and as a freshman you hardly get any
information. (student 76)
I already had several problems with klips this semester which costs time and nerves. (student 247)

One other reason for the detractors low ratings is the feeling of anonymity on the campus because
of the universitys size (in fact, the University of Cologne is one of the largest German universities
with nearly 50,000 students):
I would not recommend the University of Cologne because of many reasons. The two most important
are: 1. Because of the high number of students you really feel anonymous. 2. You are hardly integrated
at the beginning of studies. (student 47)

7 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
The free text answers given by passively satisfied students leave an ambivalent impression. Some
face the same problems as the detractors but feel comfortable on the campus:
More information especially at the beginning of studies would have been helpful. [] One often feels
lost at the beginning. [] However, the professors, tutors and lecture contents are great!!! (student 78)
I do not think about the university itself that much but I feel quite comfortable. (student 32)
Others are not satisfied with their courses of studies, which obviously affects their ratings of the
university as a whole:
My course of studies does not completely meet my expectations. (student 63)

Finally, the promoters are satisfied with the University of Cologne and their courses of studies. They
describe the atmosphere at the university as positive, are happy about the variety of extracurricular
offers such as sports and leisure activities and would recommend the location Cologne and its
university without doubts:
The University of Cologne is cosmopolitan, tolerant and variegated. There are lots of interesting offers,
e.g. possibilities for sports and leisure activities. Everything is done to make life in the city and at the
university as pleasant as possible and to support students in every way! (student 130)
The University of Cologne has fully met my expectations. I feel good here. Contacts among students
and between students and professors are special and therefore the atmosphere is pleasant. [] I would
recommend my course of studies, especially at the University of Cologne anytime. (student 20)

Discussion
The aim of this paper was to adapt the Net Promoter Score as a measure of satisfaction and loyalty to
the needs of evaluation and quality assurance in higher education. We started with theoretical
considerations about this idea and came to the conclusion that the NPS is in fact applicable to this
new context. We then examined its empirical usefulness with data from our universitys First-Year
Student Survey. Respondents answers to an established question measuring student satisfaction and
to the newly integrated NPS questions correlated strongly. Therefore, using the NPS for our purpose
seemed perfectly justified from an empirical point of view as well.

After that, we started to actually work with the Net Promoter Score itself and determined a score of
4.5 for the University of Cologne. A positive value is undoubtedly a satisfying outcome. However,
comparing this number to other universities NPS is hardly possible because it is not used very often
or serves different purposes (e.g. the evaluation of lectures, cf. Dvorakova & Faltejskova 2014). One
rare example that can serve as a reference, though, is the University of Bonn, which is located close
to Cologne and has a similar size. There, students were asked about their willingness to recommend
their university in several different surveys over a number of years yielding mostly negative Net
Promoter Scores that ranged between approx. -15 and -24 (cf. Hrsch & Rudinger 2009, p. 79-81).
The comparison of the Net Promoter Scores thus comes out in favour of the University of Cologne.
Yet, one other finding applies to both universities, namely the relatively large shares of passively
satisfied students. While Hrsch and Rudinger (2009, p. 85ff.) stay somewhat vague concerning
possibilities to use this potential for increasing their NPS, the free text answers we collected enable
us to pick up on what keeps students from recommending their alma mater. As the analysis in the
previous chapter showed, the main reasons from the detractors and passives points of view are a
lack of appropriate information, the feeling of anonymity, the organisation of the university and the
complexity of the KLIPS web portal.

Closing the customer feedback loop (Markey et al. 2009, p. 43) by revising the information supply
for prospective and first-year students or by offering easy-to-understand instructions for the
universitys online services might enable us to minimise the number of detractors, turn the passives
into promoters and consequently improve the universitys Net Promoter Score.

8 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
Outlook
Capturing the Net Promoter Score among first-year students was just a first step towards establishing
this measure as a monitoring tool for student satisfaction at the University of Cologne. We argue in
favour of taking the next steps too since the NPS clearly showed some of its qualities already, e.g. its
ability to register both emotional and rational dimensions in students assessments of the university
(e.g. the feeling of anonymity vs. the wide range of course offers).

Our route of further development comprises three stages. First, we plan on asking the NPS questions
in the next rounds of our First-Year Student Survey again. Linking data from numerous rounds will
enable us to conduct detailed analyses even for courses of studies with small numbers of students.
Second, we will integrate the NPS questions in our Graduate Tracer Studies. This will make it possible
to see if there are differences in the willingness to recommend the university between individuals
with different perspectives on their academic trainings (i.e. freshmen and graduates). Finally, we plan
on linking the NPS values to objective measures that serve as an approximation for a universitys
reputation, e.g. the number of applications. All of this will take some time and effort. But only then
will we be able to give a comprehensive assessment of the Net Promoter Scores usefulness in higher
education. With our first experience in mind, we would recommend to make the effort and give the
NPS a try.

9 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education
References
Dvorakova, L. & Faltejskova, O. (2014). Net Promoter Score Use at University Education Sector,
SGEM2014 Conference Proceedings 3(), 567578.
Gotsi, M. & Wilson, A. M. (2001). Corporate reputation: seeking a definition, Corporate
Communications: An International Journal 6(1), 2430.
Hrsch, K. & Rudinger, G. (2009). Der Net Promoter Score an der Universitt Bonn. In: Rudinger, G.,
Hrsch, K. & Krger, T. (ed.), Forschung und Beratung - Das Zentrum fr Evaluation und Methoden.
Gttingen: Bonn University Press, 7588.
Katsarova, I. (2015). Higher education in the EU. Approaches, issues and trends. Available online:
http://www.europarl.europa.eu/EPRS/EPRS-IDA-554169-Higher-education-in-the-EU-FINAL.pdf
(02.08.2015).
Keiningham, T. L., Aksoy, L., Cooil, B. & Andreassen, T. W. (2008). Linking Customer Loyalty to
Growth, MIT Sloan Management Review 49(4), 5157.
Keiningham, T. L., Cooil, B., Andreassen, T. W. & Aksoy, L. (2007). A Longitudinal Examination of Net
Promoter and Firm Revenue Growth, Journal of Marketing 71(3), 3951.
Langer, M. F. et al. (2001). Hochschulbindung. Entwicklung eines theoretischen Modells, empirische
berprfung und Ableitung von Handlungsempfehlungen fr die Hochschulpraxis. Available online:
http://www.che-ranking.de/downloads/hs_bindung_bericht0105.pdf (02.08.2015).
Lenecke, K. (2005). Hochschulbindung durch Student Services. Grundlagen, Analyse, Perspektiven.
Saarbrcken: VDM Verlag Dr. Mller.
Markey, R., Reichheld, F. & Dullweber, A. (2009). Closing the Customer Feedback Loop, Harvard
Business Review 87(12), 4347.
Pingitore, G., Morgan, N. A., Rego, L. L., Gigliotti, A. & Meyers, J. (2007). The Single-Question Trap,
Marketing Research 19(2), 913.
Reichheld, F. F. (2001). Loyalty rules! How today's leaders build lasting relationships. Boston: Harvard
Business School Press.
Reichheld, F. F. (2003). The One Number You Need to Grow, Harvard Business Review 81(12), 4654.
Reichheld, F. F. (2006). The ultimate question. Driving good profits and true growth. Boston: Harvard
Business School Press.
Schwaiger, M. (2003). Der Student als Kunde. Eine empirische Analyse der Zufriedenheit Mnchner
BWL-Studenten mit ihrem Studium, Beitrge zur Hochschulforschung 25(1), 3262.
Sharp, B. (2008). Net Promoter Score Fails the Test, Marketing Research 20(4), 2830.
Sursock, A. & Smidt, H. (2010). Trends 2010. A decade of change in European Higher Education.
Available online:
http://www.eua.be/typo3conf/ext/bzb_securelink/pushFile.php?cuid=2759&file=fileadmin/user_
upload/files/Publications/Trends_2010.pdf (02.08.2015).
Theus, K. T. (1993). Academic reputations: The process of formation and decay, Public Relations
Review 19(3), 277291.

10 Students as customers: The Net Promoter Score as a measure of satisfaction and loyalty in higher education