NiccL/i ac/NNseN

is an engagement
manager in
McKinseys Hamburg
ofce, sircif kcNic
is a principal in
the Berlin ofce,
and ueNNc v/N
biak is a director
in the Amsterdam
ofce. Copyright
McKinsey &
Company. All rights
reserved.
Customer-Sentiment Mining
Whats All the Buzz About?
nicolai johannsen, birgit knig, menno van dijk
Flippin Awesome Car!
It looks drop dead gorgeous! Performance: best that I have ever
experienced. The handling of the car on winding roads literally
puts a smile on your face. Bi-turbo engine is amazing with plenty
of horsepower and torque. Beautiful interior with leather covering
all surfaces and sport seats that hug you in spirited driving.
www.cargurus.com
Hate the Run Flats
Beautiful car: I get compliments everywhere I go. It handles well,
cruises like nothing else, and gets good mileage. However, I hate
the run at tires. They are hard and noisy and I am paranoid to
take a road trip due to stories I have heard.
www.carreviews.net
Mining the Web
for customer
responses may be
less hazardous to
the health than more
traditional mining
activities, but it can
be just as hard.
H
A
R
R
Y

G
R
U
Y
A
E
R
T

/

M
A
G
N
U
M

P
H
O
T
O
S
8 | the journal of problem solving
customer-sentiment mining | 9
W
ord of mouth has always been an important inuence on
consumer behavior. But now, the asynchronous nature of blogs
and reviews on the Internet lets consumers obtain a profusion
of opinions en masse, without having to converse directly with their peers.
A consumer can review thousands of comments on just about any possible
product whenever he or she needs the information. Today, more than 700
million people worldwide say they use blogs and reviews to make buying
decisions.1 And this group is no longer just teens and twenty-somethings.
According to two major recent surveys, Web usage among those in the
5564 age range is now 75 percent approaching the 91 percent gure for
those 1829.2
What makes online reviews persuasive? Research suggests that the most
trusted source of information about a product or service is a person
like me. A recommendation from fellow consumers tends to beat an
ad with regard to the level of trust inspired.3 When judging whether an
online reviewer is a person like me, people respond to a variety of cues:
language, personal context, expressed preferences, and so on. What makes
a review credible and interesting are these details and items of personal
information specics that an ad cannot replicate rather than the rat-
ing per se.
Online reviews are generally unprompted communications. People write
for fellow consumers, not for a market research agency and not for the man-
ufacturer. And because they are acting on their own initiative rather than,
say, responding to a set of questions on a product or service they only
discuss those features that come to mind. Here then, nally, is a source of
customer insights in which the observer does not inuence the opinions
articulated in any way. It has been a long-standing challenge to marketers
that the context in which interviews, focus groups, and other forms of inter-
active discovery take place biases peoples responses. Online reviews are a
rare case where this seems not to be a problem.
Admittedly, objections have been made to the value of these online
sources. Some ask whether companies are seeding review sites with fake
reviews. There have been isolated cases of companies attempting to create
buzz by paying people to oer apparently objective endorsements. But our
impression is that seeding is rare, because the downside of embarrassment
when such eorts are exposed far exceeds the upside of success. Regular
bloggers appear to be very good at detecting specious posts and are ruthless
about outing them.
A second objection is that the consumers who write detailed reviews
online are not representative. It is probably the case that such reviewers are
deep enthusiasts on the given topic and thus not typical of the enormous
online population. But this is also precisely what makes them credible and
why their opinions spill over into the choices of a much larger audience.
1 World Internet
usage and popula-
tion statistics 2008,
Nielsen online
global consumer
study 2007, Deloitte
& Touche online
marketing report
2007.
2 Forrester Internet
usage report 2008,
Pew/Internet report
2008. Part of what is
driving this develop-
ment is that the bias
in Internet usage
toward those with
more education and
higher incomes is
intensifying. Almost
all households with
a yearly income of
more than $75,000
use the Internet,
whereas only one in
two of those with
an income below
$30,000 is online.
Overall, this skews
usage to a higher age
bracket.
3 Nielsen online
global consumer
study 2007.
10 | the journal of problem solving
Full of information but hard to get at
Online reviews are full of a kind of detail that makes them powerful inu-
encers of consumer behavior. As such, they represent a potential goldmine
to companies. Among other benets, they highlight which of a products
features consumers most often discuss and how the focus of consumer
discussion shifts in response to a new ad campaign.
Customer reviews also identify the pros and cons of a given product
compared with the competition and allow companies to determine how
their R&D eorts pay o from one product or product series to the next.
Unfortunately, however, it is precisely the qualities that make customer
reviews so powerful that also make them hard to mine for the sentiments
they contain. There are usually thousands of posts on any given topic. This
obviously increases the statistical reliability of the information they yield,
but it also means much more data to plough through. Detailed discussions
of which features consumers like and why provide a trove of fantastic infor-
mation but that information is embedded in the prose of endless pages of
online postings. None of this material is available by count; little of it oers
quantitative ratings. There is no simple way to aggregate these opinions, and
reading through thousands of posts would clearly prove impractical.
Various commercial applications are currently available or are being
developed in response. The challenges that these applications try to resolve
include dening which sites provide the greatest wealth of information,
developing topic-specic natural-language algorithms that interpret the
data correctly, and building user interfaces that allow companies to synthe-
size ndings into the results needed for business decisions (rather than an
overload of nice-to-have data charts).
The World Wide
Web offers an
unusual challenge:
so much specic
information, so hard
to aggregate.
R
Y
A
N

M
C
V
A
Y
customer-sentiment mining | 11
A Firm solution
The Firm has developed a response to these obstacles that provides access to
a tool that generates close to 80 percent quality, along with all the data cuts
a team might need and the analysis to go with them. We worked with Bing
Liu, a professor at the University of Chicago, to dene the types of analyses
and the data format we need, as well as mechanisms that allow teams to
input industry and client data. Professor Bing, perhaps the most prominent
gure in the world of customer-sentiment mining, has spent the past ve
years developing algorithms for sentiment classication.
The software crawls the Web, extracts English-language posts, disag-
gregates them into statements about a products individual features, and
determines the sentiment of each statement. A typical analysis generates a
pictorial representation such as that shown in Exhibit 1.4
Exhibit 1: Feature comparison of three automobiles
Exhibit 1 oers a comparative analysis of three midsize cars based on a
variety of dierent features. The bars provide the number of counts by
brand and feature. The part of the bar above the line indicates the number
of positive opinions; the part below the line, the negative ones. Clearly
visible are strengths and weaknesses by brand. For instance, the BMW 3
series notches up 300 opinions that it is fun to drive but also garners 60
negative remarks about its tires. More detail on what exactly appears to
be wrong with the tires is accessible by clicking on the column for tires,
which reveals the actual statements underlying the numerical analysis
(Exhibit 2). Another click takes you to a post itself, which allows you to
assess the context in which a given statement was made useful both
for checking the quantitative evaluation and for providing illustrative
quotes for an analysis.
4 The customer-
sentiment research
conducted for this
article was carried
out by a specialized
vendor with which
the Firm has an
exclusive consulting
relationship. It was
not conducted by the
Firm in the context
of client service or
client development.
12 | the journal of problem solving
Exhibit 2: Underlying quotes for negative tires rating
The tool oers several other data cuts. You can assess the overall number
of mentions by feature or by brand, and you can see how posting activity
develops over time in response to new campaigns or new product introduc-
tions. Exhibit 3 plots the relative intensity of positive responses to the intro-
duction of the new BMW 3 series in fall 2007 against positive responses to
the Audi A4.
Exhibit 3: Comparison of positive responses
Finally, as noted, the tool allows for experimentation with dierent cuts of
product data. You can aggregate product information into summary data
about product lines and compare sentiment about dierent product lines
among competitors.
The analysis is fast and cost ecient. A typical analysis requires about
eight working days to compare, say, ten dierent brands or products across
customer-sentiment mining | 13
20 dierent features or categories of features. No traditional brand analysis
we know of can match this level of eciency and eectiveness.
How to proceed
To get started on an analysis of the kind described here, please contact the
MCAPS team at mcaps@mckinsey.com. After discussion to clarify the pre-
cise process that will yield the desired results, the rst step is to determine
whether there are enough information-rich comments on the Web that
cover the area of interest. Review sites usually work better than blogs and
forums. As a rule of thumb, 1,000 posts or more are needed.
The MCAPS team will work with you to identify which URL or URLs
to mine for each brand you need to analyze. We program the software to
crawl the URLs and extract all posts that discuss the brands selected. We
then work together to annotate a group of random sentences from the
resulting data set. This annotation step, which identies attributes or nouns
of interest to the client, is crucial to customizing the algorithm. It generally
takes a few hours and requires a good understanding of the specic product
or market category.
The annotation can be created at any level of aggregation. For one com-
pany, for example, customer service may be suciently precise, whereas
another company may want to dierentiate among call center, warranty,
and after-sales service.
The resulting list of attributes or nouns allows the software algorithm
to tabulate references across the whole data set by category. Because the
software uses a proprietary thesaurus rather than relying on a standard one,
results are properly tailored to individual needs.
Customer-sentiment
mining sorts
and aggregates
communications
rather like the post
ofce does.
C
O
M
S
T
O
C
K

/

C
O
R
B
I
S
14 | the journal of problem solving
The algorithm has been benchmarked as best in class, but it is still dou-
ble-checked manually in an internal editing step. Once this nal quality-
control step is complete, the results are provided online. All the data can be
exported to Excel for further processing, such as cluster analysis or correla-
tion analysis, as well as to facilitate transfer into Think-Cell for presentation
graphically.
W
e are still in the early days of uncovering what customer-sentiment
mining can do. For instance, there is a great deal of work under way
on the analysis of the emotional content of posts, along with many other
applications not discussed here.
Nevertheless, we hope we have provided a gateway to what is, in fact, a
kind of marketers utopia: quick, easy, inexpensive access to an enormous
data set of unbiased input on products.
For further information on this topic, visit:
http://mcaps.intranet.mckinsey.com