Electronic Commerce Research and Applications 12 (2013) 289–296

Contents lists available at SciVerse ScienceDirect

Electronic Commerce Research and Applications

journal homepage: www.elsevier.com/locate/ecra

Concept extraction and e-commerce applications

Yongzheng Zhang ⇑, Rajyashree Mukherjee, Benny Soetarman
eBay Inc., 2065 Hamilton Ave., San Jose, CA 95125, USA

a r t i c l e i n f o a b s t r a c t

Article history: Concept extraction is the technique of mining the most important topic of a document. In the e-com-
Received 11 November 2012 merce context, concept extraction can be used to identify what a shopping related Web page is talking
Received in revised form 3 March 2013 about. This is practically useful in applications like search relevance and product matching. In this paper,
Accepted 18 March 2013
we investigate two concept extraction methods: Automatic Concept Extractor (ACE) and Automatic Key-
Available online 11 April 2013
phrase Extraction (KEA). ACE is an unsupervised method that looks at both text and HTML tags. We
upgrade ACE into Improved Concept Extractor (ICE) with significant improvements. KEA is a supervised
Keywords:
learning system. We evaluate the methods by comparing automatically generated concepts to a gold
Concept extraction
Automatic keyphrase extraction
standard. The experimental results demonstrate that ICE significantly outperforms ACE and also outper-
e-Commerce forms KEA in concept extraction. To demonstrate the practical use of concept extraction in the e-com-
Product matching merce context, we use ICE and KEA to showcase two e-commerce applications, i.e. product matching
Topic-based opinion mining and topic-based opinion mining.
Ó 2013 Elsevier B.V. All rights reserved.

1. Introduction is to identify what a shopping-related page is talking about. This is

Online shopping has never been more popular than nowadays.
Many factors contribute to the prominence of online shopping.
Customers can do online shopping in minutes, which is much more
efficient than conventional shopping in local stores. There are often
good deals online with additional benefits such as free shipping
and no tax. Moreover, online shopping is becoming more secure
as more security policies are being enforced by many online
marketplaces.
Generally speaking, online shopping starts with research and
shopping research starts with search. Online shoppers often use
main stream search engines such as Google to find products in
the search results and sponsored advertisements. More than often,
returning buyers bookmark their favorite shopping sites such as
eBay1 and BestBuy2 and shop there directly.
In order to provide a more streamlined user experience in shop-
ping related research, it is critical for e-commerce sites to design a
platform of technologies that delivers structured content and
highly relevant results to the user. The ideal platform includes
search, organization, and summarization of any ‘‘shopping-
related’’3 pages on the World Wide Web. One essential component
⇑ Corresponding author.
E-mail addresses: ytzhang@ebay.com (Y. Zhang), rmukherjee@ebay.com (R.
Mukherjee), bsoetarman@ebay.com (B. Soetarman).
1
http://www.ebay.com.
2
http://www.bestbuy.com.
3 4
A shopping-related Web page is one that contains information about deals,
product specifications, professional and consumer reviews, FAQs, etc.
the motivation behind the concept extraction work in the e-com-
merce context.
Note that there is a subtle distinction between ‘‘talking about’’
and ‘‘mentioning’’. We are more interested in finding out what a
web page is ‘‘mainly about’’ instead of ‘‘also mentioning’’. By
‘‘mainly about’’ we mean the most essential topic of a Web page.
For example, there is a review page about Dell Latitude D4204.
The review also compares this model against other laptops such as
IBM ThinkPad X41. However, the main topic of this page is ‘‘Dell Lat-
itude D420 review’’. Other laptops are ‘‘mentioned’’ but they are not
the main focus. This is how we define the concept of a shopping-re-
lated Web page. We aim to identify such most essential concepts
using concept extraction techniques.
The technique of extracting concepts from Web pages is derived
from key phrase extraction from traditional text documents in
information retrieval and text mining fields. A phrase can be a sin-
gle word or a multi-word term, which is also known as n-gram.
Concept extraction is very useful in many text-related applications,
such as search, classification, clustering, and visualization.
In this paper, we investigate two existing key phrase extraction
tools: Automatic Concept Extractor (ACE) (Ramirez and Mattmann
2004), and Automatic Keyphrase Extraction (KEA) (Witten et al.
1999), in the concept extraction task.
ACE is an unsupervised method that looks at both text
and HTML tags. The Term Frequency (TF) scorer calculates the
http://www.notebookreview.com/default.asp?newsID=3101, last viewed on
March 9, 2012.

1567-4223/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.elerap.2013.03.008
290 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296
popularity of a term in the text body, and the HTML scorer assigns
significance weights to a pre-defined set of HTML tags so terms in a
pair of tags will be given a significance weight. Finally, the Concept
Miner combines the above two parts using given weights to rank
candidate concepts.
KEA is a supervised learning system. It first builds a Naive Bayes
model from training documents where concepts are manually as-
signed. Two features are used in training the model: Term Fre-
quency – Inverse Document Frequency (TFIDF) and first appearance,
which is the normalized distance in terms of number of words to
the beginning of the document. The trained model is then used
to find concepts in new documents.
Our focus in this paper is to perform various improvements to
the basic ACE to obtain Improved Concept Extractor (ICE). We
use ACE and KEA as benchmarking in experiments. In order to eval-
uate the three systems, we create a collection of 100 Web pages
from leading brand sites such as Dell, HP, and Canon. We create
a gold standard by manually assigning concepts to each page in
the collection. We tune different parameters of ACE, ICE and KEA
to generate concepts. And we use precision, recall and F1 to evaluate
the concepts. The experimental results indicate that ICE performs
significantly better than ACE and better than KEA in concept
extraction.
To further demonstrate the practical use of concept extraction
in the e-commerce context, we use ICE and KEA as examples to dis-
cuss two applications: product matching and topic-based opinion
mining. In product matching, we use concept extraction methods
to identify the most important topic of a product review Web page
and decide if the top concept is matching the manually verified
product. This can be very useful for search indexing, relevance
ranking, advertising, and many others. In topic-based opinion min-
ing, we find the topics of a collection of documents where opinions
of each topic can be mined.
The main contributions of our paper are two fold: (1) We pro-
vide an extensive analysis of concept extraction tools. We specifi-
cally upgrade an existing tool with key improvements and
demonstrate its performance by running experiments. (2) We
showcase two key applications of concept extraction in the e-com-
merce field. To our knowledge this is the first study that analyzes
how concept extraction methods can be used in e-commerce
applications.
The rest of the paper is organized as follows. Section 2 reviews
concept extraction tools in the literature and Section 3 explains in
detail how different concept extraction systems work. In Section 4,
we describe the experiments and present the evaluation results.
Then we discuss two practical e-commerce applications of concept
extraction in Section 5. Finally, Section 6 concludes our work and
describes future research directions.
2. Related work
Concept extraction is derived from key phrase extraction in
information retrieval and text mining fields. Key phrase extraction
methods often analyze a document to determine the significance of
a phrase, which can be a single word or a multi-word term. The sig-
nificance of a phrase is measured by modelling statistical features
such as frequency of occurrence and linguistic features such as
part-of-speech. The phrases above a certain threshold are often re-
ferred to key phrases or concepts.
Many key phrase extraction methods have been proposed in the
literature. TFIDF is a popular method that is widely used in infor-
mation retrieval and machine learning areas. The intuition behind
this method is that phrases, which appear frequently in one docu-
ment but rarely in the whole document collection, often have
high discrimination power between documents. TFIDF requires a
collection of documents to compute the significance score. Thus,
it has been widely used as the bag-of-words representation in
text-based applications, such as document categorization and doc-
ument clustering.
Krulwich and Burkey (1996) use a pre-defined set of heuristic
rules to extract key phrases from a document. The rules are based
on lexical features such as the use of acronyms and visual clues
such as the use of italics. The extracted key phrases are used as fea-
tures in an automatic document classification task. Turney (2000)
proposes GenEx, a key phrase extraction system that is based on
rule learning using a genetic program. More specifically, the sys-
tem consists of a pre-defined set of parameterized heuristic rules
that are tuned to the training documents by the genetic program.
The learned optimized rules are then applied on new documents
to extract key phrases. However, the above two methods heavily
depend on heuristic rule pre-defining and tuning, which is very
expensive to apply on new applications.
Ramirez and Mattmann (2004) propose a system, ACE, which is
specifically designed for concept extraction from Web pages. ACE
analyzes both the text body of a page and visual clues in various
HTML tags to extract concepts from a single Web page. In Section 3,
we will discuss why we choose ACE as a benchmarking method
and perform key improvements for the concept extraction task.
Witten et al. (1999) develop a key phrase extraction system
called Automatic Keyphrase Extraction (KEA). KEA builds a Naïve
Bayes learning model using training documents with known key
phrases, and then uses the model to find key phrases in new doc-
uments. Song et al. (2003) introduce a method which uses the
information gain measure to rank the candidate key phrases based
on the TF-IDF and distance features, which were first proposed in
KEA (Witten et al. 1999).
The above methods are designed for key phrase extraction from
individual documents. Frantzi et al. (2000) propose a method
named C-value/NC-value (CNC), which consists of both linguistic
and statistical analysis to extract key phrases automatically. It is
designed for key phrase extraction from an entire document
collection.
More recently, Parameswaran et al. (2010) propose a system
that extracts concepts from a large dataset such as user tags (e.g.
del.icio.us) or query logs of search engines (e.g. AOL). The system
uses techniques similar to association rule mining in the market
basket analysis and aims towards building a web of concepts. Fea-
tures such as frequency of occurrences and popularity among users
are used to extract core concepts while sub-concepts and super-
concepts of the core concepts are pruned. The authors claim that
the system can be applied to any large data set. However, if Web
pages are used, a lot of additional processing is needed to identify
the popular concepts.
Table 1 summarizes the concept extraction methods by the fea-
tures they have.
The first feature, supervised learning, is the process of training a
model with examples and then applying the learnt model on new
documents. It means that there is a human effort to provide train-
ing examples. In the concept extraction context, a supervised
method analyzes training documents with manual concepts to find
out rules and patterns, which can be used to find concepts in new
documents. For example, do the manual concepts have something
in common, such as do they often appear in the title or headings,
do they appear more frequently than phrases that are not con-
cepts? The second feature is whether the concept extraction tool
requires a document collection. Document collection means that
there is a crawling effort to collect training documents, which is of-
ten an expensive effort. The third feature is whether the tool works
on raw text only, or does it work better on visual clues found in
HTML tags, or both?
 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296 291

Table 1 Table 2
Summary of concept extraction methods by various features. A list of example HTML tags and corresponding significance weights.

Method Supervised? Collection? Text/HTML Tag Meaning Weight

TFIDF No Yes Text B Bold 0.8
GenEx Yes Yes Text BIG Big text 0.8
ACE No No Both DT Definition term 1.0
KEA Yes Yes Text EM Emphasis 0.3
CNC No Yes Text H1 Heading 1 1.0
H2 Heading 2 0.85
H3 Heading 3 0.7
H4 Heading 4 0.5
H5 Heading 5 0.3
Concept extraction has many practical applications, including H6 Heading 6 0.1
Web search (e.g. (Parameswaran et al. 2010)), education (e.g. (Vill- I Italic 0.3
alon and Calvo 2009)), and digital economy (e.g. (Piao et al. 2010)), Li List item 0.2
to name a few. In this work, we focus on e-commerce applications. STRONG Strong emphasis 0.8
TH Table header 0.5
We will explain why we choose ACE and KEA and describe how TITLE Document title 1.0
they work in the next section. U Underline 0.7

3. Concept extraction

In this section, we discuss how ACE and KEA extract concepts

from Web pages. These two methods have been extensively stud-
ied in the related work and we use them for benchmarking in
our experiments. We will focus on how to improve ACE to obtain
ICE.
3.1. ACE
Ramirez and Mattmann (2004) propose Automatic Concept
Extraction. We choose ACE because: (1) it is specifically designed
for extracting concepts from HTML pages; (2) it is an unsupervised
method, which means no training is needed; (3) it does not require
a document corpus and it is open source.
For a given Web page, ACE finds the concepts in the following
steps:
3.1.1. Tokenization
For a given web page, ACE first tokenizes the text body of the
page using blank spaces. This is the simplest tokenization policy
and we will discuss more complex tokenization in ICE.
3.1.2. TF scorer
The TF scorer measures the popularity of a token in a given Web
page, which is normalized with the maximum term frequency in
the document, as shown in Eq. (1).
f ðti Þ
STF ðti Þ ¼
maxni¼1 f ðt i Þ
where f(ti) is the frequency of occurrence of token i in the target
Web page.
3.1.3. HTML scorer
The HTML scorer measures the importance of a token within a
pair of HTML tags. All the HTML tags are assigned a pre-defined
weight. For example, text in the title and headings has more signif-
icance than text in a regular paragraph. Examples of HTML tags and
corresponding significance weights are summarized in Table 2.
3.1.4. Concept miner
The concept miner is a combined scorer to estimate the concept
likelihood of each token. It assigns the weighted sum of the TF
score and the HTML score to the tokens. Let STF and SHTML be the
TF score and the HTML score, respectively. Also let S be the com-
bined score, which is calculated by S = k  STF + (1  k)  SHTML. Note
that k is a number between 0 and 1 inclusive, representing the rel-
ative importance of the TF score versus the HTML score. Tokens
with a score above the pre-defined threshold, say 0.5, are consid-
ered qualifying tokens.
3.1.5. Concept derivation
ACE also derives n-grams on both sides of each qualifying token.
For example, suppose there is a qualifying token called ‘‘laptop’’
appearing in a piece of text ‘‘dell laptop battery’’. ACE derives n-
grams on both sides of the token ‘‘laptop’’. When n = 2 is used,
two derived n-grams ‘‘dell laptop’’ and ‘‘laptop battery’’ are found.
In ACE, all derived concepts will be scored using only the TF scorer.
Thus, derived concepts that are not frequent enough will be
dropped. In our improved ICE, we use the combined scorer instead.
Hence, derived concepts that are not frequent enough but have
high HTML score may still surface as a key concept.
3.2. Improved ACE (ICE)
The basic ACE is a simple and straightforward framework that
analyses both text and HTML visual clues, but does not utilize lin-
guistic and lexical features embedded in the text. We aim to up-
grade ACE by a few key improvements, which take advantage of
rich linguistic and lexical information. The improvements are sum-
marized in Table 3.
1. The first improvement is about the tokenization policy. In
the basic ACE, text is tokenized by blank spaces only. In
ð1Þ the improved ACE, we tokenize text using all non-alphanu-
merical characters.
2. The second improvement is about detecting the HTML tag
boundary. In the basic implementation, ACE disregards HTML
tag boundaries. For example, given the following text: <a href
= ‘‘http://buy.ebay.com/cell-phone’’ > cell phone < /a> <a href
= ‘‘http://www.ebay.com/’’ >Home < /a > , ACE will extract
‘‘cell phone home’’ as a candidate concept, which is not true.
In ICE, we respect tag boundaries, i.e. we do not combine tokens
from different HTML tags, so the candidate concepts become
‘‘cell phone’’, and ‘‘home’’. Similarly, we detect sentence
boundaries so tokens across two neighboring sentences are
not taken as candidates.
3. The third improvement is about how to score derived con-
cepts. The basic ACE uses only the TF scorer to score derived
concepts and normalize to the highest TF score of a derived
concept. For instance, if the parent concept is A with a score
of 0.5 and it has three derived concepts: AB, DA, and ABC (A,
B, C, D are different single words), where DA happens to have
the highest TF out of the three, then DA would get a score of
292 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296

Table 3 KEA is a domain-independent method as shown in Witten et al.

Summary of improvements to the basic ACE. (1999), which means a KEA model trained on one domain (e.g.
Improvements ACE ICE computer science) performs well on another domain (e.g. biology).
Tokenization Blank spaces Non-alphanumeric characters The training set bundled with the Java-based KEA package (Version
Tag and sentence boundary No Yes 2.0)6 is used to train a CSTR KEA learning model. This data set con-
Scoring derived concepts TF scorer Combined scorer tains 80 abstracts of Computer Science Technical Reports (CSTR)
Emphasis scorer No Yes from the New Zealand Digital Library project.7 Each abstract has five
Porter stemming No Yes
human-authored key phrases. The input to the Java program consists
of text files with the corresponding key phrases. Research in Witten
1.0. This is not reasonable in many times. In ICE, we use the et al. (1999) shows that a training set of 25 or more documents can
combined scorer instead and normalize the derived concept achieve good performance.
with regard to the parent’s score. Thus in the above example,
DA has a score of 1.0  0.5 = 0.5. 3.3.2. KEA extraction
4. The fourth improvement is that we add a new scorer, In the extraction stage, KEA uses the model to find the best set
Emphasis scorer, to the basic ACE. We observe that there of (by default 5) key phrases in new documents. More explicitly,
is often some overlapping between the title and the KEA chooses a set of candidate key phrases from new documents
emphasized text and such overlapping text often repre- and calculates their two feature values as above. Then each candi-
sents the concept of a page very well. The emphasis scorer date is assigned a score, which is the overall probability that this
detects whether there is such an overlap. Each sequence of candidate is a key phrase. Candidates with the score above a cer-
overlapping words are considered as a candidate concept, tain threshold are taken as the concepts.
whose score will be the weight of the HTML tags within
which it is. Emphasis scorer extracts concepts irrespective
4. Experiments and evaluation
of TF and HTML weights. Only when emphasis scorer fails
to extract any concept then the Concept Miner comes into
In this section, we discuss how to construct a data corpus to
picture.
experiment with ACE, ICE and KEA. We evaluate these algorithms
5. The fifth improvement is to apply Porter stemming on
using standard metrics: precision and recall.
extracted concepts so concepts share the same stem will
be considered as the same. Stemming has been a typical
4.1. Data corpus
experience in text mining applications.
Collectively with all these five key improvements, we
Evaluation of automatically generated concepts often proceeds
achieve to improve ACE by 50% in terms of accuracy. For-
either in intrinsic mode, where concepts are compared against a
mal evaluation results are presented in Section 4.
gold standard created by human subjects, or in extrinsic mode,
which measures the utility of concepts in performing a particular
3.3. KEA
task (e.g. site indexing) Zhang et al. (2007); McKeown et al.
(2005). In this task, we take the intrinsic approach. We aim to
KEA stands for Automatic Keyphrase Extraction. It is an efficient
investigate how well these concepts reveal the main topic of a gi-
and practical algorithm for extracting key phrases in information
ven Web page. In other words, we are interested in the correctness
retrieval and text mining fields. KEA is now part of the machine
and completeness of the automatically generated concepts.
learning toolkit WEKA,5 and has been proven to be successful in
We ask human subjects to create a gold standard. More specif-
many text-based applications, e.g. Frank et al. (1999); Gutwin
ically, we first construct a data corpus with 100 shopping related
et al. (1999); Turney (2002, 2003); Zhang et al. (2004, 2007, 2005).
Web pages that are randomly selected from our database of lead-
KEA is a supervised method using the Naive Bayes approach,
ing brand sites (ranked by the number of incoming links) such as
which is a probabilistic model that applies Bayes’ theorem with
Dell, HP, and Canon. Second, we ask two experts in the e-com-
feature independence assumption. In other words, it estimates
merce field to manually review each page and pick up the best
the probability of a pre-defined class given a feature set based on
up to 10 concepts as golden standard in a double blinded fashion,
the prior probability of each class and the conditional probability
i.e. they work separately and do not interfere with each other’s
of the feature set given the target class. Then all classes are ranked
decision. We observe that most pages have 3–5 concepts. The
using the probability value.
agreement on extracted concepts between the two reviewers is
KEA consists of two stages: training and extraction. In both
83%. For concepts that are picked by only one reviewer, a second
stages, n-grams (n P 1) are extracted as candidate concepts. For
round review is conducted so both reviewers agree on concepts
each candidate, two feature values, TFIDF and first occurrence, are
for all the 100 pages.
calculated. TFIDF is a standard feature often used in text mining
Once the gold standard is created, we then run ACE, ICE, and
tasks. It favors terms that appear frequently in one document but
KEA to generate concepts with different configurations. Evaluation
rarely in a document collection. First occurrence is calculated as
proceeds with comparing automatically generated concepts with
the number of words that precede the candidate’s first appearance,
human authored concepts.
divided by the number of words in the document.
4.2. Evaluation measures
3.3.1. KEA training
In the training stage, each document is manually assigned a few
We use precision, recall and F1 to evaluate the quality of concept
concepts. Then a Naı̈ve Bayes model is built based on TFIDF and
extraction. In the context of concept extraction, precision measures
first appearance. More explicitly, KEA chooses a set of candidate
the fraction of extracted concepts that are also human authored
key phrases from input documents. Those candidates that happen
concepts; and recall measures the fraction of human authored
to be human-authored key phrases are positive examples in the
KEA model construction.
6
http://www.nzdl.org/Kea.
5 7
http://www.cs.waikato.ac.nz/ml/weka. http://www.nzdl.org.
 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296
concepts that are captured by concept extraction methods. Usually
there is a tradeoff between the two, i.e. it is hard to achieve both
high precision and high recall at the same time. F1 is such a mea-
sure that balances the two.
Given the confusion matrix shown in Table 4, precision P, recall
R, and F1 can be formally defined as follows:
A
P¼
AþB
A The results are summarized in Table 6.
R¼
AþC
2PR
F1 ¼
PþR
where A is the number of overlapped concepts between human-
authored concepts (golden standard) and program-generated con-
cepts, B is the number of extracted concepts that are not truly hu-
man authored concepts, and C is the human authored concepts
that are missed by the concept extraction methods.
Alternatively, we can use the acceptable percentage measure
proposed by Turney (2000) to evaluate automatically extracted
concepts. This usually proceeds with a formal user study where
people assign 1-to-5 scores to concepts and often a gold standard
is not required (if the method does not require any) for evaluation
purpose.
4.3. Evaluation results
4.3.1. Evaluation results of ACE
First of all, we present the evaluation results of basic ACE. We
have 100 pages with manually tagged concepts. All the 100 pages
are used to test ACE. We tune the following three parameters in or-
der to obtain concepts with the best quality.
 T: threshold on final concept score, which is a weighted combi-
nation of the TF score and the HTML score and a threshold of 0.5
is used by default. We experiment with five different threshold
values from 0.5 to 0.9 in increasing step of 0.1.
 B: maximum breadth of concept, i.e. maximum number of sin-
gle words allowed in a concept. A default value of 5 is used in
ACE. We experiment with 3 to 6.
 k: significance weight of the TF scorer (0.5 by default). We
experiment with five different weights from 0.3 to 0.7 in
increasing step of 0.1. Note that the weight of the HTML scorer
is 1  k.
When a threshold on final concept score is chosen (e.g. T = 0.6),
we experiment with each configuration of B and k (denoted as
[B, k]). This leads to a total of 20 configurations.
For each Web page, we calculate the precision, recall and F1
values. The quality of ACE is measured by the average precision,
recall and F1 over 100 Web pages. We aim to find out for each
threshold T which of the 20 configurations C yields the highest
average F1. Due to space limit, we report only the evaluation re-
sults of ACE using the best configuration. The results are summa-
rized in Table 5.
As we can see in Table 5, ACE tends to favor long concepts but
achieves low precision and recall in all cases. Also as the threshold
goes up, the overall quality is decreased. As an example in the sub-
optimal case for T = 0.5, when the configuration of [4, 0.5] is used,
the precision, recall, and F1 are 0.2245, 0.2583, and 0.2402,
respectively.
4.3.2. Evaluation results of ICE
The main focus of our paper is to investigate how to improve
ACE and demonstrate its performance compared with ACE and
KEA. We experiment with ICE using same configurations as for
293
Table 4
Confusion matrix of manual concepts and extracted concepts.
Manual Non-manual
Extracted A: true positive B: false positive
Non-extracted C: false negative D: true negative
ACE. We also report the results of ICE using the best configuration.
ð2Þ
As shown in Table 6, ICE significantly outperforms ACE in preci-
sion, recall, and F1. In most cases, more than 90% of true concepts
are captured by ICE and more than 70% of extracted concepts are
truly desired. This indicates that the improvements applied in Ta-
ble 3 are greatly beneficial. They capture the essential signals, i.e.
the linguistic and lexical patterns, that can identify the most essen-
tial topic in a shopping related Web page. This is very encouraging
and promising as we look into much larger data sets in the future.
We also observe that ICE tends to favor shorter concepts than ACE.
The best quality with F1 = 0.8325 is achieved when a threshold of
0.5 is used on concepts with up to 3 words and a 30% significance
weight on the TF score.
In addition to the improvements summarized in Table 3, we
also investigate whether applying CSS (Cascading Style Sheets8)
to the Web pages will impact concept extraction. We repeat the
same experiments with an addition of applying CSS. The results
are summarized in Table 7, which indicates that the quality of ex-
tracted concepts is actually worse than without CSS. This implies
that style sheets tend to redefine features such as color and font,
which consequently dilute the signals given by HTML markups that
are critical to the ICE algorithm.
4.3.3. Evaluation results of KEA
In KEA, we do differently in the sense that we use the first 50
Web pages for training and the rest 50 for testing.9 More specifi-
cally for training, we use two sets of concepts: (1) the ICE concepts
extracted using the best configuration (0.5 for threshold of concept
score, up to three words allowed in a concept, 30% weight for TF
score and 70% for HTML score); (2) the manual concepts. Our interest
resides on how good ICE concepts are compared to manual concepts
in the task of training a KEA model.
We tune two parameters in KEA, one is the breadth, i.e. the
maximum number of words allowed in a KEA concept, the other
is the threshold on the concept score. We observe that most KEA
concepts carry a score less than 0.5 so we use a range of threshold
on concept score from 0.2 to 0.5. The evaluation results are sum-
marized in Table 8.
When using ICE concepts of the first 50 Web pages for training a
KEA model, the evaluation results on the remaining 50 Web pages
are the same (P = 0.6583, R = 0.7600, F1 = 0.7056, and B = 3) when a
threshold of 0.2, 0.3, 0.4, and 0.5 is separately used. This can be ex-
plained by the fact that most often there is a huge gap between the
top KEA concepts (usually good ones) and the bottom ones (usually
not good). For example, Cisco, 0.5504; Cisco AS5350XM, 0.5504; Uni-
versal Gateway, 0.5504; DSP, 0.0399; voice, 0.0399; IP, 0.0399; net-
work, 0.0399. Consequently, applying a lower threshold has
minor impact on the quality.
8
http://en.wikipedia.org/wiki/Cascading_Style_Sheets, last viewed on March 9,
2012.
9
In order to evaluate KEA’s sensibility and stability across different training
examples, we conduct a fivefold cross validation on the 50 training examples, i.e. the
data set is evenly split into 5 segments and each time a different segment is left aside
for testing using the model trained on the rest four segments. Our fivefold cross
validation using specific configurations (e.g. T = 0.3,B = 4) shows that KEA is not
sensible to different training examples, i.e. the average F1 values across different folds
using the same configuration are very close to each other.
294 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296

Table 5 Table 6
Evaluation results of basic ACE with the best configuration. Evaluation results of ICE with the best configuration.

T 0.5 0.6 0.7 0.8 0.9 T 0.5 0.6 0.7 0.8 0.9
C [6, 0.4] [6, 0.4] [6, 0.3] [6, 0.6] [4, 0.3] C [3, 0.3] [3, 0.3] [3, 0.3] [3, 0.3] [3, 0.3]
P 0.2496 0.2429 0.2258 0.2046 0.1987 P 0.7227 0.7286 0.7396 0.7323 0.7391
R 0.2700 0.2450 0.2350 0.2017 0.1750 R 0.9815 0.9613 0.9360 0.8889 0.8822
F1 0.2594 0.2440 0.2303 0.2031 0.1861 F1 0.8325 0.8289 0.8263 0.8030 0.8043

Next, we use the manual concepts of the top 50 Web pages for
Table 7
training a KEA model. Again, the testing results are the same
Evaluation results of ICE and applying CSS.
(P = 0.7383, R = 0.8300, F1 = 0.7815, and B = 6) when a threshold
of 0.2, 0.3, 0.4, and 0.5 is separately used. We observe that using T 0.5 0.6 0.7 0.8 0.9
C [3, 0.3] [3, 0.3] [3, 0.3] [3, 0.3] [3, 0.3]
manual concepts for training KEA greatly improves the quality of
P 0.6880 0.6930 0.6897 0.6925 0.6992
concept extraction by 8%. However, this model prefers longer con- R 0.8767 0.8667 0.8517 0.8350 0.8283
cepts. It is observed that ICE outperforms KEA by 5% (see Table 6). F1 0.7710 0.7702 0.7622 0.7571 0.7583

5. E-commerce applications

We have discussed the concept extraction algorithms in Sec-
tion 3 and evaluation results in Section 4, respectively. To show-
case the practical use of concept extraction in e-commerce
context, we present two applications in this section, i.e. product
matching, and topic-based opinion mining. Since ICE proves to be
significantly better than ACE (see Section 4), we use only ICE and
KEA in these two applications.
5.1. Product matching
Product matching is the process of identifying if a shopping re-
lated Web page is about a certain product (or some products), e.g,
iPhone 4s. This can be useful for search indexing and relevance
ranking, as well as advertising and merchandizing. Due to the
scope of this paper, we focus on measuring how well ICE and
KEA concepts match to products for product review pages.
In our work, we use consumer reviews and professional reviews
of digital cameras as examples to investigate the product matching
problem. Consumer reviews and professional reviews are mostly
about a particular product, although they might mention other
similar products too. For example, a Canon digital camera review
page might mention similar products by Sony and Samsung. We
need the capability of detecting what this page is about instead
of mentioning using the power of concept extraction. On the other
hand, in professional review pages, very often multiple products
with the same brand are reviewed for comparison purpose. Other
page types such as Frequently Asked Questions (FAQs), product
specs, and forums are good candidates of learning the product
matching problem too. Different from the above five types is the
buying guides, which often focus on suggestions of a category
rather than particular products, such as electronics, sports gear,
and media. Thus, buying guides have less impact on the product
matching problem.
To this end, we manually label 100 consumer review pages from
epinions.com and 100 professional review pages from cnet.com,
respectively. They are all about digital cameras with various
brands and models. For each page, an exact product name, i.e.
brand and model, is given manually. We observe that many prod-
uct names have as many as four words, so we allow ICE to extract
concepts with up to four words, with a threshold of 0.5 on concept
score, and 50% significance weight on TF score and 50% on HTML
score. For KEA, we use the manual concepts in Section 4 to train
a model and also allow it to produce concepts with up to four
words. Both ICE and KEA produce a few concepts for each page
but we use only the top one as extracted product name.
We use accuracy to evaluate ICE and KEA with respect to its
ability to extract brand and model from product review pages.
We calculate accuracy for brand and model, respectively. For
example, if the top one ICE concept contains the manually given
brand for 70 times out of 100 consumer reviews, then the accuracy
of ICE in brand matching on consumer reviews will be 70%. The re-
sults are summarized in Table 9. We believe the accuracy achieved
by both methods is reasonable considering only the top one con-
cept is used for matching brand and model.
As we see in Table 9, ICE does better than KEA in extracting
brand and model from both professional and consumer review
pages. This is consistent with the experiment results presented in
Section 4. We also observe that both ICE and KEA perform much
better on consumer reviews than on professional reviews. This
may be explained by the fact that consumer reviewers mostly fo-
cus on the target product and hence it is easier for concept extrac-
tion methods to find the correct product, whereas professional
reviewers often compare the target product to other brands and
models in addition to reviewing the target product. The other
interesting finding is that both ICE and KEA perform much better
on model matching than on brand matching in both professional
reviews and consumer reviews. This indicates that the reviews in
our test mention similar brands more often than similar models
with the same brand and hence it is more difficult to correctly
match brands than models. However, we believe it is more impor-
tant to correctly find the product model than the brand. That is be-
cause the model name (or number) is pretty unique in most cases
and it can identify the product accurately (even without knowing
the brand).
5.2. Topic-based opinion mining
We have shown how ICE and KEA can be useful in an important
e-commerce application, i.e. product matching. In this subsection,
we present how concept extraction can be helpful in a text mining
problem, i.e. topic-based opinion mining. Opinion mining, also
known as sentiment analysis, is a research area which aims to
identify and extract sentiments and emotions from text documents
(Liu 2012). Opinion mining has been very popular in the recent
decade and it has been used in many domains such as marketing,
customer analysis and political campaign. For example, business
and organizations want to understand customers’ voice and im-
prove their products and consumers want to compare products
from multiple vendors based on product reviews before they make
a purchase decision. One of the main sub-areas in opinion mining
is topic-based opinion mining, where the document collection is
large or has multiple topics. In such a case, one may wish to first
identify the essential topics within the corpus and then apply opin-
ion mining separately on each essential topic. This technique has
 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296
Table 8
Evaluation results of KEA using ICE concepts and manual concepts for training.
Training ICE Concepts Manual Concpets
T 0.2, 0.3, 0.4, 0.5 0.2, 0.3, 0.4, 0.5
B 3 6 concepts of discussion boards to our product managers, who are
P 0.6583 0.7383
R 0.7600 0.8300
F1 0.7055 0.7815
Table 9
Accuracy of ICE and KEA in product matching.
Accuracy BrandICE BrandKEA ModelICE
Professional reviews 0.65 0.60 0.71
Consumer reviews 0.73 0.69 0.80
been well studied and applied to attribute-based opinion mining
on product reviews and others (Liu 2012, Moghaddam and Ester
2012).
In the e-commerce context, a topic-based opinion mining sys-
tem can be very useful. We describe such a system in Zhang
et al. (2011), which first mines essential topics from discussion
boards on eBay forums10 and then mines opinions for each topic.
Such a system can quickly judge the community’s feedback and sen-
timent on our inventory, products and services, and many other
interesting things. It is also capable of detecting the trends of various
topics. Thus in this work, we focus on the first stage of the topic-
based opinion mining task, i.e. topic extraction. For each discussion
board on eBay forums, we gather threads that have ten or more posts
and index all the posts. We mine concepts from a set of posts at a
certain level, which can be the entire board, a certain thread in the
board, or a set of posts related to a certain topic, e.g. fish oil.
In this task, we use both ICE and KEA for topic extraction. We
notice that many posts from our users have few HTML tags so
we favor the TF score over the HTML score (k = 0.7). We train a
KEA model with manual concepts in Section 4 and allow it to pro-
duce concepts with up to four words for a post.
Both ICE and KEA are aimed towards extracting concepts from
individual documents in a document collection rather than from
the whole collection. Hence in order to generate a concept list for
an entire document corpus (in this case multiple posts), the output
concepts from all posts should be combined properly. We aim to do
the following:
1. Unite the top 5 concepts from each post j to obtain a single
list. Each concept i has a weight wi,j.
2. Record fi, which is the number of posts in which concept i
appears. Let Wi be the overall weight of concept i in the
P
corpus and Ai be its average weight. So W i ¼ j W i;j , and
Ai = Wi/fi.
3. Now three features, i.e. Wi, Ai, and fi can be used to re-rank
the list in order to select the top concepts for the entire cor-
pus. Preliminary tests show that in terms of acceptable per-
centage (fraction of concepts with a score of 3 or above
where a 1-to-5 score is assigned to each concept by human
subjects), fi is the best feature.
For example, the top 5 ICE concepts for the Cook’s Nook board
are: lemon juice, sour cream, black pepper, chicken wings, and green
beans. As another example, the top 5 KEA concepts for search re-
sults of ‘‘fish oil’’ in the Health & Beauty board are: multi vitamin,
coconut oil, fish oil supplements, lower cholesterol, and omega fish oil.
10
http://forums.ebay.com.
295
In terms of evaluation, it is difficult to define the concepts for a
collection of posts even for humans so popular measures like pre-
cision, recall, and accuracy do not apply. However, we can ask hu-
man subjects to rate the concepts in extrinsic mode. We show
experts in given domains and are continuously monitoring the for-
ums, and ask them to rate the concepts with a 1-to-5 score with 1
being poor and 5 being excellent. Related research in Turney
(2003); Zhang et al. (2007) defines acceptable concepts as those
that are rated excellent, good, or fair by human subjects. In our
work, acceptable concepts are those that receive a score of 3, 4,
or 5. These concepts are reasonably related to the target discussion
ModelKEA
board. In other words, they correctly and completely reveal the
main contents of the discussion board. The percentage, p, is then
0.67
0.77
formally defined as:
n3 þ n4 þ n5
p¼ P5 : ð3Þ
i¼1 ni
The average acceptable percentage of aggregated ICE concepts
for each discussion board is 81.3% and 82.0% for aggregated KEA
concepts. It is expected that ICE is only on par with or even worse
than KEA in this task (instead of outperforming KEA) because we
notice many forum posts have only a few HTML tags which may
limit ICE’s performance on the topic extraction task. In other
words, if there are more HTML tags in the forum posts, ICE will
do even better.
6. Conclusion and future work
We investigate two concept extraction methods ACE and KEA in
the e-commerce context. We discuss how to upgrade ACE with ma-
jor improvements into ICE. We construct a gold standard to evaluate
the methods using precision and recall. The evaluation results dem-
onstrate that ICE significantly outperforms ACE and that it also
outperforms the state-of-the-art method KEA. We also showcase
two important applications of concept extraction in e-commerce:
product matching and topic-based opinion mining. The main contri-
bution of our work is the application of concept extraction in the
e-commerce context. The major improvements we propose are
relevant to the shopping related Web pages and hence make ICE
an innovative approach to concept extraction in the e-commerce
applications.
Directions of future research include but are not limited to the
following: (1) perform experiments on much larger data sets; (2)
perform part-of-speech tagging and study linguistic filters (part-
of-speech patterns) in concept extraction; (3) investigate anchor
text in the concept extraction task.
Acknowledgments
We are thankful to the authors of the ACE and KEA methods for
sharing with us the source code.
References
Frank, E., Paynter, G., Witten, I., Gutwin, C., and Nevill-Manning, C. Domain-specific
keyphrase extraction. In: Proceedings of the Sixteenth International Joint
Conference on Artificial Intelligence, Stockholm, Sweden, 1999. 668–673.
Frantzi, K., Ananiadou, S., and Mima, H. Automatic recognition of multi-word terms:
the C-value/NC-value method. International Journal on Digital Libraries, 3, 2000,
115–130.
Gutwin, C., Paynter, G., Witten, I., Nevill-Manning, C., and Frank, E. Improving
browsing in digital libraries with keyphrase indexes. Journal of Decision Support
Systems, 27, 1999, 81–104.
Krulwich, B., and Burkey, C. Learning user information interests through the
extraction of semantically significant phrases. In: AAAI Spring Symposium
Technical Report SS-96-05: Machine Learning in Information Access, 1996. 110–
112.
296 Y. Zhang et al. / Electronic Commerce Research and Applications 12 (2013) 289–296
Liu, B. Sentiment Analysis and Opinion Mining (Introduction and Survey). Morgan &
Claypool Publishers, Cambridge, MA, USA, 2012.
McKeown, K., Passonneau, R., Elson, D., Nenkova, A., and Hirschberg, J. Do
summaries help? A task-based evaluation of multi-document summarization.
In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research
and Development in Information Retrieval, Salvador, Brazil, 2005. 210–217.
Moghaddam, S., and Ester, M. Tutorial: aspect-based opinion mining from product
reviews. In: 2012 Annual International ACM SIGIR Conference on Research and
Development in Information Retrieval, Portland, Oregan, 2012.
Parameswaran, A., Rajaraman, A., and Garcia-Molina, H. Towards the Web of
concepts: extracting concepts from large datasets. In: Proceedings of the 2010
VLDB Endowment, 2010. 566–577.
Piao, S., Forth, J., Gacitua, R., Whittle, J., and Wiggins, G. Evaluating tools for
automatic concept extraction: a case study from the musicology domain. In:
Proceedings of the Digital Economy All Hands Meeting - Digital Futures 2010
Conference, Nottingham, UK, 2010.
Ramirez, P., and Mattmann, C. ACE: improving search engines via automatic concept
extraction. In: Proceedings of the 2004 IEEE International Conference on
Information Reuse and Integration, Las Vegas, NV, USA, 2004. 229–234.
Song, M., Song, I., and Hu, X. KPSpotter: a flexible information gain-based keyphrase
extraction system. In: Proceedings of the Fifth ACM International Workshop on
Web Information and Data Management, 2003. 50–53.
Turney, P. Learning algorithms for keyphrase extraction. Information Retrieval, 2,
2000, 303–336.
Turney, P. Mining the Web for Lexical Knowledge to Improve Keyphrase Extraction:
Learning from Labeled and Unlabeled Data. Technical Report ERB-1096 (NRC-
44947). Institute for Information Technology, National Research Council of
Canada. Ottawa, ON, Canada, 2002.
Turney, P. Coherent keyphrase extraction via web mining. In: Proceedings of the
Eighteenth International Joint Conference on Artificial Intelligence, Acapulco,
Mexico, 2003. 434–439.
Villalon, J., and Calvo, R. A. Concept extraction from student essays, towards concept
map mining. In: Proceedings of the Ninth IEEE International Conference on
Advanced Learning Technologies, Washington, DC, USA, 2009. 221–225.
Witten, I., Paynter, G., Frank, E., Gutwin, C., and Nevill-Manning, C. KEA: practical
automatic keyphrase extraction. In: Proceedings of the Fourth ACM Conference on
Digital Libraries, Berkeley, CA, USA, 1999. 254–255.
Zhang, Y., Milios, E., and Zincir-Heywood, N. A Comparison of Keyword- and Keyterm-
Based Methods for Automatic Web Site Summarization. Technical Report CS-2004-
11. Faculty of Computer Science, Dalhousie University. Halifax, NS, Canada,
2004.
Zhang, Y., Zincir-Heywood, N., and Milios, E. Narrative text classification for
automatic key phrase extraction in web document corpora. In: Proceedings of
the Seventh ACM International Workshop on Web Information and Data
Management, Bremen, Germany, 2005. 51–58.
Zhang, Y., Milios, E., and Zincir-Heywood, N. A comparative study on key phrase
extraction methods in automatic web site summarization. Journal of Digital
Information Management, Special Issue on Web Information Retrieval, 5, 2007,
323–332.
Zhang, Y., Shen, D., and Baudin, C. Tutorial: sentiment analysis in practice. In: 2011
IEEE International Conference in Data Mining (ICDM’11), Vancouver, BC, Canada,
2011.