Cluster Analysis
20-1
Chapter Outline
1) Overview 2) Basic Concept
vi.
20-2
Chapter Outline
7) Summary
20-3
Cluster Analysis
Cluster analysis is a class of techniques used to classify objects or cases into relatively homogeneous groups called clusters. Objects in each cluster tend to be similar to each other and dissimilar to objects in the other clusters. Cluster analysis is also called classification analysis, or numerical taxonomy. Both cluster analysis and discriminant analysis are concerned with classification. However, discriminant analysis requires prior knowledge of the cluster or group membership for each object or case included, to develop the classification rule. In contrast, in cluster analysis there is no a priori information about the group or cluster membership for any of the objects. Groups or clusters are suggested by the data, not defined a priori.
20-4
Fig. 20.1
Variable 1
Variable 2
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
20-5
Fig. 20.2
Variable 1
Variable 2
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
20-6
Cluster centers. The cluster centers are the initial starting points in nonhierarchical clustering. Clusters are built around these centers, or seeds.
Cluster membership. Cluster membership indicates the cluster to which each object or case belongs.
20-7
20-8
20-9
Formulate the Problem Select a Distance Measure Select a Clustering Procedure Decide on the Number of Clusters Interpret and Profile Clusters Assess the Validity of Clustering
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
20-10
V1
6 2 7 4 1 6 5 7 2 3 1 5 2 4 6 3 4 3 4 2
V2
4 3 2 6 3 4 3 3 4 5 3 4 2 6 5 5 4 7 6 3
V3
7 1 6 4 2 6 6 7 3 3 2 5 1 4 4 4 7 2 3 2
V4
3 4 4 5 2 3 3 4 3 6 3 4 5 6 2 6 2 6 7 4
V5
2 5 1 3 6 3 3 1 6 4 5 2 4 4 1 4 2 4 2 7
V6
3 4 3 6 4 4 4 4 3 6 3 4 4 7 4 7 5 3 7 2
20-11
20-12
Nonhierarchical
Other Two-Step
Linkage Methods
Variance Methods
Centroid Methods
Sequential Threshold
Parallel Threshold
Optimizing Partitioning
Wards Method
Single Linkage Complete Linkage Average Linkage
20-14
20-15
20-16
Single Linkage
Minimum Distance
Cluster 1
Cluster 2
Complete Linkage
Maximum Distance
Cluster 1
Average Linkage
Cluster 2
Cluster 1
Average Distance
Cluster 2
20-17
Wards Procedure
Centroid Method
20-19
20-21
Label case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
20-24
20-25
20-26
Interpreting and profiling clusters involves examining the cluster centroids. The centroids enable us to describe each cluster by assigning it a name or label. It is often helpful to profile the clusters in terms of variables that were not used for clustering. These may include demographic, psychographic, product usage, media usage, or other variables.
20-27
Cluster Centroids
Table 20.3
V2
3.625 3.000
V3
6.000 1.833
V4
3.125 3.500
V5
1.750 5.500
V6
3.875 3.333
3.500
5.833
3.333
6.000
3.500
6.000
20-28
2. 3.
4.
5.
V1 V2 V3 V4 V5 V6
Iteration History
Change in Cluster Centers 1 2 3 2.154 2.102 2.550 0.000 0.000 0.000
Iteration 1 2
a. Convergence achieved due to no or small distance change. The maximum distance by which any center has changed is 0.000. The current iteration is 2. The minimum distance between initial centers is 7.746.
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
20-30
3
6 4 6 3 2 4
V1 V2 V3 V4 V5 V6
Valid Missing
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
Number of Clusters 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
AIC Change(a)
a The changes are from the previous number of clusters in the table. b The ratios of changes are relative to the change for the two cluster solution. c The ratios of distance measures are based on the current number of clusters against the previous number of clusters.
Copyright 2010 Pearson Education, Inc. publishing as Prentice Hall
20-34
Cluster Distribution
Cluster
1 2
N 6
% of Total 30.0%
6
8 20 20
30.0%
40.0% 100.0%
30.0%
40.0% 100.0% 100.0%
3
Combined Total
20-35
Cluster Profiles
Table 20.5, cont.
Fun Cluster 1 2 3 Combined Mean 1.67 3.50 5.75 3.85 Std. Deviation .516 .548 1.035 1.899
Bad for Budget Mean 3.00 5.83 3.63 4.10 Std. Deviation .632 .753 .916 1.410
Eating Out Mean 1.83 3.33 6.00 3.95 Std. Deviation .753 .816 1.069 2.012
Best Buys Mean 3.50 6.00 3.13 4.10 Std. Deviation 1.049 .632 .835 1.518
Don't Care Mean 5.50 3.50 1.88 3.45 Std. Deviation 1.049 .837 .835 1.761
Compare Prices Mean 3.33 6.00 3.88 4.35 Std. Deviation .816 1.549 .641 1.496
20-36
Clustering Variables
In this instance, the units used for analysis are the variables, and the distance measures are computed for all pairs of variables. Hierarchical clustering of variables can aid in the identification of unique variables, or variables that make a unique contribution to the data. Clustering can also be used to reduce the number of variables. Associated with each cluster is a linear combination of the variables in the cluster, called the cluster component. A large set of variables can often be replaced by the set of cluster components with little loss of information. However, a given number of cluster components does not generally explain as much variance as the same number of principal components.
20-37
SPSS Windows
Analyze>Classify>Hierarchical Cluster
Analyze>Classify>K-Means Cluster
Analyze>Classify>Two-Step Cluster
20-38
4. 5.
6.
20-40
3. Move Fun [v1], Bad for Budget [v2], Eating Out [v3], Best Buys [v4], Dont Care [v5], and Compare Prices [v6] into the CONTINUOUS VARIABLES box.
4. For DISTANCE MEASURE, select EUCLIDEAN.
20-41
Analyze>Classify>Cluster Analysis
20-42
SAS Learning Edition: Hierarchical Clustering 1. Select ANALYZE from the SAS Learning Edition menu bar. Select Multivariate>Cluster Analysis. Move V1-V6 to the Analysis variables task role. Click Cluster and select Wards minimum variance method under Cluster method. Click Results and select Simple summary statistics. Click Run.
2. 3. 4.
5. 6.
20-43
SAS Learning Edition Windows: K-Means Clustering 1. Select ANALYZE from the SAS Learning Edition menu bar. Select Multivariate>Cluster Analysis. Move V1-V6 to the Analysis variables task role. Click Cluster and select K-means algorithm as the cluster method and 3 for the Maximum number of clusters. Click Run.
2. 3. 4.
5.
20-44
20-45
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America.