The different methods of factor analysis first extract a set a factors from a
data set. These factors are almost always orthogonal and are ordered according
to the proportion of the variance of the original data that these factors explain.
In general, only a (small) subset of factors is kept for further consideration and
the remaining factors are considered as either irrelevant or nonexistent (i.e.,
they are assumed to reflect measurement error or noise).

© All Rights Reserved

8 tayangan

The different methods of factor analysis first extract a set a factors from a
data set. These factors are almost always orthogonal and are ordered according
to the proportion of the variance of the original data that these factors explain.
In general, only a (small) subset of factors is kept for further consideration and
the remaining factors are considered as either irrelevant or nonexistent (i.e.,
they are assumed to reflect measurement error or noise).

© All Rights Reserved

- Advanced Statistics With Matlab
- Caret
- Service Quality Case Study VG
- Brand Experience
- Constructing Minimal Spanning Tree Based on Rough Set Theory for Gene Selection
- digest_Dimensionality Reduction of Neural spike train data using factor analysis
- Lebot11 NIRS Sweet Potato
- A Novel Approach for Enhancing the Face Recognition Accuracy using SIFT with Multiple LDA-PCA Techniques
- IJERD (www.ijerd.com) International Journal of Engineering Research and Development IJERD : hard copy of journal, Call for Papers 2012, publishing of journal, journal of science and technology, research paper publishing, where to publish research paper, journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, how to get a research paper published, publishing a paper, publishing of research paper, reserach and review articles, IJERD Journal, How to publish your research paper, publish research paper, open access engineering journal, Engineering journal, Mathemetics journal, Physics journal, Chemistry journal, Computer Engineering, how to submit your paper, peer review journal, indexed journal, reserach and review articles, engineering journal, www.ijerd.com, research journals, yahoo journals, bing journals, International Journal of Engineering Research and Development, google journals, journal of engineering, online Submis
- d-guha paper
- Etude sur la connaissance politique aux USA
- 3. Blocker, Flint, Myers, Slater - Proactive Customer Orientation and Its Role for Creating Customer Value in Global Markets
- anisimova2007
- Research Proposal 1
- Project PCA
- Spectral Relevance Coding in Mining for Genomic Application
- Age trends in musical preferences in adulthood: 1. Conceptualization and empirical investigation
- A Novel Technique for Human Face Recognition Using Nonlinear Curvelet Feature Subspace
- 36-114-1-PB
- Kush Agra

Anda di halaman 1dari 8

Hervé Abdi1

The University of Texas at Dallas

Introduction

The different methods of factor analysis first extract a set a factors from a

data set. These factors are almost always orthogonal and are ordered according

to the proportion of the variance of the original data that these factors explain.

In general, only a (small) subset of factors is kept for further consideration and

the remaining factors are considered as either irrelevant or nonexistent (i.e.,

they are assumed to reflect measurement error or noise).

In order to make the interpretation of the factors that are considered rel-

evant, the first selection step is generally followed by a rotation of the factors

that were retained. Two main types of rotation are used: orthogonal when the

new axes are also orthogonal to each other, and oblique when the new axes are

not required to be orthogonal to each other. Because the rotations are always

performed in a subspace (the so-called factor space), the new axes will always

explain less variance than the original factors (which are computed to be opti-

mal), but obviously the part of variance explained by the total subspace after

rotation is the same as it was before rotation (only the partition of the variance

has changed). Because the rotated axes are not defined according to a statistical

criterion, their raison d’être is to facilitate the interpretation.

In this article, I illustrate the rotation procedures using the loadings of vari-

ables analyzed with principal component analysis (the so-called R-mode), but

the methods described here are valid also for other types of analysis and when

analyzing the subjects’ scores (the so-called Q-mode).

Before proceeding further, it is important to stress that because the rotations

always take place in a subspace (i.e., the space of the retained factors), the choice

of this subspace strongly influences the result of the rotation. Therefore, in the

practice of rotation in factor analysis, it is strongly recommended to try several

sizes for the subspace of the retained factors in order to assess the robustness

of the interpretation of the rotation.

Notations

1 In: Lewis-Beck M., Bryman, A., Futing T. (Eds.) (2003). Encyclopedia of Social Sciences

Address correspondence to

Hervé Abdi

Program in Cognition and Neurosciences, MS: Gr.4.1,

The University of Texas at Dallas,

Richardson, TX 75083–0688, USA

E-mail: herve@utdallas.edu http://www.utdallas.edu/∼herve

1

To make the discussion more concrete, I will describe rotations within the

principal component analysis (pca) framework. Pca starts with a data matrix

denoted Y with I rows and J columns, where each row represents a unit (in gen-

eral subjects) described by J measurements which are almost always expressed

as Z-scores. The data matrix is then decomposed into scores (for the subjects)

or components and loadings for the variables (the loadings are, in general, cor-

relations between the original variables and the components extracted by the

analysis). Formally, pca is equivalent to the singular value decomposition of

the data matrix as:

Y = P∆QT (1)

with the constraints that QT Q = PT P = I (where I is the identity matrix), and

∆ being a diagonal matrix with the so-called singular values on the diagonal.

The orthogonal matrix Q is the matrix of loadings (or projections of the original

variables onto the components), where one row stands for one of the original

variables and one column for one of the new factors. In general, the subjects’

score matrix is obtained as F = P∆ (some authors use P as the scores).

Most of the rationale for rotating factors comes from Thurstone (1947) and

Cattell (1978) who defended its use because this procedure simplifies the factor

structure and therefore makes its interpretation easier and more reliable (i.e.,

easier to replicate with different data samples).

Thurstone suggested five criteria to identify a simple structure. According to

these criteria, still often reported in the literature, a matrix of loadings (where

the rows correspond to the original variables and the columns to the factors) is

simple if:

1. each row contains at least one zero;

2. for each column, there are at least as many zeros as there are columns

(i.e., number of factors kept);

3. for any pair of factors, there are some variables with zero loadings on one

factor and large loadings on the other factor;

4. for any pair of factors, there is a sizable proportion of zero loadings;

5. for any pair of factors, there is only a small number of large loadings.

Rotations of factors can (and used to) be done graphically, but are mostly

obtained analytically and necessitate to specify mathematically the notion of

simple structure in order to implement it with a computer program.

Orthogonal rotation

An orthogonal rotation is specified by a rotation matrix denoted R, where

the rows stand for the original factors and the columns for the new (rotated)

factors. At the intersection of row m and column n we have the cosine of the

2

angle between the original axis and the new one: rm,n = cos θm,n . For example

the rotation illustrated in Figure 1 will be characterized by the following matrix:

· ¸ · ¸

cos θ1,1 cos θ1,2 cos θ1,1 − sin θ1,1

R= = , (2)

cos θ2,1 cos θ2,2 sin θ1,1 cos θ1,1

with a value of θ1,1 = 15 degrees. A rotation matrix has the important property

of being orthonormal because it corresponds to a matrix of direction cosines and

therefore RT R = I.

x2

y2

θ 2,2

θ1,2 y1

θ 1,1

θ2,1 x1

an old axis m and a new axis n is denoted by θm,n .

Varimax

Varimax, which was developed by Kaiser (1958), is indubitably the most

popular rotation method by far. For varimax a simple solution means that each

factor has a small number of large loadings and a large number of zero (or small)

loadings. This simplifies the interpretation because, after a varimax rotation,

each original variable tends to be associated with one (or a small number) of

factors, and each factor represents only a small number of variables. In addition,

the factors can often be interpreted from the opposition of few variables with

positive loadings to few variables with negative loadings.

Formally varimax searches for a rotation (i.e., a linear combination) of

the original factors such that the variance of the loadings is maximized, which

amounts to maximizing

2 2

X¡

2

¢

V= qj,` − q̄j,` , (3)

3

Table 1: An (artificial) example for pca and rotation. Five wines are described

by seven variables.

For For

Hedonic meat dessert Price Sugar Alcohol Acidity

Wine 1 14 7 8 7 7 13 7

Wine 2 10 7 6 4 3 14 7

Wine 3 8 5 5 10 5 12 5

Wine 4 2 4 7 16 7 11 3

Wine 5 6 2 4 13 3 10 3

Table 2: Wine example: Original loadings of the seven variables on the first two

components.

For For

Hedonic meat dessert Price Sugar Alcohol Acidity

Factor 1 −0.3965 −0.4454 −0.2646 0.4160 −0.0485 −0.4385 −0.4547

Factor 2 0.1149 −0.1090 −0.5854 −0.3111 −0.7245 0.0555 0.0865

2 2

with qj,` being the squared loading of the jth variable on the ` factor, and q̄j,`

being the mean of the squared loadings (for computational stability, each of

the loadings matrix is generally scaled to length one prior to the minimization

procedure).

Other orthogonal rotations

There are several other methods for orthogonal rotation such as the quar-

timax rotation, which minimizes the number of factors needed to explain each

variable, and the equimax rotation which is a compromise between varimax

and quartimax. Other methods exist, but none approaches varimax in pop-

ularity.

An example of orthogonal varimax rotation

To illustrate the procedure for a varimax rotation, suppose that we have

5 wines described by the average rating of a set of experts on their hedonic

dimension, how much the wine goes with dessert, how much the wine goes with

meat; each wine is also described by its price, its sugar and alcohol content, and

its acidity. The data are given in Table 1.

A pca of this table extracts four factors (with eigenvalues of 4.7627, 1.8101,

0.3527, and 0.0744, respectively), and a 2 factor solution (corresponding to the

components with an eigenvalue larger than unity) explaining 94% of the variance

is kept for rotation. The loadings are given in Table 2.

The varimax rotation procedure applied to the table of loadings gives a

clockwise rotation of 15 degrees (corresponding to a cosine of .97). This gives the

new set of rotated factors shown in Table 3. The rotation procedure is illustrated

in Figures 2 to 4. In this example, the improvement in the simplicity of the

4

Table 3: Wine example: Loadings, after varimax rotation, of the seven vari-

ables on the first two components.

For For

Hedonic meat dessert Price Sugar Alcohol Acidity

Factor 1 −0.4125 −0.4057 −0.1147 0.4790 0.1286 −0.4389 −0.4620

Factor 2 0.0153 −0.2138 −0.6321 −0.2010 −0.7146 −0.0525 −0.0264

such a small data set is already very simple. The first dimension remains linked

to price and the second dimension now appears more clearly as the dimension

of sweetness.

x2

Hedonic

Acidity x1

Alcohol

For meat

Price

For dessert

Sugar

Figure 2: Pca: Original loadings of the seven variables for the wine example.

Oblique Rotations

In oblique rotations the new axes are free to take any position in the fac-

tor space, but the degree of correlation allowed among factors is, in general,

small because two highly correlated factors are better interpreted as only one

factor. Oblique rotations, therefore, relax the orthogonality constraint in order

to gain simplicity in the interpretation. They were strongly recommended by

Thurstone, but are used more rarely than their orthogonal counterparts.

5

x2 y2

Hedonic

Acidity x1

Alcohol

For meat

θ = 15 o

Price

y1

For dessert

Sugar

Figure 3: Pca: The loading of the seven variables for the wine example showing

the original axes and the new (rotated) axes derived from varimax.

seven variables on the first two components.

For For

Hedonic meat dessert Price Sugar Alcohol Acidity

Factor 1 −0.8783 −0.9433 −0.4653 0.9562 0.0271 −0.9583 −0.9988

Factor 2 −0.1559 −0.4756 −0.9393 −0.0768 −0.9507 −0.2628 −0.2360

For oblique rotations, the promax rotation has the advantage of being fast

and conceptually simple. Its name derives from procrustean rotation because

it tries to fit a target matrix which has a simple structure. It necessitates two

steps. The first step defines the target matrix, almost always obtained as the

result of a varimax rotation whose entries are raised to some power (typically

between 2 and 4) in order to “force” the structure of the loadings to become

bipolar. The second step is obtained by computing a least square fit from the

varimax solution to the target matrix.

Even though, in principle, the results of oblique rotations could be presented

graphically they are almost always interpreted by looking at the correlations

between the rotated axis and the original variables. These correlations are

interpreted as loadings.

For the wine example, using as target the varimax solution raised to the 4th

power, for a promax rotation gives the loadings shown in Table 4. From these

6

y2

Hedonic

Acidity

Alcohol y1

For dessert

Sugar

Figure 4: Pca: The loadings after varimax rotation of the seven variables for

the wine example.

loadings one can find that, the first dimension once again isolates the price

(positively correlated to the factor) and opposes it to the variables: alcohol,

acidity, “going with meat,” and hedonic component. The second axis shows

only a small correlation with the first one (r = −.01). It corresponds to the

“sweetness/going with dessert” factor. Once again, the interpretation is quite

similar to the original pca because the small size of the data set favors a simple

structure in the solution.

Oblique rotations are much less popular than their orthogonal counterparts,

but this trend may change as new techniques, such as independent component

analysis, are developed. This recent technique, originally created in the domain

of signal processing and neural networks, derives directly from the data

matrix an oblique solution that maximizes statistical independence.

*References

[1] Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component anal-

ysis. New York: Wiley.

[2] Kim, J.O. & Mueller, C.W. (1978). Factor analysis: statistical methods and

practical issues. Thousand Oaks (CA): Sage Publications.

[3] Kline, P. (1994). An easy guide to factor analysis. Thousand Oaks (CA):

Sage Publications.

7

[4] Mulaik, S.A. (1972). The foundations of factor analysis. New York: McGraw-

Hill.

[5] Nunnally, J.C. & Berstein, I.H. (1994). Psychometric theory (3rd Edition).

New York: McGraw-Hill.

[6] Reyment, R.A. & Jöreskog, K.G. (1993). Applied factor analysis in the nat-

ural sciences. Cambridge: Cambridge University Press.

[7] Rummel, R.J. (1970). Applied factor analysis. Evanston: Northwestern Uni-

versity Press.

- Advanced Statistics With MatlabDiunggah olehRohit Vishal Kumar
- CaretDiunggah olehSrijan Sharma
- Service Quality Case Study VGDiunggah olehapi-3854027
- Brand ExperienceDiunggah olehIrina Găgeanu
- Constructing Minimal Spanning Tree Based on Rough Set Theory for Gene SelectionDiunggah olehAdam Hansen
- digest_Dimensionality Reduction of Neural spike train data using factor analysisDiunggah olehJie Fu
- Lebot11 NIRS Sweet PotatoDiunggah olehjohneiver
- A Novel Approach for Enhancing the Face Recognition Accuracy using SIFT with Multiple LDA-PCA TechniquesDiunggah olehIJRASETPublications
- IJERD (www.ijerd.com) International Journal of Engineering Research and Development IJERD : hard copy of journal, Call for Papers 2012, publishing of journal, journal of science and technology, research paper publishing, where to publish research paper, journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, how to get a research paper published, publishing a paper, publishing of research paper, reserach and review articles, IJERD Journal, How to publish your research paper, publish research paper, open access engineering journal, Engineering journal, Mathemetics journal, Physics journal, Chemistry journal, Computer Engineering, how to submit your paper, peer review journal, indexed journal, reserach and review articles, engineering journal, www.ijerd.com, research journals, yahoo journals, bing journals, International Journal of Engineering Research and Development, google journals, journal of engineering, online SubmisDiunggah olehIJERD
- d-guha paperDiunggah olehKoushick Roy
- Etude sur la connaissance politique aux USADiunggah olehNicolas Gregoire
- 3. Blocker, Flint, Myers, Slater - Proactive Customer Orientation and Its Role for Creating Customer Value in Global MarketsDiunggah olehEllaMu
- anisimova2007Diunggah olehbinh
- Research Proposal 1Diunggah olehDivya Shukla
- Project PCADiunggah olehAbhay
- Spectral Relevance Coding in Mining for Genomic ApplicationDiunggah olehCS & IT
- Age trends in musical preferences in adulthood: 1. Conceptualization and empirical investigationDiunggah olehsaxycb
- A Novel Technique for Human Face Recognition Using Nonlinear Curvelet Feature SubspaceDiunggah olehBilal Saeed
- 36-114-1-PBDiunggah olehHectorDavidSuarezCarvajal
- Kush AgraDiunggah olehDeepali Nanda
- f33-cai.pdfDiunggah olehMJA
- Estimsting Water DemandDiunggah olehKiran Sharma
- Common Factors in International Bond Returns - Driessen, Melenberg, Nijman, Wp 2000Diunggah olehamerd
- Student Mathematics II Course 2016Diunggah olehBiswa Rocks
- Measuring Social Impact of Terrorism%5B2%5DDiunggah olehAbdul Rauf
- Tata Nano -- Hundal & Grover 2010 Consumer Behavior Towards Tata Nano -- Perceptual StudyDiunggah olehMegh Mehta
- Dard as 2011Diunggah olehKelly De Leon Tuliao
- mm-35Diunggah olehSomasekhar Chowdary Kakarala
- 04721823Diunggah olehRajesh Veerabadran
- Factor and item response theory analysis of the Protean and Boundaryless Career Attitude Scales.docxDiunggah olehVahmi Brian Owen D'sullivansevenfoldimerz

- Measuring Value Relevance in a (Possibly) Inefficient MarketDiunggah oleh1111sami
- UK Corporate Governance Code 2014Diunggah olehSimon Yossef
- The e!ect of international institutional factors on properties of accountingDiunggah oleh1111sami
- Ball 2006.pdfDiunggah oleh1111sami
- fair value disclosureDiunggah oleh1111sami
- Do Investors Perceive Marking-to-Model as Marking-to-Myth? Early Evidence from FAS 157 DisclosureDiunggah oleh1111sami

- 0620_s14_qp_53Diunggah olehBrianChan
- catalogo john deereDiunggah olehMichel Clayton
- Jupiter CycleDiunggah olehClement Li
- Gordon Solar System (1)Diunggah olehLali Tha
- Maths Year4 Paper 1 Final ExamDiunggah olehfildzahanny
- Dr Zulfiqar Ali Bhutto Dairy Development Presentation Final1Diunggah olehNazir Ahmed Jamali
- EX 3200 Data SheetDiunggah olehPankaj
- GPSA Control Valve SizingDiunggah olehsudharshanan
- Millikan's PaperDiunggah olehSzs315
- IS 341-1973Diunggah olehKarishma Prabhu
- Femtocell vs WifiDiunggah olehAri Eka Reza
- Lightbulbs_teachers - NotesDiunggah olehapi-3839240
- ko kp ka.pdfDiunggah olehFranklin Ronaldo Ginting
- Alfa Laval LOPX 710Diunggah olehIvan Maltsev
- 9A01503 Water Resources Engineering IDiunggah olehsivabharathamurthy
- Ma da yang acupunctureDiunggah olehKeefer Michael
- SONY PLAYSTATIONDiunggah olehThienHau
- Die CastingDiunggah olehthasarathanr1993_939
- Ford Ranger Bronco II TFI Ignition Diagnostics.pdfDiunggah olehMatthester
- Group of Soviet Forces GermanyDiunggah olehBatteriefuhrer
- MAX4465-MAX4469Diunggah olehLam Bhan
- MAITRIDiunggah olehAS Vatsal
- ProfileDiunggah olehgaganesan1972
- Part F - Particular Spec Civil (Astana-Sejingkat Substation)Diunggah olehAnonymous UUw70xirbl
- Working With In Commensurable Knowledge Traditions - TurnbullDiunggah olehDiego Osorio Cáceres
- Design ApproachDiunggah olehbaoshi3382
- Pancreatic HormonesDiunggah olehUsman Ali Akbar
- ata 9Diunggah olehchinna_jet
- Process of Accreditation and Registration (1).pdfDiunggah olehAkshat Mittal
- Chapter 1 Quick Check AnswersDiunggah olehfucku