524015

research-article2014
ASMXXX10.1177/1073191114524015AssessmentMorizot

Article
Assessment

Construct Validity of Adolescents

2014, Vol. 21(5) 580606
The Author(s) 2014
Reprints and permissions:
Self-Reported Big Five Personality Traits: sagepub.com/journalsPermissions.nav
DOI: 10.1177/1073191114524015

Importance of Conceptual Breadth and asm.sagepub.com

Initial Validation of a Short Measure

Julien Morizot1

Abstract
While there are a number of short personality trait measures that have been validated for use with adults, few are
specifically validated for use with adolescents. To trust such measures, it must be demonstrated that they have adequate
construct validity. According to the view of construct validity as a unifying form of validity requiring the integration of
different complementary sources of information, this article reports the evaluation of content, factor, convergent, and
criterion validities as well as reliability of adolescents self-reported personality traits. Moreover, this study sought to
address an inherent potential limitation of short personality trait measures, namely their limited conceptual breadth. In
this study, starting with items from a known measure, after the language-level was adjusted for use with adolescents, items
tapping fundamental primary traits were added to determine the impact of added conceptual breadth on the psychometric
properties of the scales. The resulting new measure was named the Big Five Personality Trait Short Questionnaire
(BFPTSQ). A group of expert judges considered the items to have adequate content validity. Using data from a community
sample of early adolescents, the results confirmed the factor validity of the Big Five structure in adolescence as well as
its measurement invariance across genders. More important, the added items did improve the convergent and criterion
validities of the scales, but did not negatively affect their reliability. This study supports the construct validity of adolescents
self-reported personality traits and points to the importance of conceptual breadth in short personality measures.

Keywords
personality trait, Big Five, psychopathology, assessment, psychometric properties, adolescence

Understanding personality trait development across the life
course has overarching theoretical and practical implica-
tions (Caspi & Shiner, 2006). Whereas most adults have
the cognitive capabilities to reflect on the various aspects
of their self and report their thoughts, values, motivations,
feelings, and behaviors, is it the same for adolescents?
While there has been growing interest in the validity and
reliability of youths self-reported personality traits (Soto,
John, Gosling, & Potter, 2008; Tackett, Slobodskaya, etal.,
2012), there are still few short assessment instruments spe-
cifically validated for this developmental period. To trust
these self-reported assessments by adolescents, it should
be demonstrated that they have adequate construct validity.
This article reports the evaluation of construct validity of
adolescents personality traits. According to the view of
construct validity as a unifying form of validity requiring
the integration of different complementary sources of
information (Messick, 1995; Simms & Watson, 2007),
content validity, factor validity, convergent validity, crite-
rion validity, and reliability were evaluated. This study also
sought to address an inherent potential limitation of short
personality trait measures, namely their limited conceptual
breadth.
A Developmental Perspective on Big
Five Personality Traits
There appears to be a current consensus that a taxonomy of
five broad or higher-order personality traits can help clas-
sify and account for the variation in most of the numerous
existing primary traits (which are also sometimes called
facets, subdomains, or lower-order traits; Goldberg, 1990,
1993; John, Neumann, & Soto, 2008; Markon, Krueger, &
Watson, 2005; McCrae & Costa, 1997). These Big Five
broad traits are Openness to Experience, Extraversion,
Agreeableness, Conscientiousness, and Neuroticism (or its
1
University of Montreal, Montreal, Quebec, Canada
Corresponding Author:
Julien Morizot, School of Psychoeducation, University of Montreal, C.P.
6128, succ. Centre-Ville, Montreal, Quebec, Canada, H3C 3J7.
Email: julien.morizot@umontreal.ca

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Morizot 581
positive pole, Emotional Stability). Openness represents
individual differences in intellectual curiosity, imagination,
appreciation of different ideas and artistic expressions, and
different social and political values; Extraversion reflects
individual differences in sociability, assertiveness, activity
level, appreciation of exciting activities, and the propensity
to express positive emotions; Agreeableness reveals indi-
vidual differences in prosociality, empathy, collaboration,
and helpfulness with others; Conscientiousness represents
individual differences in the propensity to be organized, to
plan things ahead, to control impulses, and to respect and
abide conventional social norms and rules; and Neuroticism
refers to individual differences in the propensity to experi-
ence negative emotions such as anxiety, fear, depressed
mood, and irritability and to have low self-worth.
The early evidence for the existence of the Big Five traits
originated mainly from factor analytic studies with adult
samples (Goldberg, 1990, 1993; John etal., 2008). Still,
there is a growing body of research supporting the notion
that the Big Five traits can be recognized and measured in
childhood and adolescence (Caspi & Shiner, 2006; Shiner,
2010). On one hand, research on temperament or personal-
ity traits in infancy and early childhood suggests that many
primary traits identifiable within each of the broad Big Five
traits in adults are not yet developed at this early age.
Indeed, most models of infant and early childhood
personality suggest that there are three rather than five
broad dimensions, namely Positive Emotionality/Surgency,
Negative Emotionality, and Effortful Control, which are
conceptually similar to Extraversion, Neuroticism, and
Conscientiousness, respectively (Rothbart & Bates, 2006;
Shiner, 2010). On the other hand, differing from these early
stages of life, there is cumulating evidence from different
areas of research suggesting that arguably all of the primary
traits identified in adults are also identifiable by early ado-
lescence, at least in an emerging form.
First, individual differences in the physiological, endo-
crinal, and neural systems hypothesized to underlie adult
personality traits (DeYoung & Gray, 2009) can be measured
by early adolescence, though various brain circuits related
to these systems actually change or mature during the
entire second decade of life (Spear, 2010). Second, helped
by their growing capacity to think abstractly, early adoles-
cents can create an abstraction of their self by combining
several of their attributes (Harter, 2006). Moreover, the self-
concept starts becoming more differentiated in early adoles-
cence, which means that early adolescents are more able
than children to describe their self-attributes differently
according to different social situations or roles (Harter,
2006). In the same vein, there is evidence that by early ado-
lescence, around age 12, autobiographical life narratives are
more globally coherent than at earlier ages (Habermas & de
Silveira, 2008). This suggests that a coherent sense of self is
manifested at least by early adolescence.
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Third, a number of studies using item-level factor
analyses of self-reports from instruments constructed for
adults tend to recover reasonably well the Big Five factor
structure with early adolescents samples (e.g., Allik,
Laidra, Realo, & Pullmann, 2004; McCrae etal., 2002;
W. D. Parker & Stumpf, 1998; Scholte, van Aken, & van
Lieshout, 1997; Soto etal., 2008). In general, these stud-
ies suggest that by the age of 10 to 12, there is systematic
and meaningful covariation between Big Five trait indica-
tors (items tapping different primary traits), even though
within-trait coherence (internal consistency) and between-
trait differentiation (low factor correlations) tend to
increase throughout the entire adolescence (Soto etal.,
2008). In addition to self-reports, studies using item-level
factor analyses of instruments not a priori designed to
measure the Big Five traits and using informant reports
such as parents and teachers tend to recover well the Big
Five trait structure in late childhood and early adoles-
cents samples (e.g., Goldberg, 2001; Tackett, Krueger,
Iacono, & McGue, 2008; Tackett, Slobodskaya, etal.,
2012).
Based on this evidence, it is reasonable to argue for the
construction of a developmentally sensitive personality trait
measure for adolescents assessing the same primary traits
that are typically assessed in adulthood. Of course, the
items of such instrument need to be either developmentally
neutral or sensitive to this developmental period (e.g., refer-
ring to school rather than work) and have a language level
appropriate for young adolescents.
Apart from the factor structure replicated across differ-
ent developmental periods, there is also evidence that Big
Five personality traits have adequate predictive validity in
childhood, adolescence, and adulthood. There is mounting
evidence that the Big Five traits are related, both concur-
rently and prospectively, to various positive and negative
consequential life outcomes in adulthood (Ozer & Benet-
Martnez, 2006), even beyond classical constructs such as
socioeconomic status and intelligence (Roberts, Kuncel,
Shiner, Caspi, & Goldberg, 2007). Clinical and develop-
mental psychopathology research in recent years has led to
mounting evidence that personality traits are meaningful
correlates or predictors of various psychopathologies
(Malouff, Thorsteinsson, & Schutte, 2005; Widiger &
Smith, 2008), notably mood and anxiety disorders (Klein,
Dyson, Kujawa, & Kotov, 2012; Klein, Kotov, & Bufferd,
2011; Kotov, Gamez, Schmidt, & Watson, 2010), antisocial
behavior (Jones, Miller, & Lynam, 2011; Miller & Lynam,
2001), and substance use and abuse (Ball, 2005; Terracciano,
Lockenhoff, Crum, Bienvenu, & Costa, 2008). Research on
children and adolescents also supports the validity of per-
sonality traits for predicting psychopathology (Caspi, 2000;
De Pauw & Mervielde, 2010; Krueger, Caspi, Moffitt,
Silva, & McGee, 1996; Tackett, 2006; Tackett, Martel, &
Kushner, 2012).
582 Assessment 21(5)
Short Personality Trait Measures and
Conceptual Breadth
Initial evidence for the existence of the Big Five originated
from early factor analytic studies of hundreds of personality
trait descriptors or adjectives (Goldberg, 1990, 1993; John
etal., 2008). Each of these five broad traits thus encom-
passes several primary personality traits. In other words, the
Big Five traits are constructs with large conceptual breadth
or bandwidth (Cattell, 1966). A number of integrative
instruments measuring these broad traits and some of their
corresponding primary traits have been developed (see
chapters in Boyle, Matthews, & Saklofske, 2008). However,
because users of personality measures are often faced with
limited testing time, particularly in research situations,
complete integrative personality trait measures are often
considered too lengthy. For example, in large community-
based longitudinal studies, in which personality traits are
only one of several constructs being assessed, using shorter
measures becomes practically inevitable. Not surprisingly,
short measures of the Big Five traits are currently quite
popular. A number of such measures exists, most of them
having satisfactory basic psychometric properties (e.g., Big
Five QuestionnaireChildren Version [BFQ-C]
Barbaranelli, Caprara, Rabasca, & Pastorelli, 2003; Mini-
International Personality Item Pool Big Five Measure
[Mini-IPIP]Donnellan, Oswald, Baird, & Lucas, 2006;
International Personality Item Pool Big Five Measure
[IPIP50]Goldberg, 1999; Ten-Item Personality Inventory
[TIPI]Gosling, Rentfrow, & Swann, 2003; Big Five
Inventory [BFI]John, Donahue, & Kentle, 1991; John
etal., 2008; NEO Five-Factor Inventory [NEO-FFI-3]
McCrae & Costa, 2010; Five Factor Model Rating Form
[FFMRF]Mullins-Sweatt, Jamerson, Samuel, Olson, &
Widiger, 2006; Big Five Inventory10 [BFI-10]
Rammstedt & John, 2007; Big Five Mini Markers
[BFMM]Saucier, 1994; Mini Modular Markers [3M40]
Saucier, 2002).
Various authors have reviewed issues pertaining to the
validity and usefulness of short measures in general (e.g.,
Silverstein, 1990; Smith, McCarthy, & Anderson, 2000),
and conceptual and statistical strategies have been proposed
to shorten long versions (Marsh, Ellis, Parada, Richards, &
Heubeck, 2005; Stanton, Sinar, Balzer, & Smith, 2002).
Most of these authors have pointed to some nontrivial short-
comings of short measures of conceptually complex or
broad constructs. One central shortcoming is the limited
conceptual breadth. Indeed, conceptually broad constructs
typically consist of several primary traits. In principle, for a
short measure to be considered a valid alternative to the
full-length one, each of its primary traits should be repre-
sented. A short measure of a broad construct has limited
conceptual breadth when some primary traits are propor-
tionally less represented, or worse, not represented at all
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
(Loevinger, 1957; Smith, Fischer, & Fister, 2003). In other
words, such a measure does not have adequate content
validity (Haynes, Richards, & Kubany, 1995).
As argued by different scholars, the main reason why
short personality measures have reduced conceptual breadth
has to do with the common ways items are retained (Marsh
etal., 2005; Saucier & Goldberg, 2002; Smith etal., 2000;
Stanton etal., 2002). These authors point to three common
empirical methods to develop short personality trait scales:
factor saturation (selecting items with the highest factor
loadings), factor discrimination (selecting univocal items
without cross-loadings), and internal consistency maximi-
zation (selecting items contributing the most to internal
consistency). These methods are useful, but they all tend to
lead to the same shortcoming: scales with limited concep-
tual breadth in which items tapping some fundamental pri-
mary personality traits are absent (see Saucier & Goldberg,
2002). However, it is critical to preserve an adequate con-
ceptual breadth in a personality measure for two related rea-
sons. First, when representative sampling of all the primary
traits embedded in a broad trait is not maintained, the mea-
sures content validity is compromised (Haynes etal., 1995;
Smith etal., 2000; Smith etal., 2003). When content valid-
ity is not adequate, it affects almost all other psychometrics
properties (Haynes etal., 1995). Second, preserving con-
ceptual breadth is important when personality trait mea-
sures are used to predict external criteria (i.e., criterion
validity). As pointed out by different scholars, all things
being equal, a conceptually broader scale will tend to be
related to a broader array of criteria (Loevinger, 1957;
Saucier & Goldberg, 2002). This is particularly important
for short Big Five measures because they are often used in
community and longitudinal studies to predict a multitude
of criterion variables. Of course, measures of conceptually
broad predictors generally tend to provide optimal predic-
tion of equally complex and heterogeneous criteria, whereas
measures of narrower predictors are most efficient in pre-
dicting specific criteria (Cronbach & Gleser, 1965;
Loevinger, 1957). As noted by Hogan and Roberts (1996),
psychologists and other social scientists generally study
conceptually complex or broad constructs or syndromes. In
this context, short measures of broad constructs like Big
Five personality traits preserving conceptual breadth are
important.
An examination of the content of most existing Big Five
short measures indicates that they suffer from a limited con-
ceptual breadth. Indeed, a number of theoretically impor-
tant primary traits are usually absent, which ipso facto
limits content validity, but may also impact other psycho-
metric properties. The point is not to argue that existing
short personality trait measures are inappropriate or not
useful, but merely to notice that most, if not all, of them
have limited conceptual breadth. Yet evidence is still lack-
ing regarding the impact of this limitation on other
Morizot 583
psychometric properties, such as convergent validity with
independent measures of related constructs or criterion
validity with consequential outcomes.
The Present Study
Rationale for Items Modification and Addition
In this study, the impact of adding conceptual breadth in a
short Big Five personality trait measure was evaluated.
While existing studies typically aimed at reducing the num-
ber of items, the reverse strategy was used in this study.
Indeed, starting with an existing measure, a number of
items tapping missing fundamental primary traits were
added to the scales.
The BFI (John etal., 1991; John etal., 2008) was selected
as an appropriate measure to provide an initial item pool
because of its adequate basic psychometric properties and,
more specifically, its satisfactory content validity as com-
pared with other noncommercial short personality trait
measures. Moreover, this measure contains short verbal
statement items, which are preferable to single-trait adjec-
tive items because the former are more likely to be easily
and unequivocally understood by early adolescents
(Barbaranelli etal., 2003). Also, single-trait adjectives are
often at a higher level of abstraction than short verbal state-
ments, making them potentially harder to translate precisely
in different languages (Saucier & Goldberg, 2002). The
items measuring the BFI traits posted on the public domain
International Personality Item Pool website were used as
the initial items pool (http://ipip.ori.org/newBFITable.htm).
The items are also available in John etal. (2008). As will be
explained below, because of all the modifications made to
the original item pool, the resulting new measure was
named the Big Five Personality Trait Short Questionnaire
(BFPTSQ), to avoid confusion with the original question-
naire. The BFPTSQ items are presented in the appendix.
Back translation from the original French version (with all
the modifications and additions described next) was used to
make this English version.
Because this study was conducted with French Canadian
early adolescents, it was important to ensure that all items
would be well understood by the participants. A number of
researchers have reported that adolescents have difficulty
reading or understanding many items in personality ques-
tionnaires (e.g., Allik etal., 2004; De Fruyt, Mervielde,
Hoekstra, & Rolland, 2000; McCrae, Costa, & Martin,
2005), arguably because these measures were constructed
and validated for use with adults. Therefore, the first step
was to translate and revise the items. Three researchers and
three graduate students familiar with personality theory (and
the Big Five model more specifically), experienced in psy-
chometric test construction, as well as in research with ado-
lescents first translated the items into French. Modifications
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
were made until a consensus was reached. Afterward, the
items were reviewed for language level. The criterion was
that adolescents starting high school (at ages 12 or 13 in the
province of Quebec) be able to understand the items. This
led to minor modifications to most of the items. A number of
items were slightly simplified, and for a few of them an
additional descriptor or stem connected by a comma was
added. Even if it is generally not recommended to have mul-
tiple stems in an item, the objective was merely to provide a
synonym or to elaborate.
Before adding new items, one Openness item from the
original item pool was deleted because it was judged less
relevant for adolescents and not central to the target con-
struct (prefer work that is routine). Moreover, an
Extraversion item that was judged equivocal was removed
(generates a lot of enthusiasm) and replaced with an item
tapping social dominance or leadership (17).
The second step was to add new items representing
important primary personality traits. To keep the measure
short, with a testing time of 10 to 15 minutes for adolescents
(5 to 10 minutes for adults), a maximum of 50 items overall,
with 10 items per scale, was decided on. The goal was not
to add items for all missing primary traits in the original
item pool, but to add only a few of the most fundamental.
Scholarly reviews on trait structure and empirical
research with adolescents guided the choice of which pri-
mary traits were candidates for adding items. For Openness,
the original BFI scale has good content coverage for the
primary traits of Intellectual Inquisitiveness, Creativity, and
Aesthetic and Artistic Appreciation. It was decided that
only one item tapping Openness to Cultural Diversity (31r)
would be added. This primary trait is part of some compre-
hensive Openness models (McCrae & Sutin, 2009). For
instance, it would be represented by the facet Openness to
Values in the NEO-PI-3 (McCrae & Costa, 2010). As dis-
cussed by McCrae and Sutin (2009), this social attitude can
be considered part of Openness because open persons are
generally lower in ethnocentrism and thus, more open to
different cultures and lifestyles. Assessing individual differ-
ences in this social attitude is important because it is associ-
ated with behavioral and emotional adjustment of both
adolescents and teachers attending multicultural high
schools in large metropolitan areas (Garca-Coll &
Magnuson, 1997). Among existing short Big Five mea-
sures, there are no items tapping Openness to Cultural
Diversity in the BFI, the IPIP50, the NEO-FFI-3, the
BFMM, or the 3M40.
For Extraversion, the original BFI scale has good content
coverage for the primary traits of Sociability, Expressiveness,
Assertiveness, and Activity. It was decided that an item tap-
ping Sensation Seeking (42) and one tapping Joyfulness or
Positive Emotions (47) would be added. These two primary
traits are part of various comprehensive models of Extraversion
(Wilt & Revelle, 2009). For instance, they would correspond
584 Assessment 21(5)
to Excitement Seeking and Positive Emotions in the NEO-PI-3
(McCrae & Costa, 2010). Sensation Seeking is certainly a
fundamental personality trait that has proven useful for
understanding the development of negative life outcomes
(Zuckerman, 2009). For instance, it is related to antisocial
behavior (Jones etal., 2011). Assessing individual differences
in Sensation Seeking in adolescents can be useful for predict-
ing substance use and to decide which adolescents should
participate in a preventive intervention (Sargant, Tanski,
Stoolmiller, & Hanewinkel, 2010). Joyfulness is also an
important primary trait, as some scholars consider it to be the
core aspect of Extraversion (Tellegen & Waller, 2008; Watson
& Clark, 1997). Assessing individual differences in Joyfulness
is important because it is related to various life outcomes,
such as attachment styles (Shiota, Keltner, & John, 2006),
stress recovery, and resilience (Ong, Bergeman, Bisconti, &
Wallace, 2006). Among existing short Big Five measures,
there are no items tapping Sensation Seeking in the BFI, the
BFQ-C, the IPIP50, the BFMM, or the 3M40, while there are
no items tapping Joyfulness in the BFI, the IPIP50, the
BFMM, or the 3M40.
For Agreeableness, the original BFI scale has good
content coverage for the primary traits of Altruism,
Compassion, Forgiveness, and Cooperation. It was
decided that an item tapping Machiavellianism (48r)
would be added. This constitutes a ubiquitous primary
personality trait in the human experience (Jones &
Paulhus, 2009; Wilson, Near, & Miller, 1996) and is argu-
ably a central construct for studying antisocial behavior
development and psychopathy (Barry, Kerig, Stellwagen,
& Barry, 2010). This primary trait is part of various com-
prehensive Agreeableness models and would be repre-
sented by the facet Straightforwardness and, to a lesser
extent, Altruism in the NEO-PI-3 (McCrae & Costa,
2010).1 Assessing individual differences in this construct
is important because it is clearly associated with serious
conduct problems and antisocial behavior in children,
adolescents, and adults (Frick etal., 2003; Jones etal.,
2011). Among existing short Big Five measures, there are
no items tapping Machiavellianism in the BFI, the BFQ-
C, the IPIP50, the BFMM, or the 3M40.
For Conscientiousness, the original BFI scale has
good content coverage for the primary traits of
Dutifulness, Self-Discipline, Planfulness, and Order. It
was decided that an item tapping Impulsiveness (49r)
would be added. Impulsiveness constitutes without a
doubt a fundamental primary personality trait (Madden
& Bickel, 2010) and is part of most comprehensive mod-
els of Conscientiousness (Roberts, Jackson, Fayard,
Edmonds, & Meints, 2008). In the NEO-PI-3, it would be
represented by the facet Deliberation (McCrae & Costa,
2010).2 Assessing individual differences in impulse con-
trol is important because it is related to executive func-
tions and plays a key role in various psychopathologies,
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
such as ADHD and antisocial behavior (Crews &
Boettiger, 2009; Jones etal., 2011; Newman & Wallace,
1993; Nigg, 2006). Among existing short Big Five mea-
sures, there are no items tapping Impulsiveness in the
BFI, the BFQ-C, the IPIP50, the NEO-FFI-3, the BFMM,
or the 3M40.
Finally, for Emotional Stability, the original BFI scale
has good content coverage for the primary traits of
Anxiousness, Fearfulness, Worry, and Depressed Mood. It
was decided that an item tapping Low Self-Worth (45r) and
another one tapping Irritability (50r) would be added. It is
clear that Self-Worth (or Self-Esteem) is a crucial trait of
the human experience (Bosson & Swann, 2009). Irritability
(or Anger) is also a central construct in the human personal-
ity (Barefoot & Boyle, 2009; Snaith & Taylor, 1985). Both
of these primary traits are part of various comprehensive
models of Neuroticism (Clark & Watson, 2008; Tellegen &
Waller, 2008). For instance, in the NEO-PI-3 (McCrae &
Costa, 2010), low Self-Worth would be represented by the
facet Vulnerability and, to a lesser extent, by Depression,
while Irritability would be represented by the facet Angry
Hostility. Assessing individual differences in Self-Worth is
important because it is related to psychopathology develop-
ment, particularly internalizing psychopathologies (Mann,
Hosman, Schaalma, & de Vries, 2004) and aggression and
antisocial behavior (Donnellan, Trzesniewski, Robins,
Moffitt, & Caspi, 2005). Assessing individual differences in
Irritability is also important because it is related to the
development of internalizing psychopathologies (Stringaris,
Cohen, Pine, & Leibenluft, 2009), ADHD (Leibenluft,
Cohen, Gorrindo, Brook, & Pine, 2006), reactive aggres-
sion (Scarpa & Raine, 1997), antisocial behavior (Jones
etal., 2011), and substance use (Tarter, Blackson, Brigham,
Moss, & Caprara, 1995). Among existing short Big Five
measures, there are no items tapping Irritability in the BFI
and the NEO-FFI-3, while there are no items tapping Low
Self-Worth in the BFI, the BFQ-C, the IPIP50, the BFMM,
or the 3M40.
Evaluation of Construct Validity and Impact of
Added Conceptual Breadth
To ensure that the modified items as well as the newly added
items were valid, a group of experts in personality trait the-
ory and test construction were first asked to rate the rele-
vance or representativeness of the items with regard to their
target Big Five trait. Factor validity was then assessed in two
ways. First, to ensure that all items were associated with
their target broad trait, factor analyses were conducted.
Second, given the well-documented gender differences in
personality trait means (Else-Quest, Hyde, Goldsmith, &
Van Hulle, 2006; Schmitt, Realo, Voracek, & Allik, 2008),
measurement invariance across genders was also evaluated.
Indeed, before testing mean differences, there is a need to
Morizot 585
test whether the constructs of interest are the same in both
groups, and few studies have fully tested measurement
invariance of personality traits across genders (Marsh etal.,
2010). Reliability was also estimated. Given the secondary
objective of this study, reliability estimates from the scales
with original content (i.e., without the new items) and those
with added items were compared statistically. It was hypoth-
esized that given their small number, the added items would
have no or marginal impact on reliability estimates. Indeed,
the conceptual breadth of the scales is increased, which
could reduce their reliability; but they also become longer,
which should slightly increase reliability. Convergent valid-
ity was also assessed using the NEO-PI-3 scales (McCrae &
Costa, 2010). This questionnaire can arguably be considered
a gold standard for convergent validity given its widespread
use in personality research. Again, to determine whether the
BFPTSQ scales with added items provided enhanced con-
vergent validity, the correlations derived from the original-
content scales and those derived from the added-item scales
were compared using statistical tests. It was hypothesized
that the scales with added items would have significantly
higher correlations with each of the broad Big Five traits, as
well as with primary traits from the NEO-PI-3 correspond-
ing to specific content of the newly added items. Finally,
criterion validity was assessed by correlating the personality
scales with consequential outcomes. Specifically, because
several studies showed that they are related to personality
traits, measures of externalizing and internalizing psychopa-
thology (e.g., De Pauw & Mervielde, 2010; Klein etal.,
2012; Tackett, 2006; Tackett, Martel etal., 2012; Widiger &
Smith, 2008) as well as academic achievement (e.g., Noftle
& Robins, 2007; Poropat, 2009) were used. Based on these
studies, it was hypothesized that the externalizing psychopa-
thology and substance use scales would be negatively related
to Agreeableness and Conscientiousness and positively
related to Extraversion. For internalizing psychopathology
scales, it was expected that they would be negatively related
to Emotional Stability and Extraversion. It was also hypoth-
esized that academic achievement would be negatively
related to Conscientiousness and Openness. Finally, because
of their larger conceptual breadth, it was hypothesized that
the added-item scales would provide significantly higher
correlations to each relevant outcome.
Method
Procedure and Participants
The data come from the first assessment of an ongoing pro-
spective longitudinal study on personality, antisocial behav-
ior, and psychopathology development during adolescence.
Eight French-language high schools from four different
school boards of the larger Montreal metropolitan and
Quebec City areas were initially targeted to participate
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
(Quebec, Canada). These schools were a priori targeted to
sample adolescents from various socioeconomic statuses
and ethnic backgrounds. After the project was presented to
all school administrations, one school declined to partici-
pate. In total, 41 classes participated in the first assessment:
29 regular classes, 8 international curriculum classes (i.e.,
enriched curriculum), and 4 specialized classes for special-
needs students (i.e., with behavior problems and/or learning
difficulties).
Before the data collection, the project was sanctioned by
the University of Montreals ethical review board as well as
all the school boards ethical committees. All the adolescents
signed a consent form, completed a form in which they pro-
vided their contact information (email, home address, etc.),
and were given an envelope to bring to their parents. The
envelop contained a letter for the parents explaining the
study, a consent form, a questionnaire about their child, a
questionnaire about themselves, and a preaddressed and
prepaid envelop to return all the material. All parents who
did not return the signed consent form were contacted by
phone. Only six parents refused to allow their child to par-
ticipate in the study.
For the data collection, two trained research assistants
visited each group between January and June 2009.
Because the initial questionnaire in this longitudinal study
was extensive, the adolescents were given two regular
class periods (75 minutes each) to fill it out. All those who
were absent from school the day of the assessment were
contacted by phone to set a new date for the data collec-
tion. Moreover, to minimize missing data, all adolescents
who were unable to fully complete the questionnaire dur-
ing the two periods were contacted and asked to return the
missing responses by email. Two $20 gift certificates were
randomly drawn in each group among the participating
adolescents.
The questionnaires of 1,036 adolescents were gathered
in the first assessment of this study. Personality data were
available for 1,028 of them (only a subsample of 598 filed
the NEO-PI-3). Their mean age was 12.71 (SD = 0.59), with
most aged 12 (35.5%) or 13 (59.1%) and only a few aged 14
(4.4%) or 15 (1.0%). The sample was equally proportioned
by gender (females, n = 520, 50.2%). Consistent with the
province of Quebec population, most of them were
Caucasian (74%), while Blacks (4.9%), Hispanics (3.3%),
Asians (3.2%), Arabs (7.4%), Canadian First Nations
(2.8%), and multiethnic persons (4.4%) made up the rest of
the sample. The large majority were born in Canada
(90.8%). Most of them lived with their two biological par-
ents (68.1%), while 16.2% lived in a shared custody situa-
tion with their biological mother and father, 5.0% with their
biological mother and her new partner, 1.7% with their bio-
logical father and his new partner, 6.1% with their biologi-
cal mother alone, 1.2% with their biological father alone,
and the rest with another family member or an adoptive
586 Assessment 21(5)
family or in foster care (1.7%). With regard to siblings,
65.5% (M = 0.98, SD = 0.98) of these adolescents lived with
at least one brother, while 61.6% lived with at least one sis-
ter (M = 0.90, SD = 0.93).
Measures
Personality
Big Five Personality Trait Short Questionnaire (BFPTSQ).Even
though the questionnaire resulting from this study is simi-
lar in content to the BFI developed by John and his col-
leagues (1991, 2008), it is a different measure. First, most
items were modified in two ways: First, the language was
simplified for use with early adolescents, and, second,
new items corresponding to important primary traits were
added. Because of these differences and to avoid confu-
sion with the original measure, the new one was given a
different name: the Big Five Personality Trait Short Ques-
tionnaire (BFPTSQ). The BFPTSQ has 50 items, 10 for
each trait. Respondents use a 5-point Likert-type response
format (totally disagree = 0, disagree a little = 1, neutral
opinion = 2, agree a little = 3, totally agree = 4). Just like
the original instrument, the introduction sentence, I see
myself as someone who, is presented at the top of each
page. The French and English versions of the BFPTSQ are
available from the author.
NEO-Personality Inventory-3 (NEO-PI-3).The NEO-
PI-3 (McCrae & Costa, 2010) assesses the broad and
primary personality traits of the five-factor model. The
broad traits are Openness, Extraversion, Agreeableness,
Conscientiousness, and Neuroticism. There is a total of
30 primary-trait scales, six for each of the broad trait.
The major difference between the NEO-PI-3 and its pre-
vious edition (NEO-PI-R) is that 37 items were replaced
or modified to be easier to read for adolescents, as well
as to be better indicators of their target primary trait
(McCrae etal., 2005). This inventory includes 240 items
(8 per primary trait) with a 5-point Likert-type response
scale (strongly disagree = 0 to strongly agree = 4). In
the present sample, internal consistency coefficients
are satisfactory for almost all scales, except for a few
primary-trait scales. For the broad traits, internal con-
sistency estimates are all high, with coefficients ranging
from .79 (Openness) to .92 (Conscientiousness). For the
primary traits, they range from .65 to .79, but fall below
.60 for one primary trait of Extraversion (Activity, .55),
one of Agreeableness (Compliance, .54), two of Neu-
roticism (Angry Hostility, .58; Impulsiveness, .56), and
three of Openness (Feelings, .56; Actions, .37; Values,
.43). Even though internal consistency is substandard
for some primary traits, particularly for Openness, other
researchers have observed the same phenomenon using
the same instrument with adolescents (e.g., De Fruyt
etal., 2000).
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Outcomes
Psychopathology. To measure psychopathology, seven
scales from the Youth Inventory Version 4 (Gadow &
Sprafkin, 1999) were used. This inventory is a self-report
measure of the most prevalent DSM-IV mental disorders
during adolescence. All items were written to correspond
directly to the symptoms of DSM-IV disorders. The ado-
lescents rated the last 12-month frequency of all items on
a 4-point scale (never = 0, sometimes = 1, often = 2, and
very often = 3). While the scales retained the DSM disor-
der labels, they can all be computed as either categorical
(i.e., a symptom count threshold allowing identification
of the presence of a disorder) or dimensional (i.e., a sum
of the items, which represents a frequency or severity
score). The dimensional scales were used in this study.
Three scales of externalizing psychopathology and four
scales of internalizing psychopathology were selected:
For externalizing the scales are Attention Deficit Hyper-
activity Disorder (ADHD, 18 items), Conduct Disor-
der (CD, 15 items), and Oppositional Defiant Disorder
(ODD, 8 items); and for internalizing, the scales are
Major Depression Disorder (MDD, 12 items), Bipolar
Disorder (BD, 10 items), Generalized Anxiety Disorder
(GAD, 8 items), and Social Phobia (SOP, 4 items). The
internal consistency of the scales is satisfactory in this
sample, with coefficients ranging from .78 to .91.
Substance Use. Substance use was assessed with modi-
fied items from the Measures of Quebec Adolescents Social
and Personal Adjustment (Le Blanc, 1996). This instrument
assesses several dimensions of adolescent adjustment and
was validated on a large representative sample of the Que-
bec adolescent population. Adolescents were asked whether
they had used eight substances during the past 12 months.
The items correspond to major psychoactive substance cat-
egories, namely (a) drank alcohol to get drunk (not just for
tasting or for a dinner), (b) smoked cannabis (marijuana,
pot, hash, weed), (c) inhaled substances such as glue, gas,
paint stripper, Dust-Off or other chemical substances, (d)
used stimulants such as Ecstasy, MDMA, XTC, etc., (e)
used other stimulants such as methamphetamines (meth),
cocaine, crack, or crystal meth (Ice), (f) used halluci-
nogens such as LSD, acids, PCP, mescaline, angel dust,
magic mushrooms, etc., (g) used tranquilizers such as
GHB, Valium, Librium, Ativan, Xanax, etc., and (h) used
analgesics such as heroin, morphine, opium, codeine, Oxy-
contin, Demerol, etc. All items were rated on a 4-point fre-
quency scale (never = 0, rarely (1 or 2 times) = 1, often = 2,
and very often = 3). Internal consistency is adequate in this
sample with a coefficient of .86.
Grade Point Average (GPA).The GPA scale was com-
puted using grades from official final report cards that were
transmitted by the schools. The courses included in the cur-
riculum are French (native language), English (secondary
language), sciences, mathematics, history, geography, arts,
Morizot 587
physical education, and ethics and religions. Scores can
vary from 0 to 100, with 60 as the passing grade, as per the
norms in effect in the Canadian province of Quebec. Inter-
nal consistency is .91 in this sample.
Statistical Analyses
All analyses were conducted using Mplus Version 6.12
(Muthn & Muthn, 2010). Unless otherwise noted, all
analyses were conducted using the robust maximum likeli-
hood estimator (MLR), which provides adjusted standard
errors and statistical fit tests that are robust to nonnormality
in the data. Confidence intervals (95%) were calculated and
reported to provide information regarding the precision of
all relevant parameter estimates (Cumming, 2012).
Content validity. To assess content validity, six expert
judges on both personality theory and construction of
personality measures were solicited to rate all items for
their relevance or representativeness with regard to their
target Big Five scale.3 The raters were asked to estimate
the relevance of each item using a 4-point response scale:
not at all representative = 0, somewhat representative
= 1, very representative = 2, extremely representative =
3. With these ratings, a Content Validity Index (Polit &
Beck, 2006) can be computed: one for individual items
(I-CVI) and one for scales (S-CVI). The I-CVI is com-
puted by adding up the number of raters who gave a rating
of 2 or 3 and dividing the result by the number of raters.
When there are six raters or more, an item is considered
relevant if the I-CVI is .78 or higher (Polit & Beck, 2006).
The S-CVI is computed by adding up the I-CVI values
of the items forming a scale and dividing the result by
the number of raters. When there are six raters or more,
a scale is considered to have adequate content validity
if the S-CVI is at least .80, but preferably .90 or higher
(Polit & Beck, 2006).
Factor validity. For all factor analyses presented here-
after, the items were treated as continuous. This was
justified by recent simulation studies which suggest that
Likert-type scales with five or more response categories
can be treated as continuous without resulting in biased
parameters (Beauducel & Herzberg, 2006; Rhemtulla,
Brosseau-Liard, & Savalei, 2012). Moreover, following
Marsh etal. (2010), all factor models were estimated
with and without a priori correlated uniquenesses (CUs),
which are used to reflect the fact that some items relate
to the same primary trait (or subdomain), share similar
content (but reversed scoring), or share the same word.4
This was necessary because CUs are common in person-
ality trait measures and failure to include them can lead
to biased parameter estimates (Marsh etal., 2010; Marsh
& Hau, 1996). As noted by Marsh etal. (2010), using ex
post facto CUs should generally be avoided, but in the
case of Big Five personality trait measures, many CUs,
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
such as those included in this study, are theoretically or
conceptually defensible.
Factor validity was assessed using two types of models.
The a priori five-factor structure was first tested with inde-
pendent clusters model confirmatory factor analysis (ICM-
CFA). The initial ICM-CFA model (M1) was specified so
that answers to the 50 items would be explained by five
correlated factors, with each item having a nonzero loading
on its target factor and zero loadings on all other factors,
while item uniquenesses are uncorrelated. However, a
number of researchers have noted that a good-fitting ICM-
CFA model is rarely, if ever, obtained with personality trait
structures measured by several items (e.g., Church &
Burke, 1994; Marsh etal., 2010; McCrae, Zonderman,
Costa, Bond, & Paunonen, 1996; J. D. A. Parker, Bagby, &
Summerfeldt, 1993). One possible explanation is that in an
ICM-CFA model, a simple structure is specified (i.e., non-
zero loading on the target factor, and all cross-loadings
fixed to 0). Yet, the Big Five personality trait structure was
uncovered using exploratory factor analysis (EFA), which
estimates all possible factor loadings. In other words, an
EFA is closer to the true model. In consequence, the factor
validity was also tested using exploratory structural equa-
tion modeling (ESEM; Asparouhov & Muthn, 2009;
Morin, Marsh, & Nagengast, 2013). An ESEM model
closely resembles EFA, but with some important advan-
tages. For instance, in addition to allowing all loadings to
be estimated on all factors, it also (a) provides standard
errors for all parameters so that their statistical significance
can be tested, (b) allows correlated item uniqueness to be
included, (c) allows multiple-group measurement invari-
ance tests to be conducted, and (d) provides fit indices and
statistical tests routinely available in CFA. The a priori
ESEM five-factor model (M2) was tested so that all target
loadings and cross-loadings are estimated, pending mini-
mal constraints imposed on the unrotated solution for iden-
tification purposes (for technical details on identification,
see Asparouhov & Muthn, 2009; Morin etal., 2013). One
potential drawback of ESEM is rotational indeterminacy.
Indeed, a model could converge on different solutions
depending on the chosen rotation method (for technical
details on rotation issues, see Asparouhov & Muthn,
2009; Morin etal., 2013; Morin & Maano, 2011).
Consequently, different factor loading patterns can be
found, which influence the size of factor correlations, even
though models with different rotation methods have equal
implication for the covariance matrix, and thus provide the
same fit to the data. In this study, six rotations available in
Mplus were tested: Geomin (Mplus default), Geomin with
an epsilon value of .5 (a value recommended in previous
ESEM studies of Big Five measures; Marsh etal., 2010),
CF-varimax, CF-quartimax, CF-equamax, and finally tar-
get loading rotation (all loadings on the nontarget factors
are given a start value of 0).
588 Assessment 21(5)
Once the best factor model was identified, the measure-
ment invariance across genders of this model was evaluated
using a series of increasingly stringent multiple-group mod-
els (see Meredith, 1993; Morin & Maano, 2011; Vandenberg
& Lance, 2000): configural invariance (MG1; all loadings,
intercepts, and uniquenesses are freely estimated, with the
latent variances constrained to 1 and latent means con-
strained to 0), metric invariance (MG2; loadings constrained
to invariance, which allows free estimation of the factor
variances in one group), scalar invariance (MG3; intercepts
constrained to invariance, which allows free estimation of
the factor means in one group), strict invariance (MG4;
uniquenesses constrained to equality), CU invariance
(MG5), variance/covariance invariance (MG6; all of which
must be done simultaneously in ESEM), and latent means
invariance (MG7). For each model in this sequence, the
imposed constraints are additive and the preceding model
serves as a reference.
The assessment of model fit was based on various indi-
ces (West, Taylor, & Wu, 2012). The chi-square test was
estimated for all models. A nonsignificant chi-square sug-
gests a good fitting model. However, because this test is
known to be overly sensitive to increasing sample size, to
minor departure from multivariate normality and to minor
(substantively irrelevant) model misspecifications, addi-
tional fit indices were considered more explicitly: the com-
parative fit index (CFI), the TuckerLewis index (TLI), the
root mean square error of approximation (RMSEA) and its
90% confidence interval, and the standardized root mean
square residual (SRMR). Values of .95 or above for the CFI
and TLI, of .06 or below for the RMSEA, and of .08 or
below for the SRMR have been suggested as being indica-
tive of a good model fit (Hu & Bentler, 1999). However,
these criteria are often considered too restrictive for many
applications; therefore, values of .90 or above for the CFI
and TLI, of .08 or below for the RMSEA, and of .10 or
below for the SRMR suggest an acceptable fit of the model
(Bentler, 1990; Marsh, Hau, & Wen, 2004). For the RMSEA
90% CI, values below .05 for the lower bound and below
.08 for the upper bound suggest acceptable fit (MacCallum,
Browne, & Sugawara, 1996).
For the assessment of change in model fit in invariance
tests, the SatorraBentler scaled chi-square test (Satorra,
2000) was computed for all multiple-group models. This
test takes into account the scaling correction in MLR esti-
mation. However, because the chi-square test tends to be
overly sensitive to sample size and minor departure from
multivariate normality, researchers examine changes more
closely in other fit indices. Cheung and Rensvold (2002)
suggested using change in CFI, where values below .01
indicate that the invariance hypothesis should not be
rejected, values between .01 and .02 suggest the possibility
of non-invariance, and values above .02 support the rejec-
tion of the invariance hypothesis (Cheung & Rensvold,
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
2002; Vandenberg & Lance, 2000). Chen (2007) suggested
using changes in RMSEA, where values below .015 indi-
cate that the invariance hypothesis should not be rejected.
Despite all these suggested criteria, it is important to
note that since there are still very few applications of ESEM
testing complex factor structures with 50 items or more and
given that the adequacy of the aforementioned fit indices
and proposed cutoff scores have not been rigorously tested
with these models, their adequacy for assessing model fit in
ESEM still needs to be thoroughly evaluated (Marsh etal.,
2010; Morin etal., 2013). These proposed cutoff values
should thus be considered as useful, but rough guidelines in
an ESEM context.
Reliability. Reliability of the scales was estimated using
different statistical indices. First the traditional Cronbachs
alpha coefficient was computed. However, this coefficient
is recognized as a limited estimator of reliability because
it is adequate only if three conditions are met: the items
are strongly unidimensional, the items are essentially tau-
equivalent (i.e., they all have high loadings), and there
are no correlated errors or uniquenesses (Raykov, 1998).
Therefore, the latent variable model composite reliability,
denoted by Rho (), suggested by Raykov (1997, 2012)
was also computed. Composite reliability is essentially
the total amount of true score variance in relation to the
total scale score variance and thus corresponds to the tra-
ditional concept of reliability in classical test theory. The
latent variable approach to reliability tends to provide less
biased estimates than Cronbachs alpha (Raykov, 2012).
Moreover, given that failure to account for correlated item
uniquenesses can lead to systematic overestimation of
reliability, the conditional on CUs was also computed
(Raykov, 2001, 2012). Given their importance in deter-
mining estimate precision, 95% confidence intervals were
computed for all estimates. Finally, because the main
objective of this study was to assess whether scales with
added items provide different reliability estimates, the
method proposed by Raykov (2007b, 2012) to calculate
the difference between two composite reliability estimates
was used. Using the MODEL CONSTRAINT option in
Mplus, composite reliability was specified for both scales,
and a change parameter (Delta) was then computed along
with a formal statistical significance test. This method also
allows computation of a 95% confidence interval for the
delta parameter.
Convergent and criterion validities. For convergent valid-
ity, the scales were correlated with their corresponding
scales from the NEO-PI-3 (McCrae & Costa, 2010), while
for criterion validity, the scales were correlated with dif-
ferent consequential outcome scales: three scales of exter-
nalizing psychopathology, four scales of internalizing
psychopathology, one of substance use, and one of aca-
demic achievement. To determine whether the added-item
scales provide higher correlations than the original-content
Morizot 589
scales (i.e., improved convergent and criterion validity),
the structural equation modeling approach suggested by
Raykov (2007a, 2012) was used to compare the correla-
tions of scales with differing number of items. Using the
MODEL CONSTRAINT option in Mplus, both correlations
were specified and a correlation change parameter (Delta)
was then computed, along with a formal statistical signifi-
cance test. This method also allows computation of a 95%
confidence interval for the delta parameter.
Results
Content Validity
In general, the six experts ratings all point to adequate
content validity. No item received a rating of 0 (not rele-
vant). For individual items, 76% (38 out of 50) have a
perfect agreement with an I-CVI = 1. All the other items
have an I-CVI above the recommended criterion of .78,
except three items, which have an I-CVI of .666: Items 42
(Sensation Seeking), 45 (Low Self-Worth), and 48
(Machiavellianism). Nonetheless, four out of six raters
identified these three items, which are all newly added
ones, as very representative. Regarding content validity
of the scales, all received highly favorable ratings from the
experts, with all estimates above the recommended crite-
rion. The S-CVI is .966 for Conscientiousness, .949 for
Openness, Extraversion, and Emotional Stability, and .933
for Agreeableness.
Factor Validity
The goodness-of-fit statistics from the different factor ana-
lytic models are presented in Table 1. All indices suggest
that ICM-CFA clearly does not fit the data (M1). Adding a
priori CUs (M1b) significantly improved the fit, but it was
still a poor-fitting model. Fitting an ESEM model (M2)
largely improved fit over the ICM-CFA model as suggested
by the large 2, CFI, and RMSEA. After the results
from all the tested rotations had been examined, the target
loading rotation was selected because it provided a some-
what clearer factor loading pattern (i.e., the target loadings
tended to be slightly larger, while the cross-loadings tended
to be smaller than with other rotation methods) and the fac-
tor correlations were only slightly higher.5 The fit of this
model, however, remains unacceptable because the CFI and
TLI values were below the acceptable criterion. A model
adding a priori CUs (M2b) again significantly improved the
fit to the data. In contrast to the preceding models, this
ESEM with CUs shows fit indices all in the satisfactory
range, with CFI and TLI above .90, as well as RMSEA and
SRMR below .06.
Table 2 presents the standardized factor loadings from
the ESEM model with CUs (M2b). Most target loading
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
items are substantial and are clearly statistically related to
their expected factor. Still, among the target loadings, 10
out of 50 have a value below .30, though they are statisti-
cally related to their expected factor. The standard errors
tend to be small for all items, including those with low load-
ings. Examination of the confidence intervals suggests that
all target loadings are relevant as none includes a value of 0.
Apart from the 50 target loadings, there is a total of 105
statistically significant cross-loadings, with 58 clearly sig-
nificant at .001 and 47 of lesser significance at .05.
Examination of each factor shows that for Openness, the
anchor item (21) taps ingenuity and creativity. The new
Openness to Cultural Diversity item (31r), although it is
clearly statistically related to Openness, has a somewhat low
standardized loading. As expected, this item has a significant
cross-loading on Agreeableness. For Extraversion, the
anchor item (22r) taps talkativeness and liveliness. The new
Sensation Seeking (42) and Joyfulness (47) items are clearly
statistically related to Extraversion, but also tend to have
small loadings. As expected, Item 47 has a significant cross-
loading on Emotional Stability. For Agreeableness, the
anchor item (28r) taps warmth and kindness. The new
Machiavellianism item (48r) is strongly related to
Agreeableness and, in fact, has one of the highest loadings
on that factor. For Conscientiousness, the anchor item (4)
taps thoroughness or diligence. The new Impulsiveness item
(49r) is statistically related to Conscientiousness, but has a
somewhat low loading. Moreover, this is a complex item
because it has significant cross-loadings on all other factors,
particularly on Agreeableness. Finally, for Emotional
Stability, the anchor item (40r) taps nervousness. The new
Low Self-Worth (45r) and Irritability (50r) items are clearly
related to Emotional Stability and are in fact items with
some of the highest loadings on that factor. As expected,
Item 45r has a significant cross-loading on Extraversion, as
does Item 50r on Agreeableness.
Table 3 presents the latent factor correlations and their
95% confidence intervals from the ICM-CFA and ESEM
models. As expected, the factor correlations from ESEM
are substantially smaller than those from ICM-CFA. While
the average absolute factor correlation for ICM-CFA is .27
(SD = .17), it is .13 (SD = .08) for ESEM. For example, in
both models, the largest correlation is between Agreeableness
and Conscientious; it is .605 in ICM-CFA, but decreases to
.348 in ESEM. The second largest correlation in both mod-
els is between Openness and Extraversion; it is .447 in
ICM-CFA, but decreases to .269 in ESEM. Interestingly,
the lowest and only nonsignificant correlation in ICM-CFA
is between Openness and Emotional Stability, but it
becomes negative and significant in ESEM.
The goodness-of-fit statistics from the gender invariance
tests are presented in Table 1. Fitting the ESEM model with
all freely estimated parameters for males and females sepa-
rately provides acceptable fit to the data (MG1). Constraining
590 Assessment 21(5)

Table 1. Goodness-of-Fit Statistics From the Confirmatory Factor Analytic and Exploratory Structural Equation Models.

Model 2 (df) CFI TLI RMSEA 90% CI SRMR Ref S2 (df) CFI RMSEA
M1: ICM-CFA 5178.65* .680 .663 .058 [.056, .060] .081
(1165)
M1b: ICM-CFA with CUs 4285.04* .750 .732 .052 [.050, .053] .079 M1 782.43* +.070 .006
(1144) (21)
M2: ESEM 2604.68* .871 .839 .040 [.038, .042] .034 M1 2419.94* +.191 .018
(985) (180)
M2b: ESEM with CUs 1811.17* .932 .914 .029 [.027, .031] .028 M2 780.65* +.061 .011
(964) (21)
Gender invariance
MG1: Configural invariance 2895.64* .922 .901 .031 [.029, .034] .033
(1928)
MG2: invariant 3141.31* .921 .910 .030 [.028, .032] .042 MG1
255.03* .001 .001
(2153) (225)
MG3: , invariant 3431.88* .901 .890 .033 [.031, .035] .045 MG2 301.80* .020 +.003
(2198) (45)
MG3b: , partially invariant 3307.64* .911 .900 .031 [.029, .034] .044 MG2 138.13* .010 +.001
(2194) (41)
MG4: , , invariant 3481.55* .901 .892 .033 [.031, .035] .051 MG3b 171.17* .010 +.002
(2244) (50)
MG5: , , , CUs invariant 3518.22* .899 .891 .033 [.031, .035] .051 MG4 35.77 .002 +.000
(2265) (21)
MG6: , , , CUs, / invariant 3555.29* .898 .890 .033 [.031, .035] .054 MG5 36.63 .001 +.000
(2280) (15)
MG7: , , , CUs, /, invariant 3757.23* .882 .873 .035 [.033, .037] .064 MG6 220.36* .016 +.002
(2285) (5)

Note. ICM-CFA = independent clusters model confirmatory factor analysis; ESEM = exploratory structural equation modeling; 2 = chi square;
df = degrees of freedom; CFI = comparative fit index; TLI = TuckerLewis index; RMSEA = root mean square error of approximation; 90% CI = 90%
confidence interval of the RMSEA; SRMR = standardized root mean square residual; Ref = reference model; S2 = SatorraBentler scaled chi-square
difference test; df = change in degrees of freedom; CFI = change in CFI; RMSEA = change in RMSEA; = factor loadings; = intercepts;
= uniquenesses; = factor variances; = factor covariances; = factor means.
*p < .01.

all factor loadings to equality did not significantly worsen
the fit (MG2). This result is noteworthy, because such a test
involves many more parameters (i.e., all target loadings and
cross-loadings are constrained to equality) than in ICM-
CFA. Constraining intercepts to equality across genders,
however, did seem to suggest the possibility of non-
invariance (MG3). Indeed, the scaled chi-square difference
test is significant and the change in CFI is at the upper limit
of the cutoff criterion suggesting the possibility of non-
invariance. Therefore, a model with partial invariance of
intercepts (MG3b) was estimated (Byrne, Shavelson, &
Muthn, 1989). Based on modification indices, four items
were freed across groups: females had higher item inter-
cepts for Items 26, 41, and 47 and a lower item intercept for
Item 40. This model provided a better fit than the model
with fully invariant intercepts. Constraining item unique-
nesses across genders did not lead to a significant worsen-
ing in model fit (MG4). Indeed, even though the scaled
chi-square test is significant, the change in CFI is not sig-
nificant and the change in RMSEA is again trivial. Another
model constraining the CUs to equality across genders did
not lead to a significant worsening of fit (MG5), suggesting
that these parameters are not gender-specific. Overall, even
if the fit indices tended to gradually decrease with the
increases in equality constraints (CFI and TLI), the RMSEA
remained in the acceptable range for all models, even con-
sidering its confidence intervals. All these results reason-
ably suggest that the BFPTSQ factor structure show full
measurement invariance across genders.
Constraining the variances/covariances across genders
did not lead to a significant worsening of fit (MG6). As
expected, however, constraining the latent factor means to
equality across genders did lead to a significant worsening
in fit (MG7). Examination of the modification indices and
the results of the preceding model (MG6) revealed a num-
ber of significant mean differences between males and
females. With the males means fixed to 0 and the variances
fixed to 1 in both groupswhich allows interpretation of
these differences in terms of traditional effect sizethe
females latent means were significantly lower in Emotional
Stability (.755, p < .0001), significantly higher in
Agreeableness (.578, p < .0001) and Conscientiousness

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
 Table 2. Standardized Factor Loadings From the Exploratory Structural Equation Model of the BFPTSQ Items.

Openness Extraversion Agreeableness Conscientiousness Emotional Stability

Item 95% CI 95% CI 95% CI 95% CI 95% CI

1 .504** [.373, .634] .219** [.146, .292] .133** [.068, .198] .047 [.129, .035] .051 [.009, .112] .640
6 .304** [.215, .393] .255** [.178, .332] .065 [.034, .163] .061 [.144, .022] .135** [.207, .064] .792
11 .471** [.365, .576] .074 [.154, .006] .044 [.114, .027] .242** [.155, .329] .089* [.027, .151] .687
16 .534** [.427, .640] .067 [.003, .137] .107* .038, .176] .156** [.235, .077] .005 [.054, .064] .711
21 .577** [.446, .708] .115* [.039, .191] .103* [.040, .166] .032 [.117, .053] .080* [.021, .140] .622
26 .306** [.205, .407] .049 [.030, .129] .137* [.060, .215] .063 [.020, .146] .105* [.179, .030] .850
31r .269** [.144, .388] .077 [.161, .008] .218** [.133, .303] .044 [.046, .134] .009 [.068, .087] .898
36 .408** [.325, .490] .091* [.160, .023] .041 [.120, .039] .196** [.122, .270] .063 [.002, .128] .778
41 .214** [.107, .321] .070 [.154, .014] .076 [.010, .162] .033 [.055, .122] .007 [.068, .083] .947
46 .363** [.271, .456] .131* [.206, .056] .066 [.014, .147] .011 [.072, .095] .003 [.072, .065] .871

2 .180** [.105, .255] .525** [.454, .595] .036 [.035, .107] .028 [.035, .091] .078* [.139, .016] .642
7r .239** [.307, .170] .642** [.581, .704] .005 [.059, .069] .011 [.068, .046] .192** [.124, .260] .521
12 .079 [.007, .164] .440** [.369, .512] .110* [.029, .192] .107* [.032, .181] .007 [.056, .071] .740
17 .240** [.167, .313] .468** [.401, .535] .142** [.211, .074] .101* [.036, .166] .093* [.034, .153] .601
22r .193** [.270, .115] .726** [.656, .796] .021 [.091, .050] .109* [.173, .044] .021 [.094, .053] .504
27 .209** [.138, .280] .448** [.382, .513] .167** [.232, .102] .228** [.166, .291] .191** [.130, .252] .539
32r .151* [.240, .062] .668** [.594, .742] .021 [.093, .050] .002 [.067, .064] .234** [.156, .313] .467
37 .053 [.035, .142] .628** [.553, .703] .144** [.066, .221] .011 [.057, .078] .043 [.094, .008] .576
42 .116* [.018, .215] .289** [.205, .373] .068 [.155, .019] .051 [.135, .033] .039 [.034, .112] .882
47 .104* [.001, .206] .296** [.214, .379] .149** [.061, .237] .091* [.175, .007] .226** [.300, .152] .846

3r .022 [.101, .057] .156** [.229, .083] .387** [.308, .466] .050 [.025, .126] .070* [.004, .136] .795
8 .201** [.116, .386] .053 [.018, .124] .264** [.164, .365] .180** [.094, .266] .064 [.137, .009] .770
13r .021 [.057, .098] .078* [.138, .017] .420** [.350, .489] .206** [.137, .274] .112* [.048, .175] .686

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
18 .228** [.144, .313] .083* [.157, .008] .223** [.120, .326] .053 [.140, .035] .051 [.029, .132] .873
23 .192** [.082, .302] .161** [.068, .253] .269** [.166, .373] .031 [.059, .120] .034 [.115, .048] .845
28r .078 [.180, .025] .114** [.050, .177] .550** [.478, .621] .047 [.116, .022] .110* [.046, .173] .665
33 .208** [.105, .312] .144** [.064, .224] .391** [.280, .502] .188** [.099, .278] .105* [.178, .033] .620
38r .004 [.125, .116] .153** [.222, .085] .542** [.466, .617] .167** [.097, .238] .088* [.018, .158] .570
43 .219** [.116, .322] .306** [.221, .392] .326** [.223, .429] .029 [.059, .117] .039 [.114, .037] .718
48r .060 [.143, .024] .143** [.205, .081] .504** [.435, .573] .045 [.027, .118] .044 [.018, .106] .690

(continued)

591
 592
Table 2.(continued)

Openness Extraversion Agreeableness Conscientiousness Emotional Stability

Item 95% CI 95% CI 95% CI 95% CI 95% CI

4 .057 [.030, .145] .018 [.071, .034] .009 [.057, .075] .716** [.649, .792] .089* [.143, .036] .474
9r .155** [.239, .070] .048 [.023, .118] .308** [.228, .387] .213** [.135, .291] .024 [.046, .095] .747
14 .045 [.042, .133] .052 [.003, .108] .031 [.095, .033] .681** [.611, .750] .088* [.142, .034] .536
19r .244** [.317, .171] .002 [.061, .065] .017 [.067, .102] .587** [.520, .655] .035 [.031, .101] .643
24r .123* [.211, .034] .062 [.008, .133] .194** [.115, .274] .416** [.337, .494] .032 [.040, .105] .728
29 .093* [.022, .163] .058 [.003, .113] .009 [.058, .077] .618** [.557, .679] .048 [.105, .009] .574
34 .100* [.021, .179] .074* [.019, .129] .089* [.152, .025] .713** [.654, .773] .029 [.084, .025] .472
39 .141* [.046, .237] .047 [.019, .112] .012 [.092, .069] .441** [.359, .522] .096* [.164, .028] .754
44r .096* [.187, .005] .088* [.158, .017] .005 [.082, .092] .472** [.393, .552] .236** [.164, .307] .692
49r .125* [.220, .030] .223** [.288, .158] .317** [.240, .395] .278** [.206, .351] .165** [.098, .232] .607

5r .195** [.276, .113] .217** [.147, .287] .152** [.073, .230] .164** [.090, .237] .259** [.183, .335] .756
10 .211** [.119, .303] .072 [.007, .150] .086* [.170, .003] .032 [.107, .043] .589** [.508, .669] .622
15r .001 [.063, .061] .053 [.003, .110] .011 [.070, .048] .118** [.174, .062] .678** [.623, .739] .538
20r .154** [.224, .084] .033 [.029, .095] .022 [.049, .094] .012 [.079, .056] .537** [.475, .598] .661
25 .121* [.030, .211] .024 [.099, .051] .125* [.207, .043] .020 [.059, .099] .468** [.394, .543] .782
30r .115* [.213, .017] .003 [.065, .072] .250** [.169, .331] .014 [.089, .061] .379** [.306, .452] .744
35 .260** [.164, .357] .030 [.103, .044] .093* [.180, .006] .003 [.070, .077] .614** [.535, .693] .608
40r .057 [.007, .120] .065* [.012, .118] .020 [.080, .040] .096* [.151, .041] .773** [.728, .819] .404

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
45r .081 [.162, .001] .210** [.144, .276] .035 [.113, .043] .144** [.072, .216] .459** [.387, .532] .670
50r .076 [.177, .025] .085* [.152, .017] .354** [.279, .430] .080* [.008, .151] .350** [.279, .421] .644

Note. Shaded entries are the target loading items. Item numbers with an r are reverse scored. = factor loadings; = uniquenesses; 95% CI = 95% confidence interval.
*p < .05. **p < .001.
Morizot 593

Table 3. Point and Interval Estimate of Factor Correlations of the BFPTSQ.

1. Openness 2. Extraversion 3. Agreeableness 4. Conscientiousness 5. Emotional Stability

95% CI 95% CI 95% CI 95% CI 95% CI

1. .269** [.211, .327] .032 [.092, .028] .227** [.130, .324] .119** [.181, .058]
2. .447** [.349, .545] .025 [.100, .050] .078* [.009, .146] .179** [.102, .256]
3. .259** [.144, .373] .059 [.064, .181] .348** [.279, .418] .141** [.081, .201]
4. .318** [.221, .416] .187** [.095, .278] .605** [.527, .682] .155** [.087, .223]
5. .041 [.045, .127] .367** [.281, .453] .235** [.141, .328] .137** [.048, .226]

Note. Latent factor correlations from the final exploratory structural equation model (ESEM) are presented above the diagonal, while latent
correlations from the independent clusters model confirmatory factor analysis (ICM-CFA) are presented below the diagonal. = factor covariance/
correlation; 95% CI = 95% confidence interval.
*p < .05. **p < .001.

Table 4. Point and Interval Reliability Estimates of the BFPTSQ Scales.

Original-Content Scale Added-Items Scale

95% CI 95% CI 95% CI

Openness
No CU .716 .712** [.685, .740] .712 .702** [.672, .732] .011* [.020, .002]
With CUs .648** [.613, .683] .646** [.609, .682] .002 [.013, .009]
Extraversion
No CU .819 .828** [.811, .846] .802 .813** [.795, .832] .015** [.022, .008]
With CUs .783** [.758, .808] .777** [.754, .801] .006 [.014, .003]
Agreeableness
No CU .701 .705** [.677, .734] .722 .723** [.694, .752] .018* [.007, .029]
With CUs .655** [.618, .693] .690** [.658, .723] .035** [.024, .047]
Conscientiousness
No CU .795 .796** [.777, .815] .796 .792** [.772, .812] .004 [.009, .002]
With CUs .778** [.756, .800] .776** [.753, .799] .002 [.008, .004]
Emotional Stability
No CU .792 .802** [.782, .821] .808 .812** [.793, .830] .010* [.002, .018]
With CUs .764** [.741, .788] .786** [.765, .808] .022 [.013, .031]

Note. = Cronbachs alpha internal consistency coefficient; = latent variable model composite reliability estimate; 95% CI = 95% confidence interval;
= delta parameter (composite reliability difference estimate); CU = correlated uniqueness.
*p < .05. **p < .001.

(.344, p < .001), and marginally higher in Openness (.158,
p < .10). There was no significant gender difference in
Extraversion.
Reliability
The point and interval estimates of reliability and the
comparison tests between reliability estimates () are
presented in Table 4. In general, all estimates suggest
that the BFPTSQ scales have adequate reliability. The
traditional and without CUs tend to be similar across
all scales. As expected, however, conditional on CUs
are all noticeably lower, decreasing below .70 for
Openness and Agreeableness. The estimates tend to be
somewhat higher for Extraversion and Emotional
Stability, while they tend to be the lowest for Openness
and Agreeableness.
Overall, the delta parameters () resulting from the
comparison tests between the original-content scales and
the added-item scales tend to suggest that both scales pro-
vide similar reliability estimates. This is supported by the
fact that the confidence intervals of all composite reliabil-
ity estimates are overlapping. This demonstrates that the
newly added items are not detrimental to scale reliability,
despite the fact that a number of them show low factor
loadings. In fact, when considering without CUs, the
results showed that the added items significantly decrease
reliability for Openness and Extraversion, while they sig-
nificantly increase reliability for Agreeableness and
Emotional Stability. However, when the conditional on

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
594 Assessment 21(5)

Table 5. Point and Interval Correlation Estimates Between BFPTSQ and NEO-PI-3 Scales.

Original-Content Scale Added-Items Scale

NEO-PI-3 Scales r 95% CI r 95% CI r 95% CI

Openness
Openness .591** [.540, .642] .629** [.583, .675] .038** [.020, .056]
Fantasy .377** [.308, .446] .385** [.315, .455] .008 [.014, .031]
Aesthetics .459** [.394, .524] .509** [.448, .570] .050** [.030, .071]
Feelings .317** [.244, .389] .331** [.260, .401] .014 [.010, .038]
Actions .186** [.108, .265] .171** [.092, .249] .016 [.039, .008]
Ideas .465** [.406, .524] .512** [.459, .566] .047** [.026, .068]
Values .196** [.116, .276] .204** [.128, .280] .008 [.015, .031]
Extraversion
Extraversion .650** [.602, .698] .694** [.650, .737] .043** [.030, .057]
Warmth .476** [.407, .545] .494** [.427, .562] .019* [.002, .036]
Gregariousness .453** [.387, .518] .477** [.413, .542] .025** [.010, .040]
Assertiveness .662** [.617, .708] .653** [.607, .699] .010 [.022, .003]
Activity .481** [.417, .544] .490** [.427, .553] .010 [.006, .025]
Excitement Seeking .347** [.282, .412] .435** [.375, .495] .088** [.072, .104]
Positive Emotions .340** [.262, .418] .390** [.314, .465] .050** [.032, .067]
Agreeableness
Agreeableness .612** [.565, .659] .671** [.628, .713] .059** [.047, .070]
Trust .472** [.408, .536] .477** [.413, .541] .005 [.010, .020]
Straightforwardness .432** [.373, .492] .532** [.479, .586] .100** [.086, .114]
Altruism .593** [.545, .640] .598** [.549, .646] .005 [.008, .018]
Compliance .436** [.376, .497] .471** [.410, .531] .034** [.020, .048]
Modesty .076 [.000, .162] .114* [.024, .204] .039** [.023, .054]
Tender-Mindedness .322** [.248, .396] .337** [.264, .409] .015* [.001, .030]
Conscientiousness
Conscientiousness .789** [.760, .818] .804** [.776, .832] .015** [.006, .024]
Competence .580** [.528, .632] .598** [.547, .649] .018* [.006, .029]
Order .616** [.568, .664] .616** [.568, .664] .000 [.011, .012]
Dutifulness .674** [.629, .719] .681** [.635, .726] .007 [.005, .018]
Achievement Striving .614** [.566, .662] .604** [.555, .652] .010 [.022, .001]
Self-Discipline .723** [.685, .761] .727** [.689, .764] .004 [.006, .014]
Deliberation .467** [.408, .527] .519** [.462, .576] .052** [.039, .064]
Emotional Stability
Neuroticisma .733** [.695, .771] .777** [.745, .809] .044** [.030, .058]
Anxiety .692** [.650, .734] .680** [.638, .722] .012 [.027, .004]
Angry Hostility .469** [.407, .532] .510** [.451, .569] .040** [.024, .057]
Depression .560** [.504, .616] .619** [.569, .670] .059** [.044, .075]
Self-Conscientiousness .486** [.427, .545] .534** [.480, .588] .048** [.032, .065]
Impulsiveness .285** [.205, .366] .319** [.242, .396] .034** [.015, .053]
Vulnerability .639** [.590, .687] .662** [.617, .706] .023* [.009, .037]

Note. In this table, each Big Five personality trait is correlated with its corresponding higher-order and primary-trait scales from the NEO-PI-3. 95%
CI = 95% confidence interval; r = delta parameter (correlation difference estimate).
a.Because Emotional Stability is simply the opposite pole, all correlations with Neuroticism and its primary traits from the NEO-PI-3 are actually
negative and are presented in absolute value to simplify the table.
*p < .05. **p < .001.

CUs are consideredwhich are arguably closer to the true
modelsall differences between reliability estimates are
nonsignificant, except for Agreeableness, which continues
to suggest that the added item increases reliability.
Convergent Validity
The point and interval correlation estimates between the
BFPTSQ and NEO-PI-3 scales as well as the comparison
tests between correlations (r) are presented in Table 5. The

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Morizot 595
overall pattern of correlations suggests adequate convergent
validity. All correlations between the broad-trait scales
are high, mostly around .70. For the scales with added
items, they range from .629 for Openness to .804 for
Conscientiousness. The correlations between the BFPTSQ
scales and the corresponding NEO-PI-3 primary-trait scales
are generally lower, but all are clearly significant. The only
exception is the correlation between BFPTSQ Agreeableness
and Modesty, which is quite low and even nonsignificant
for the original-content scale.
Overall, the delta parameters (r) resulting from the
comparison tests between the original-content scales and
those with added items suggest that the new scales signifi-
cantly improved convergent validity. Indeed, all correla-
tions between the broad Big Five trait scales are significantly
higher for the added-item scales. However, the confidence
intervals of all correlation pairs overlap somewhat. With
regard to the primary-trait scales, the expected results are
observed. Indeed, for Extraversion, the increase in correla-
tion is largest for Excitement Seeking and Positive
Emotions. For Agreeableness, the increase is largest for
Straightforwardness, while for Conscientiousness, it is larg-
est for Deliberation. For Emotional Stability, the increase in
correlation is largest for Depression, Self-Consciousness,
and Angry Hostility. In addition to these expected improved
correlations, several other correlations with primary-trait
scales also significantly increased, which is to be expected
since all primary traits have a common variance associated
with their corresponding broad Big Five trait. Again, the
confidence intervals of all correlation pairs overlap
somewhat.
Criterion Validity
The point and interval correlation estimates between the
BFPTSQ and outcome scales as well as the comparison
tests between correlations (r) are presented in Table 6. The
pattern of correlations between the Big Five and outcome
scales generally suggests adequate criterion validity.
Openness is related to lower scores in CD and to higher
scores in BD and GAD. As expected, Openness is most
strongly related to higher scores in GPA. Extraversion is
related to higher scores in ADHD, CD, BD, and substance
use, but to lower scores in MDD and SOP. As anticipated,
Extraversion is most strongly related to lower scores in SOP
and to higher scores in ADHD. Agreeableness is related to
lower scores on all psychopathology scales and substance
use, but to higher scores in GPA. As projected, Agreeableness
is most strongly related to the three externalizing psychopa-
thology scales, particularly CD and ODD. Conscientiousness
is also related to lower scores on all psychopathology scales
and substance use, but to higher scores in GPA. As hypoth-
esized, Conscientiousness is most strongly related to higher
scores in GPA and to lower scores in ADHD. Emotional
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Stability is related to lower scores on all psychopathology
scales, except CD and substance use. Emotional Stability is
not related to GPA. As expected, Emotional Stability is
most strongly related to the four internalizing psychopa-
thology scales, particularly MDD, GAD, and SOP.6
Overall, the delta parameters (r) resulting from the
comparison tests between the original-content scales and
those with added items suggest that that the new scales sig-
nificantly improved criterion validity. For Openness, the
added-item scale increases the negative relations with CD
(i.e., it becomes more negative), but reduces the positive
relation with BD. For Extraversion, the added-item scale
increases the positive correlations with ADHD, ODD, and
BD, but reduces the negative relations with MDD and SOP.
For Agreeableness, the added-item scale increases the nega-
tive relation with ADHD, CD, ODD, and BD, but does not
change the relations with internalizing psychopathology
scales or GPA. For Conscientiousness, the added-item scale
increases the negative relations with all psychopathology
scales, but does not change the relations with GPA. Finally,
for Emotional Stability, the added-item scale increases the
negative relations with ADHD, ODD, MDD, BD, and SOP.
However, the confidence intervals of all correlation pairs
overlap somewhat.
Discussion
The general objective of this study was to evaluate the con-
struct validity of adolescents self-reported personality
traits. In accordance with the view of construct validity as a
unifying form of validity requiring the integration of differ-
ent complementary sources of information (Messick, 1995;
Simms & Watson, 2007), content validity, factor validity,
convergent validity, criterion validity, and reliability were
evaluated. A secondary objective was to evaluate the poten-
tially beneficial impact of increasing the conceptual breadth
of scales from a short personality measure. Starting with an
item pool from an existing measure, the language level was
first adjusted for use with adolescents, and items tapping
fundamental primary personality traits that were missing
from this pool were added to each of the Big Five scales.
This led to a new measure which was named the Big Five
Personality Trait Short Questionnaire (BFPTSQ). Overall,
the results of this study supported the construct validity of
adolescents self-reported Big Five personality traits. The
results also support the idea that adding conceptual breadth
in a short personality trait measure can have a significant
positive impact on some of its psychometric properties.
Construct Validity of the BFPTSQ
For the evaluation of content validity, experts in personality
theory and questionnaire construction were asked to rate the
adequacy or representativeness of the items. All items were
596 Assessment 21(5)

Table 6. Point and Interval Estimates of Correlations Between BFPTSQ and Outcomes Scales.

Original-Content Scale Added-Items Scale

Outcome Scales r 95% CI r 95% CI r 95% CI

Openness
ADHD .057 [.006, .120] .000 [.064, .064] .057** [.075, .038]
CD .094* [.161, .027] .134** [.198, .071] .040** [.060, .020]
ODD .003 [.065, .059] .042 [.104, .020] .039** [.059, .019]
MDD .063* [.001, .127] .051 [.012, .114] .012 [.031, .007]
BD .090* [.027, .153] .071* [.007, .134] .019* [.038, .000]
GAD .093* [.029, .156] .095* [.032, .157] .002 [.018, .021]
SOP .030 [.095, .035] .016 [.080, .048] .014 [.006, .033]
SUBS .017 [.094, .061] .037 [.105, .031] .020 [.046, .006]
GPA .262** [.192, .333] .279** [.210, .348] .017 [.003, .037]
Extraversion
ADHD .178** [.111, .244] .219** [.155, .283] .041** [.026, .057]
CD .105** [.041, .168] .094* [.028, .160] .011 [.028, .007]
ODD .056 [.011, .123] .074* [.007, .141] .018* [.003, .033]
MDD .140** [.208, .072] .107** [.176, .038] .034** [.020, .048]
BD .153** [.087, .220] .194** [.129, .259] .040** [.027, .054]
GAD .088* [.157, .020] .057 [.126, .011] .031** [.017, .045]
SOP .341** [.399, .282] .324** [.383, .265] .016* [.002, .031]
SUBS .121** [.064, .179] .112** [.056, .168] .009 [.038, .020]
GPA .061 [.015, .137] .075* [.002, .153] .014 [.003, .032]
Agreeableness
ADHD .308** [.371, .244] .338** [.399, .277] .030** [.044, .017]
CD .393** [.458, .328] .429** [.490, .369] .036** [.052, .021]
ODD .442** [.497, .387] .480** [.532, .427] .037** [.050, .024]
MDD .200** [.261, .138] .207** [.270, .145] .007 [.020, .005]
BD .169** [.233, .106] .210** [.272, .147] .040** [.053, .027]
GAD .136** [.200, .072] .143** [.208, .079] .007 [.020, .006]
SOP .164** [.228, .101] .157** [.222, .093] .007 [.007, .020]
SUBS .124** [.197, .052] .140** [.214, .066] .016 [.003, .001]
GPA .323** [.255, .391] .314** [.246, .382] .009 [.024, .005]
Conscientiousness
ADHD .512** [.564, .461] .560** [.607, .512] .047** [.058, .037]
CD .361** [.425, .298] .401** [.460, .341] .039** [.052, .027]
ODD .384** [.442, .327] .439** [.492, .386] .055** [.066, .043]
MDD .264** [.324, .204] .297** [.356, .237] .033** [.044, .021]
BD .252** [.316, .188] .299** [.360, .237] .047** [.058, .035]
GAD .133** [.198, .068] .171** [.235, .106] .038** [.050, .027]
SOP .126** [.191, .061] .141** [.207, .076] .015* [.027, .004]
SUBS .155** [.249, .062] .178** [.268, .087] .022** [.035, .010]
GPA .404** [.340, .467] .401** [.338, .463] .003 [.015, .010]
Emotional Stability
ADHD .223** [.291, .154] .264** [.332, .196] .041** [.057, .026]
CD .001 [.069, .071] .036 [.107, .036] .037** [.051, .022]
ODD .282** [.346, .217] .327** [.390, .264] .045** [.061, .030]
MDD .527** [.576, .479] .574** [.619, .530] .047** [.060, .034]
BD .236** [.301, .171] .274** [.336, .211] .037** [.053, .022]
GAD .606** [.651, .560] .616** [.661, .570] .010 [.022, .002]
SOP .428** [.483, .373] .467** [.520, .414] .040** [.053, .026]
SUBS .013 [.065, .090] .010 [.087, .067] .023** [.036, .010]
GPA .036 [.039, .111] .069 [.005, .144] .033** [.016, .051]

Note. All psychopathology scales are dimensional scores representing the sum of the frequency for all items. ADHD = Attention Deficit Hyperactivity
Disorder; CD = Conduct Disorder; ODD = Oppositional Defiant Disorder; MDD = Major Depression Disorder; BP = Bipolar Disorder; GAD =
Generalized Anxiety Disorder; SOP = Social Phobia; SUBS = Substance Use; GPA = Grade Point Average; 95% CI = 95% confidence interval; r = delta
parameter (correlation difference estimate).

p < .10. *p < .05. **p < .001.

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Morizot 597
identified as valid indicators of their target trait according to
the usual criteria (Polit & Beck, 2006), which suggests
that the BFPTSQ scales have adequate content validity.
However, three items, all newly added ones, were identified
as somewhat less representative, namely Items 42 (Sensation
Seeking), 45 (Low Self-Worth), and 48 (Machiavellianism).
This suggests that even though most scholars would agree
that these items represent important personality traits, there
appears to be no clear consensus among experts as to their
place within the Big Five taxonomy.
The tests of the BFPTSQ factor validity revealed five
notable results. First, the Big Five factor structure was well
recovered in a sample of French Canadian adolescents.
There are still few published item-level factor analyses of
the Big Five structure with adolescents self-reports, but the
results of this study tend to replicate those from other stud-
ies with adolescent samples (e.g., Allik etal., 2004; McCrae
etal., 2002; W.D. Parker & Stumpf, 1998; Soto etal., 2008).
Second, all the newly added items were significantly asso-
ciated with their target Big Five trait. However, some of
these added items tended to show low factor loadings. This
is perhaps not so surprising because, in accordance with the
classic bandwidth-fidelity dilemma (Cronbach & Gleser,
1965), if there is an increase in the conceptual breadth of a
scale composed of repeated items measuring only a few pri-
mary traits, these new items will inevitably have somewhat
lower loadings. Even though factor loadings are commonly
considered meaningful when they exceed .30 or .40, this
popular rule of thumb is generally not recommended for
deciding whether an item is part of a factor. Indeed, as
pointed out by Preacher and MacCallum (2003), now that
some statistical programs calculate the standard error asso-
ciated with each factor loading, it is advisable to use statisti-
cal significance and confidence intervals. Also be taken into
account is the fact that including CUs in an ESEM model
typically results in lower loading estimates for some items.
In factor analyses of the NEO-FFI items, Marsh etal. (2010)
also observed that a number of items show standardized
loadings below the common recommendation of .30. Apart
from the low loadings, another characteristic of the factor
solution observed in this study is that there were a number
of complex items (i.e., items with significant loadings on
two or more factors). For instance, the Impulsiveness item
(48r) added to the Conscientiousness scale is one such item.
Even though it would be conceptually preferable to have no
such complex items (Donnellan etal., 2006), given that
Impulsiveness is such a fundamental personality trait,
researchers must deal with the trade-off between a full rep-
resentation of primary traits within a Big Five measure and
the increased complexity of interpreting scale scores with
complex items.
A third notable result from factor validity tests is that it
is difficultif even possibleto achieve good model fit to
the data by using ICM-CFA for a complex structure like the
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Big Five when it is measured by several items. Indeed, as
recently demonstrated by a number of researchers, it is
more appropriate to use ESEM to model all possible factor
loadings because it is closer than ICM-CFA to the true
model of the Big Five structure (e.g., Furnham, Guenole,
Levine, & Chamorro-Premuzic, 2013; Marsh etal., 2010;
Rosellini & Brown, 2011). In this study, out of a total of 250
possible factor loadings (number of items times the number
of factors), there was a total of 105 (42%) significant cross-
loadings. It should be noted, however, that the fit of the
ESEM model is acceptable, but far from excellent accord-
ing to typical criteria suggested for practical fit indices (i.e.,
Hu & Bentler, 1999). This is perhaps not surprising because
as Marsh, Hau, Balla, and Grayson (1998) noted, as the
number of indicators in a factor model increases, there tends
to be a decrease in fit, even for properly specified models.
A fourth interesting result from the factor analyses is that
factor correlations are considerably lower with ESEM than
with ICM-CFA. As shown by Marsh etal. (2010), when
researchers use ICM-CFA, thereby fixing to 0 the numerous
significant cross-loadings that are actually expected in a
Big Five personality trait structure, factor correlations are
vastly inflated because this is the way these cross-loadings
can be represented (see also Asparouhov & Muthn, 2009).
Using ESEM provides factor correlations that are probably
closer to the true population parameters and supports the
discriminant validity among the Big Five traits as measured
by the BFPTSQ.
A fifth interesting finding from the factor analyses is the
measurement invariance across genders. This is important
because to make valid factor mean comparisons, research-
ers must demonstrate at least that the loadings and inter-
cepts are invariant across groups (Millsap & Olivera-Aguilar,
2012). Unfortunately, these verifications are still rarely
done in personality research. The results of this study sug-
gest that it is reasonable to assume that the BFPTSQ factor
structure shows measurement invariance across genders.
That is, factor loadings, item uniquenesses, CUs, and factor
variances/covariances (or correlations) were all fairly
invariant across males and females. However, the item
intercepts were not fully invariant and four of them were
freed across groups. Two of these items were from Openness
and were significantly higher for females, and, interest-
ingly, both tap artistic interests (26 and 41r). This suggests
that females are significantly more disposed than males to
have artistic interests during adolescence. Another intercept
tapping joyfulness (47) was significantly higher for females,
which suggests they are more prone than males to experi-
ence positive emotions during adolescence. Finally, the
intercept of a reversed item tapping nervousness (40r) was
significantly lower for females, which suggests that females
are more predisposed than males to experience anxiety.
Non-invariant item intercepts suggest that the mean differ-
ences observed across groups in the corresponding factors
598 Assessment 21(5)
are not uniquely due to latent factor mean differences, but
also in part to differences in these invariant item intercepts
(Millsap & Olivera-Aguilar, 2012). However, even though
researchers would want all intercepts to be invariant across
groups, partial invariance is certainly acceptable for making
valid mean comparisons when there are only a few invariant
items (Byrne etal., 1989).
Backed up with this measurement invariance, the
results showed that there are sizable mean gender differ-
ences in adolescents personality traits. In order of mag-
nitude, females have significantly lower levels of
Emotional Stability, significantly higher levels of
Agreeableness and Conscientiousness, and marginally
higher levels of Openness. There was no significant gen-
der difference in Extraversion in this sample. These
results are consistent with those from a recent meta-anal-
ysis with adult samples, except for the absence of differ-
ences in Extraversion (Schmitt etal., 2008). However,
when gender differences in Extraversion are observed in
adolescents, they are typically rather small (Allik etal.,
2004). A study by Costa, Terracciano, and McCrae (2001)
could help in interpreting these results. These authors
observed that while mean gender differences tended to be
observed across all primary traits subsumed by
Neuroticism and Agreeableness, gender differences were
not consistent across primary traits for Openness and
Extraversion. For these two broad traits, some primary
traits favored women, while others favored men.
Differences within broad trait thus explained that mean
gender differences were rather small for Openness and
Extraversion. It is thus possible that the particular items
included in the BFPTSQ scales partly explain the absence
of mean difference in Extraversion.
With regard to the reliability of the BFPTSQ scales,
overall, the results suggest that the estimates are all accept-
able. Of importance, the comparison tests (i.e., delta param-
eters) between the original-content scales and the BFPTSQ
ones with added items revealed that the new items did not
improve reliability, nor did they have a detrimental impact.
Of course, only one or two items were added per scale, so
the absence of significant influence on reliability estimates
is not unexpected. Still, it is interesting that reliability esti-
mates were not significantly decreased, considering that
some of these new items tended to have somewhat low fac-
tor loadings. Another noteworthy observation concerning
reliability is that the latent variable model composite reli-
ability () estimates were systematically lower when mod-
els including CUs were considered. It is known that the
traditional Cronbachs alpha coefficient will be accurate
only when the items are clearly unidimensional and essen-
tially tau-equivalent and the uniquenesses are uncorrelated
(Raykov, 2001, 2012). The results of this study confirmed
that CUs that are not accounted for tend to inflate reliability
estimates of personality trait scales.
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Concerning convergent validity, overall, the correla-
tions with the NEO-PI-3 (McCrae & Costa, 2010)
scales suggest adequate validity of the BFPTSQ scales.
All the correlations between the broad-trait scales
are high, ranging from .629 for Openness to .804 for
Conscientiousness. The comparison tests confirmed
that, compared with those with original content,
the BFPTSQ scales with added items significantly
increased these correlations for all Big Five traits.
Moreover, the correlations between the BFPTSQ scales
and their target NEO-PI-3 primary-trait scales were
generally moderate to high, and all were significant.
Because nearly 43% (13 out of 30) of the correlations
with NEO-PI-3 primary-trait scales did not signifi-
cantly increase with the added items, this suggests that
the correlation increases are not merely a matter of
increased general variance. Interestingly, the compari-
son tests confirmed that the BFPTSQ scales with new
items showed significantly higher correlations with a
number of NEO-PI-3 primary-trait scales. Critically,
the comparison tests generally confirmed expectations
regarding the BFPTSQ scales as compared with those
with the original content: for Extraversion, the increase
in correlation was larger for Excitement Seeking and
Positive Emotions; for Agreeableness, it was for
Straightforwardness; for Conscientiousness, it was for
Deliberation (which could also be called Impulsiveness
based on its content); and for Emotional Stability, it
was for Depression and Self-Consciousness. Contrary
to expectations, however, the BFPTSQ Openness scale
with a new item tapping Openness to Cultural Diversity
did not significantly increase the relation with the
NEO-PI-3 Openness to Values scale, which is surpris-
ing since it contains similar items. It should be noted
that for comparison tests, when the confidence inter-
vals are considered, all overlap somewhat. Even though
nonoverlapping 95% confidence intervals are rather
stringent tests of the difference between two estimates
(Cumming, 2012), this suggest that the correlation
increase flagged as statistically significant as per the
delta parameters tends to be small.
As for criterion validity, overall the correlations with
the outcome measures suggest adequate concurrent
validity of the BFPTSQ scales. As expected based on
scholarly reviews (Klein etal., 2012; Tackett, Martel,
etal., 2012; Widiger & Smith, 2008) and meta-analytic
findings derived with adult samples (Malouff etal.,
2005), externalizing psychopathology scales tended
to be related to low levels of Agreeableness and
Conscientiousness and, to a lesser extent, to low levels
of Emotional Stability and high levels of Extraversion.
However, the BFPTSQ Extraversion scale did not show
a significantly higher correlation for substance use. A
higher correlation was expected because of the added
Morizot 599
Sensation Seeking item, which is typically related to
substance use (Sargant etal., 2010; Zuckerman, 2009).
Internalizing psychopathology scales, meanwhile,
tended to be related to low levels of Emotional Stability,
Agreeableness, and Conscientiousness and, to a lesser
extent, to low levels of Extraversion. An exception is
bipolar disorder, which was related to high levels of
Extraversion, even though it is considered a mood disor-
der. As expected based on Malouff etal.s (2005) meta-
analytic findings, Openness showed small but significant
relations to some psychopathology scales. Indeed, this
meta-analysis found that for self-reports, Openness is
positively related to psychopathology, while for observer
ratings, there is a negative relation. In the present study,
Openness was negatively related to CD and positively
related to MDD, BD and GAD. However, compared
with the other correlations, they tended to be small.
Moreover, as expected based on meta-analytic findings
(Poropat, 2009), Conscientiousness and Openness were
both positively related to GPA. In this sample of adoles-
cents, Agreeableness was also clearly related to GPA,
much more strongly than is typically observed in adult
samples (Noftle & Robins, 2007). The comparison tests
generally confirmed that, compared with those with the
original content, the BFPTSQ scales with added items
provide several significantly higher correlations with
outcome scales. Because 33% (15 out of 45) of the cor-
relations with outcome scales did not increase signifi-
cantly with the added items, this again suggests that the
correlation increases are not simply a matter of increased
general variance. When the 95% confidence intervals
are considered for comparison tests, all overlap some-
what, which suggests that the correlation increase
flagged as statistically significant as per the delta param-
eters tends to be small.
Impact of Added Conceptual Breadth
This study showed that by adding only a few items tapping
important primary personality traits, factor validity is ade-
quate, reliability is not affected, but convergent and crite-
rion validities are somewhat improved. Even if the BFPTSQ
is far from being optimal in terms of content coverage and
psychometric properties, the results of this study underscore
the significance of its added conceptual breadth.
Because of the limited time available in many assess-
ment situations, short personality measures are popular.
In principle, to be valid, short measures should have
items tapping all of the most important primary traits
subsumed in a given Big Five trait (Haynes etal., 1995;
Smith etal., 2003). Therefore, the minimal number of
items in a short scale should arguably be the number of
primary traits encompassed in a given Big Five trait. If
some items tapping fundamental primary traits do not
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
load strongly (but significantly) on their target factor,
they should not be excluded a priori for this reason. In
other words, results of factor or reliability analyses
should not supersede issues of conceptual breadth and
validity in questionnaire construction (Cizek, Rosenberg,
& Koos, 2008; Loevinger, 1957). This should rather be
taken as a challenge for personality researchers to find
the place of these primary personality traits in the com-
plex multivariate space of the Big Five taxonomy. This
challenge is difficult, however, because even though
personality psychologists have reached a reasonable
consensus on the hierarchical nature and number of
broad personality traits (Markon etal., 2005), they are
not even close to a consensus concerning the nature and
number of fundamental primary traits.
Limitations and Future Research
The BFPTSQ psychometric properties evaluated in this
study are generally satisfactory, but a few limitations
should be mentioned. First, all the psychometric proper-
ties of the BFPTSQ were evaluated for one sample of
adolescents. Even though efforts were made to gather a
large, very roughly representative sample of adolescents,
the results need to be replicated. Second, because study-
ing personality trait development is important, a cross-
validation should be carried out with an adult sample.
Third, the French-language version of the BFPTSQ was
validated, so it would be important to replicate these
results using the English- or other-language versions.
Fourth, it would be interesting to evaluate whether the
factor structure and measurement invariance across gen-
ders can be replicated with reports from informants, such
as parents and teachers. Fifth, there is a potential shared
method effect in the criterion validity tests because self-
report measures were used for the assessment of both
personality traits and outcomes. It would be important to
replicate these results using different informants or meth-
ods. Sixth, the evaluation of criterion validity was con-
ducted with a limited number of outcomes (i.e.,
psychopathology and achievement). Clearly, personality
traits have been shown to be associated with a wide range
of other positive and negative life outcomes, so addi-
tional predictive studies using these new scales are
needed. Seventh, this study provided only a preliminary
evaluation of incremental validity. Indeed, before mak-
ing any strong claims about the incremental validity of
the BFPTSQ, it would be important to demonstrate that
its scales significantly add to the prediction of conse-
quential outcomes beyond what can be predicted by
scales from other known short Big Five measures.
Despite these limitations, the results of this study suggest
that the BFPTSQ appears to be a potentially useful alter-
native to existing Big Five short measures.
600 Assessment 21(5)

Appendix
Big Five Personality Trait Short Questionnaire (BFPTSQ) Items
I see myself as someone who . . .
Openness
1 Is original, often has new ideas.
6 Is curious about many different things.
11 Is ingenious, reflects a lot.
16 Has a lot of imagination.
21 Is inventive, creative.
26 Likes artistic or aesthetic experiences.
31 R Is not really interested in different cultures, their customs and values.
36 Likes to reflect, tries to understand complex things.
41 R Has few artistic interests.
46 Is sophisticated when it comes to art, music or literature.
Extraversion
2 Likes to talk, expresses his/her opinion.
7 R Is reserved or shy, has difficulty approaching others.
12 Is full of energy, likes to always be active.
17 Is a leader, capable of convincing others.
22 R Is rather quiet, does not talk a lot.
27 Shows self-confidence, is able to assert himself/herself.
32 R Is timid, shy.
37 Is extraverted, sociable.
42 Likes exciting activities, which provide thrills.
47 Has a tendency to laugh and have fun easily.
Agreeableness
3 R Has a tendency to criticize others.
8 Is helpful and generous with others.
13 R Provokes quarrels or arguments with others.
18 Is lenient, forgives easily.
23 Generally trusts others.
28 R Can be distant and cold towards others.
33 Is considerate and kind to almost everyone.
38 R Can sometimes be rude or mean towards others.
43 Likes to cooperate with others.
48 R Can deceive and manipulate people to get what he/she want.
Conscientiousness
4 Works conscientiously, does the things he/she has to do well.
9 R Can be a little careless and negligent.
14 Is a reliable student/worker, who can be counted on.
19 R Has a tendency to be disorganized, messy.
24 R Has a tendency to be lazy.
29 Perseveres until the task at hand is completed.
34 Does things efficiently, works well and quickly.
39 Plans things that need to be done and follows through the plans.
44 R Is easily distracted, has difficulty remaining attentive.
49 R Can do things impulsively without thinking about the consequences.
Emotional Stability
5 R Has a tendency to be easily depressed, sad.
10 Is generally relaxed, handles stress well.
15 R Can be tense, stressed out.
20 R Worries a lot about many things.
25 Is emotionally stable, not easily upset.
30 R Can be moody.
35 Stays calm in tense or stressful situations.
40 R Can easily become nervous.
45 R Has a tendency to feel inferior to others.
50 R Has a tendency to be easily irritated.

Note. R = reversed-score item. Boldface item numbers represent newly added items.

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Morizot 601
Acknowledgments
I want to thank all the school board members, school principals, as
well as all the adolescents, parents, and teachers, who participated
in the study. Thank you also to Alexandre Morin and Dave
Miranda for their comments on an early draft of this article.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect
to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for
the research, authorship, and/or publication of this article: This
research was made possible by a grant from the Fonds qubcois
pour la recherche sur la socit et la culture (FQRSC).
Notes
1. From the perspective of the HEXACO model, the
Machiavellianism item would be associated with the sixth
factor, namely Honesty-Humility (Ashton, Lee, & Goldberg,
2004).
2. It should be noted that the Impulsiveness facet in the
NEO-PI-3 is associated with Neuroticism, but given its con-
tent this facet should arguably be called Compulsiveness. It
should also be noted that Impulsiveness can be considered
a multidimensional construct. In the model identified by
Whiteside and Lynam (2001), this added item would be con-
sidered part of the Lack of Premeditation subscale.
3. Brent Donnellan, Lewis Goldberg, Dave Miranda, Julie
Pozzebon, Brent Roberts, and Antonio Terracciano rated the
relevance of the items. These researchers were considered
experts because they worked and published on topics related
to personality trait structure and test construction and valida-
tion. Note that these researchers neither participated in nor
endorsed this research and the resulting new measure.
4. A total of 21 a priori correlated uniquenesses (CUs) were
posited: for Openness, 11-36, 16-21, 26-41r, 26-46, 1-16; for
Extraversion, 7r-32r, 2-22r, 12-42, 2-27; for Agreeableness,
18-23, 8-33, 23-33, 23-43; for Conscientiousness, 29-39, 19r-
24r, 19r-39, 29-44r; and for Emotional Stability, 10-35, 10-15r,
5r-25, 5r-45r. Most of these CUs are from a same primary trait
or subdomain and have similar content but in reversed scoring
or share the same word. Note that seven additional a priori
CUs were expected but were not statistically significant in the
sample used in this study, so they were not retained in the
models: for Openness, 1-21, 41r-46; for Extraversion, 17-27;
for Agreeableness, 18-43; for Conscientiousness 9r-19r, 4-14;
and for Emotional Stability, 30r-50r.
5. It should be noted that, compared with other rotation methods
that are fully exploratory, the Target Rotation can be thought
of as a more confirmatory method guided by theory where
small cross-loadings are expected even if the main factor
structure can be defined a priori.
6. Even though several moderate and significant correlations
between the Big Five traits and consequential outcomes
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
provide some support for the criterion validity of the
BFPTSQ scales, a potential problem is content overlap
between constructs. Indeed, there is a persistent concern
among scholars that the relations between personality and
psychopathology could be fallacious because some personal-
ity trait measures actually include items clearly tapping dis-
order-specific psychopathology (Lengua, West, & Sandler,
1998; Uliaszek etal., 2009). To ensure that the correlations
observed in this study are not solely explained by overlap-
ping items between constructs, trimmed personality scales
with potential overlapping items removed were computed
and correlations with some of the psychopathology scales
were reestimated. For Agreeableness, deleting Items 13 and
48 from the scale provided a significantly lower correlation
with CD, but it remained significant and of similar magnitude
(r = .076, SE = .011, p < .001, 95% CI = .097, .055).
For Conscientiousness, deleting Item 44 from the scale pro-
vided a significantly lower correlation with ADHD, but it
remained significant and of similar magnitude (r = .030,
SE = .005, p < .001, 95% CI = .039, .020). For Emotional
Stability, deleting Items 20 and 40 from the scale provided
a significantly lower correlation with GAD, but it remained
significant and of similar magnitude (r = .028, SE = .006,
p < .001, 95% CI = .040, .015). Finally, for Emotional
Stability, deleting Items 5 and 45 from the scale provided a
significantly lower correlation with MDD, but it remained
significant and of similar magnitude (r = .071, SE = .007,
p < .001, 95% CI = .086, .057). Overall, these results are
consistent with Lengua etal.s (1998) results and suggest that
there are robust and meaningful individual differences in per-
sonality traits that are related to adolescent psychopathology.
References
Allik, J., Laidra, K., Realo, A., & Pullmann, H. (2004). Personality
development from 12 to 18 years of age: Changes in mean lev-
els and structures of traits. European Journal of Personality,
18, 445-462.
Ashton, M. C., Lee, K., & Goldberg, L. R. (2004). A hierarchical
analysis of 1,710 English personality-descriptive adjectives.
Journal of Personality and Social Psychology, 87, 707-721.
Asparouhov, T., & Muthn, B. O. (2009). Exploratory structural
equation modeling. Structural Equation Modeling, 16, 397-
438.
Ball, S. A. (2005). Personality traits, problems, and disorders:
Clinical applications to substance use disorders. Journal of
Research in Personality, 39, 84-102.
Barbaranelli, C., Caprara, G. V., Rabasca, A., & Pastorelli, C.
(2003). A questionnaire for measuring the Big Five in late
childhood. Personality and Individual Differences, 34, 645-
664.
Barefoot, J. C., & Boyle, S. H. (2009). Hostility and proneness
to anger. In M. R. Leary & R. H. Hoyle (Eds.), Handbook of
individual differences in social behavior (pp. 210-226). New
York, NY: Guilford.
Barry, C. T., Kerig, P. K., Stellwagen, K. K., & Barry, T.
(Eds.). (2010). Narcissism and Machiavellianism in
youth: Implications for the development of adaptive
602 Assessment 21(5)
and maladaptive behavior. Washington, DC: American
Psychological Association Press.
Beauducel, A., & Herzberg, P. Y. (2006). On the performance
of maximum likelihood versus means and variance adjusted
weighted least squares estimation in CFA. Structural Equation
Modeling, 13, 186-203.
Bentler, P. M. (1990). Comparative fit indexes in structural mod-
els. Psychological Bulletin, 107, 238-246.
Bosson, J. K., & Swann, W. B. (2009). Self-esteem. In M. R. Leary
& R. H. Hoyle (Eds.), Handbook of individual differences in
social behavior (pp. 527-546). New York, NY: Guilford.
Boyle, G. J., Matthews, G., & Saklofske, D. (Eds.). (2008).
Handbook of personality theory and testing, Volume 2:
Personality measurement and assessment. London, UK: Sage.
Byrne, B. M., Shavelson, R. J., & Muthn, B. (1989). Testing for
the equivalence of factor covariance and mean structures:
The issue of partial measurement invariance. Psychological
Bulletin, 105, 456-466.
Caspi, A. (2000). The child is father of the man: Personality con-
tinuities from childhood to adulthood. Journal of Personality
and Social Psychology, 78, 158-172.
Caspi, A., & Shiner, R. L. (2006). Personality development. In W.
Damon & R. M. Lerner (Ser. Eds.), N. Eisenberg (Vol. Ed.),
Handbook of child psychology, Vol. 3: Social, emotional, and
personality development (6th ed., pp. 300-365). New York,
NY: John Wiley.
Cattell, R. B. (1966). Psychological theory and scientific method.
In R. B. Cattell (Ed.), Handbook of multivariate experimental
psychology (pp. 1-18). Chicago, IL: Rand McNally.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of mea-
surement invariance. Structural Equation Modeling, 14, 464-504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-
of-fit indexes for testing measurement invariance. Structural
Equation Modeling, 9, 233-255.
Church, A. T., & Burke, P. J. (1994). Exploratory and confirmatory
tests of the Big-Five and Tellegens three- and four- dimen-
sional models. Journal of Personality and Social Psychology,
66, 93-114.
Cizek, G. J., Rosenberg, S. L., & Koos, H. H. (2008). Sources
of validity evidence for educational and psychological tests.
Educational and Psychological Measurement, 68, 397-412.
Clark, L. A., & Watson, D. (2008). Temperament: An organizing
paradigm for trait psychology. In O. P. John, R. W. Robins,
& L. A. Pervin (Eds.), Handbook of personality: Theory and
research (3rd ed., pp. 265-286). New York, NY: Guilford.
Costa, P. T., Terracciano, A., & McCrae, R. R. (2001). Gender
differences in personality traits across cultures: Robust
and surprising findings. Journal of Personality and Social
Psychology, 81, 322-331.
Crews, F. T., & Boettiger, C. A. (2009). Impulsivity, frontal lobes
and risk for addiction. Pharmacology, Biochemistry, and
Behavior, 93, 237-247.
Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and
personnel decisions (2nd ed.). Urbana: University of Illinois
Press.
Cumming, G. (2012). Understanding the new statistics: Effect
sizes, confidence intervals, and meta-analysis. New York,
NY: Routledge.
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
De Fruyt, F., Mervielde, I., Hoekstra, H. A., & Rolland, J.-P.
(2000). Assessing adolescents personality with the NEO-
PI-R. Assessment, 7, 329-345.
De Pauw, S. S. W., & Mervielde, I. (2010). Temperament, person-
ality, and developmental psychopathology: A review based
on the conceptual dimensions underlying childhood traits.
Child Psychiatry and Human Development, 41, 313-329.
DeYoung, C. G., & Gray, J. R. (2009). Personality neuroscience:
Explaining individual differences in affect, behavior, and
cognition. In P. J. Corr & G. Matthews (Eds.), Cambridge
handbook of personality psychology (pp. 323-346). New
York, NY: Cambridge University Press.
Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E.
(2006). The Mini-IPIP Scales: Tiny-yet-effective measures of
the Big Five factors of personality. Psychological Assessment,
18, 192-203.
Donnellan, M. B., Trzesniewski, K. H., Robins, R. W., Moffitt, T.
E., & Caspi, A. (2005). Low self-esteem is related to aggres-
sion, antisocial behavior, and delinquency. Psychological
Science, 16, 328-335.
Else-Quest, N. M., Hyde, J. S., Goldsmith, H. H., & Van Hulle,
C. A. (2006). Gender differences in temperament: A meta-
analysis. Psychological Bulletin, 132, 33-72.
Frick, P. J., Cornell, A., Bodin, D., Dane, H., Barry, C. T., &
Loney, B. R. (2003). Callous-unemotional traits and develop-
mental pathways to severe conduct problems. Developmental
Psychology, 39, 246-260.
Furnham, A., Guenole, N., Levine, S. Z., & Chamorro-Premuzic,
T. (2013). The NEO Personality InventoryRevised: Factor
structure and gender invariance from exploratory struc-
tural equation modeling analyses in a high-stakes setting.
Assessment, 20, 14-23.
Gadow, K. D., & Sprafkin, J. (1999). Youths Inventory 4 (YI-4):
Manual. Stony Brook, NY: Checkmate Plus.
Garca-Coll, C. T., & Magnuson, K. (1997). The psychological
experience of immigration: A developmental perspective.
In A. Booth (Ed.), Immigration and the family: Research
and policy on US immigrants (pp. 91-131). Hillsdale, NJ:
Lawrence Erlbaum.
Goldberg, L. R. (1990). An alternative description of personal-
ity: The Big-Five factor structure. Journal of Personality
and Social Psychology, 59, 1216-1229.
Goldberg, L. R. (1993). The structure of phenotypic personality
traits. American Psychologist, 48, 26-34.
Goldberg, L. R. (1999). A broad-bandwidth, public-domain,
personality inventory measuring the lower-level facets of
several five-factor models. In I. Mervielde, I. Deary, F.
De Fruyt, & F. Ostendorf (Eds.), Personality psychology
in Europe (Vol. 7, pp. 7-28). Tilburg, Netherlands: Tilburg
University Press.
Goldberg, L. R. (2001). Analyses of Digmans child-personality
data: Derivation of Big-Five factor scores from each of six
samples. Journal of Personality, 69, 710-743.
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very
brief measure of the Big-Five personality domains. Journal of
Research in Personality, 37, 504-528.
Habermas, T., & de Silveira, C. (2008). The development of global
coherence in life narratives across adolescence: Temporal,
Morizot 603
causal, and thematic aspects. Developmental Psychology, 44,
707-721.
Harter, S. (2006). The self. In W. Damon & R. M. Lerner (Ser.
Eds.), N. Eisenberg (Vol. Ed.), Handbook of child psychol-
ogy, vol. 3: Social, emotional, and personality development
(6th ed., pp. 505-570). New York, NY: John Wiley.
Haynes, S. N., Richards, D. C. S., & Kubany, E. S. (1995). Content
validity in psychological assessment: A functional approach
to concepts and methods. Psychological Assessment, 7, 238-
247.
Hogan, J., & Roberts, B. W. (1996). Issues and non-issues in
the fidelity/bandwidth tradeoff. Journal of Organizational
Behavior, 17, 627-637.
Hu, L., & Bentler, P. M. (1999). Cut-off criteria for fit indexes
in covariance structure analysis: Conventional criteria versus
new alternatives. Structural Equation Modeling, 6, 1-55.
John, O., Donahue, E., & Kentle, R. (1991). The Big Five
InventoryVersions 4a and 54 (Tech. Rep., Institute of
Personality Assessment and Research). Berkeley: University
of California, Berkeley.
John, O. P., Neumann, L. P., & Soto, C. J. (2008). Paradigm shift
to the integrative Big-Five trait taxonomy: History, measure-
ment, and conceptual issues. In O. P. John, R. W. Robins,
& L. A. Pervin (Eds.), Handbook of personality: Theory and
research (3rd ed., pp. 114-158). New York, NY: Guilford.
Jones, D. N., & Paulhus, D. L. (2009). Machiavellianism. In M.
R. Leary & R. H. Hoyle (Eds.), Handbook of individual dif-
ferences in social behavior (pp. 93-108). New York, NY:
Guilford.
Jones, S. E., Miller, J. D., & Lynam, D. R. (2011). Personality,
antisocial behavior, and aggression: A meta-analytic review.
Journal of Criminal Justice, 39, 329-337.
Klein, D. N., Dyson, M. W., Kujawa, A. J., & Kotov, R. (2012).
Temperament and internalizing disorders. In M. Zentner & R.
L. Shiner (Eds.), Handbook of temperament (pp. 541-561).
New York, NY: Guilford.
Klein, D. N., Kotov, R., & Bufferd, S. J. (2011). Personality and
depression: Explanatory models and review of the evidence.
Annual Review of Clinical Psychology, 7, 269-295.
Kotov, R., Gamez, W., Schmidt, F., & Watson, D. (2010). Linking
big personality traits to anxiety, depressive, and substance use
disorders: A meta-analysis. Psychological Bulletin, 136, 768-
821.
Krueger, R. F., Caspi, A., Moffitt, T. E., Silva, P. A., & McGee,
R. (1996). Personality traits are differentially linked to mental
disorders: A multitrait-multidiagnosis study of an adolescent
birth cohort. Journal of Abnormal Psychology, 105, 299-312.
Le Blanc, M. (1996). MASPAQ: Mesures de ladaptation per-
sonnelle et sociale pour les adolescents qubcois: Manuel
et guide dutilisation [Measures of Quebec Adolescents
Social and Personal Adjustment (3rd ed.)]. Montreal,
Canada: Research Group on Problem Adolescents, School of
Psychoeducation, University of Montreal.
Leibenluft, E., Cohen, P., Gorrindo, T., Brook, J. S., & Pine,
D. S. (2006). Chronic versus episodic irritability in youth:
A community-based, longitudinal study of clinical and
diagnostic Associations. Journal of Child and Adolescent
Psychopharmacology, 16, 456-466.
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Lengua, L. J., West, S. G., & Sandler, I. N. (1998). Temperament
as a predictor of symptomatology in children: Addressing the
contamination of measures. Child Development, 69, 164-181.
Loevinger, J. (1957). Objective tests as instruments of psychologi-
cal theory. Psychological Reports, 3, 635-694.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996).
Power analysis and determination of sample size for covari-
ance structure modeling. Psychological Methods, 1, 130-149.
Madden, G. J., & Bickel, W. K. (Eds.). (2010). Impulsivity:
The behavioral and neurological science of discounting.
Washington, DC: American Psychological Association.
Malouff, J. M., Thorsteinsson, E. B., & Schutte, N. S. (2005). The
relationship between the Five-Factor model of personality
and symptoms of clinical disorders: A meta-analysis. Journal
of Psychopathology and Behavioral Assessment, 27, 101-114.
Mann, M., Hosman, C. M. H., Schaalma, H. P., & de Vries, N. K.
(2004). Self-esteem in a broad-spectrum approach for mental
health promotion. Health Education Research, 19, 357-372.
Markon, K. E., Krueger, R. F., & Watson, D. (2005). Delineating
the structure of normal and abnormal personality: An integra-
tive hierarchical approach. Journal of Personality and Social
Psychology, 88, 139-157.
Marsh, H. W., Ellis, L., Parada, R., Richards, G. E., & Heubeck, B.
(2005). A short version of the Self-Description Questionnaire
II: Operationalizing criteria for short-form evaluation
with new applications of confirmatory factor analyses.
Psychological Assessment, 17, 81-102.
Marsh, H. W., & Hau, K. T. (1996). Assessing goodness of fit:
Is parsimony always desirable? Journal of Experimental
Education, 64, 364-390.
Marsh, H. W., Hau, K. T., Balla, J. R., & Grayson, D. (1998).
Is more ever too much: The number of indictors per factor
in confirmatory factor analysis. Multivariate Behavioral
Research, 33, 181-220.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden
rules: comment on hypothesis testing approaches to setting
cutoff values for fit indices and dangers in overgeneral-
izing Hu & Bentlers (1999) findings. Structural Equation
Modeling, 11, 320-341.
Marsh, H. W., Ldtke, O., Muthn, B. O., Asparouhov, T., Morin,
A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look
at the Big-Five factor structure through exploratory structural
equation modeling. Psychological Assessment, 22, 471-491.
McCrae, R. R., & Costa, P. T. (1997). Personality trait structure
as a human universal. American Psychologist, 52, 509-516.
McCrae, R. R., & Costa, P. T. (2010). NEO Inventories for the
NEO Personality Inventory-3 (NEO-PI-3), NEO Five-Factor
Inventory-3 (NEO-FFI-3), NEO Personality Inventory-
Revised (NEO-PI-R): Professional manual. Lutz, FL:
Psychological Assessment Resources.
McCrae, R. R., Costa, P. T., & Martin, T. A. (2005). The NEO-PI-3:
A more readable revised NEO Personality Inventory. Journal
of Personality Assessment, 84, 261-270.
McCrae, R. R., Costa, P. T., Terracciano, A., Parker, W. D., Mills,
C. J., De Fruyt, F., & Mervielde, I. (2002). Personality trait
development from age 12 to age 18: Longitudinal, cross-sec-
tional and cross-cultural analyses. Journal of Personality and
Social Psychology, 83, 1456-1468.
604 Assessment 21(5)
McCrae, R. R., & Sutin, A. R. (2009). Openness to experience. In
M. R. Leary & R. H. Hoyle (Eds.), Handbook of individual
differences in social behavior (pp. 257-273). New York, NY:
Guilford.
McCrae, R. R., Zonderman, A. B., Costa, P. T., Bond, M. H., &
Paunonen, S. (1996). Evaluating the replicability of factors in
the revised NEO Personality Inventory: Confirmatory factor
analysis versus procrustes rotation. Journal of Personality &
Social Psychology, 70, 552-566.
Meredith, W. (1993). Measurement invariance, factor analysis and
factorial invariance. Psychometrika, 58, 525-543.
Messick, S. (1995). Validity of psychological assessment:
Validation of inferences from persons responses and perfor-
mances as scientific inquiry into score meaning. American
Psychologist, 50, 741-749.
Miller, J. D., & Lynam, D. (2001). Structural models of personal-
ity and their relation to antisocial behavior: A meta-analytic
review. Criminology, 39, 765-798.
Millsap, R. E., & Olivera-Aguilar, M. (2012). Investigating mea-
surement invariance using confirmatory factor analysis. In R.
H. Hoyle (Ed.), Handbook of structural equation modeling
(pp. 209-231). New York, NY: Guilford.
Morin, A. J. S., & Maano, C. (2011). Cross-validation of the short
form of the Physical Self-Inventory (PSI-S) using exploratory
structural equation modeling (ESEM). Psychology of Sport
and Exercise, 12, 540-554.
Morin, A. J. S., Marsh, H. W., & Nagengast, B. (2013). Exploratory
structural equation modeling. In G. R. Hancock & R. O.
Mueller (Eds.), Structural equation modeling: A second
course (2nd ed., pp. 395-436). Charlotte, NC: Information
Age.
Mullins-Sweatt, S. N., Jamerson, J. E., Samuel, D. B., Olson, D.
R., & Widiger, T. A. (2006). Psychometric properties of an
abbreviated instrument of the five-factor model. Assessment,
13, 119-137.
Muthn, L. K., & Muthn, B. O. (2010). Mplus users guide
[Version 6.12, computer software]. Los Angeles, CA: Muthn
& Muthn.
Newman, J. P., & Wallace, J. F. (1993). Diverse pathways to defi-
cient self-regulation: Implications for disinhibitory psychopa-
thology in children. Clinical Psychology Review, 13, 690-720.
Nigg, J. T. (2006). Temperament and psychopathology. Journal of
Child Psychology and Psychiatry, 47, 395-422.
Noftle, E. E., & Robins, R. W. (2007). Personality predictors of
academic outcomes: Big Five correlates of GPA and SAT
scores. Journal of Personality and Social Psychology, 93,
116-130.
Ong, A. D., Bergeman, C. S., Bisconti, T. L., & Wallace, K. A.
(2006). Psychological resilience: Positive emotions, and suc-
cessful adaptation to stress in later life. Journal of Personality
and Social Psychology, 91, 730-749.
Ozer, D., & Benet-Martnez, V. (2006). Personality and the
prediction of consequential outcomes. Annual Review of
Psychology, 57, 401-421.
Parker, J. D. A., Bagby, R. M., & Summerfeldt, L. J.
(1993). Confirmatory factor analysis of the Revised
Neo-Personality Inventory. Personality and Individual
Differences, 15, 463-466.
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Parker, W. D., & Stumpf, H. (1998). A validation of the five-
factor model of personality in academically talented youth
across observers and instruments. Personality and Individual
Differences, 25, 1005-1025.
Polit, D. F., & Beck, C. T. (2006). The Content Validity Index:
Are you sure you know whats being reported? Critique and
recommendations. Research in Nursing and Health, 29, 489-
497.
Poropat, A. E. (2009). A meta-analysis of the five-factor model
of personality and academic performance. Psychological
Bulletin, 135, 322-338.
Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom
Swifts electric factor analysis machine. Understanding
Statistics, 2, 13-43.
Rammstedt, B., & John, O. P. (2007). Measuring personality in one
minute or less: A 10-item version of the Big Five Inventory
in English and German. Journal of Research in Personality,
41, 203-212.
Raykov, T. (1997). Estimation of composite reliability for con-
generic measures. Applied Psychological Measurement, 21,
173-184.
Raykov, T. (1998). Coefficient alpha and composite reliability with
interrelated nonhomogeneous items. Applied Psychological
Measurement, 22, 375-385.
Raykov, T. (2001). Bias of coefficient alpha for fixed conge-
neric measures with correlated errors. Applied Psychological
Measurement, 25, 69-76.
Raykov, T. (2007a). Evaluation of revision effect on criterion
validity of multiple-component measuring instruments.
Multivariate Behavioral Research, 42, 415-434.
Raykov, T. (2007b). Reliability if deleted, not alpha if deleted:
Evaluation of scale reliability following component deletion.
British Journal of Mathematical and Statistical Psychology,
60, 201-206.
Raykov, T. (2012). Scale construction and development using
structural equation modeling. In R. H. Hoyle (Ed.), Handbook
of structural equation modeling (pp. 472-492). New York,
NY: Guilford.
Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When
can categorical variables be treated as continuous? A com-
parison of robust continuous and categorical SEM estima-
tion methods under suboptimal conditions. Psychological
Methods, 17, 354-373.
Roberts, B. W., Jackson, J. J., Fayard, J. V., Edmonds, G., &
Meints, J. (2009). Conscientiousness. In M. R. Leary & R.
H. Hoyle (Eds.), Handbook of individual differences in social
behavior (pp. 369-381). New York, NY: Guilford.
Roberts, B. W., Kuncel, N., Shiner, R. N., Caspi, A., & Goldberg,
L. R. (2007). The power of personality: The comparative
validity of personality traits, socio-economic status, and
cognitive ability for predicting important life outcomes.
Perspectives in Psychological Science, 2, 313-345.
Rosellini, A. J., & Brown, T. A. (2011). The NEO Five-Factor
Inventory: Latent structure and relationships with dimensions
of anxiety and depressive disorders in a large clinical sample.
Assessment, 18, 27-38.
Rothbart, M. K., & Bates, J. E. (2006). Temperament. In W.
Damon & R. M. Lerner (Ser. Eds.), N. Eisenberg (Vol. Ed.),
Morizot 605
Handbook of child psychology, vol. 3: Social, emotional, and
personality development (6th ed., pp. 99-166). New York,
NY: John Wiley.
Sargant, J. D., Tanski, S., Stoolmiller, M., & Hanewinkel, R.
(2010). Using sensation seeking to target adolescents for sub-
stance use interventions. Addiction, 105, 506-514.
Satorra, A. (2000). Scaled and adjusted restricted tests in multi-
sample analysis of moment structures. In R. D. H. Heijmans,
D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multi-
variate statistical analysis: A Festschrift for Heinz Neudecker
(pp. 233-247). London, UK: Kluwer.
Saucier, G. (1994). Mini-Markers: A brief version of Goldbergs
unipolar Big-Five markers. Journal of Personality Assessment,
63, 506-516.
Saucier, G. (2002). Orthogonal markers for orthogonal factors:
The case of the Big Five. Journal of Personality Research,
36, 1-31.
Saucier, G., & Goldberg, L. R. (2002). Assessing the Big Five:
Applications of 10 psychometric criteria to the development
of marker scales. In B. De Raad & M. Perugini (Eds.), Big
Five assessment (pp. 29-58). Gottingen, Germany: Hogrefe
& Huber.
Scarpa, A., & Raine, A. (1997). Psychophysiology of anger and
violent behavior. Psychiatric Clinics of North America, 20,
375-394.
Schmitt, P., Realo, A., Voracek, M., & Allik, J. (2008). Why cant
a man be more like a woman? Sex differences in Big Five
personality traits across 55 cultures. Journal of Personality
and Social Psychology, 94, 168-182.
Scholte, R. H. J., van Aken, M. A. G., & van Lieshout, C. F. M.
(1997). Adolescent personality factors in self-ratings and peer
nomination and their prediction of peer acceptance and peer
rejection. Journal of Personality Assessment, 69, 534-554.
Shiner, R. L. (2010). Mapping the landscape of personal-
ity in childhood and adolescence. Social and Personality
Psychology Compass, 4, 1084-1097.
Shiota, M. N., Keltner, D., & John, O. P. (2006). Positive emotion
dispositions differentially associated with Big Five personal-
ity and attachment style. Journal of Positive Psychology, 1,
61-71.
Silverstein, A. B. (1990). Short forms of individual intelligence
tests. Psychological Assessment, 2, 3-11.
Simms, L. J., & Watson, D. (2007). The construct validation
approach to personality scale construction. In R. W. Robins,
R. C. Fraley, & R. F. Krueger (Eds.), Handbook of research
methods in personality psychology (pp. 240-258). New York,
NY: Guilford.
Smith, G. T., Fischer, S., & Fister, S. M. (2003). Incremental valid-
ity principles in test construction. Psychological Assessment,
15, 467-477.
Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the
sins of short-form development. Psychological Assessment,
12, 102-111.
Snaith, R. P., & Taylor, C. M. (1985). Irritability: Definition,
assessment, and associated factors. British Journal of
Psychiatry, 147, 127-136.
Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The
developmental psychometrics of Big Five self-reports:
Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015
Acquiescence, factor structure, coherence, and differentia-
tion from ages 10 to 20. Journal of Personality and Social
Psychology, 94, 718-737.
Spear, L. P. (2010). The behavioral neuroscience of adolescence.
New York, NY: Norton.
Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002).
Issues and strategies for reducing the length of self-report
scales. Personnel Psychology, 55, 167-194.
Stringaris, A., Cohen, P., Pine, D. S., & Leibenluft, E. (2009).
Adult outcomes of youth irritability: A 20-year prospective
community-based study. American Journal of Psychiatry,
166, 1048-1054.
Tackett, J. L. (2006). Evaluating model of the personality-psycho-
pathology relationship in children and adolescents. Clinical
Psychology Review, 26, 584-599.
Tackett, J. L., Krueger, R. F., Iacono, W. G., & McGue, M. (2008).
Personality in middle childhood: A hierarchical structure and
longitudinal connections with personality in late adolescence.
Journal of Research in Personality, 42, 1456-1462.
Tackett, J. L., Martel, M. M., & Kushner, S. C. (2012).
Temperament, externalizing disorders, and attention-deficit/
hyperactivity disorder. In M. Zentner & R. L. Shiner (Eds.),
Handbook of temperament (pp. 562-580). New York, NY:
Guilford.
Tackett, J. L., Slobodskaya, H. R., Mar, R. A., Deal, J., Halverson,
C. F., Baker, S. R., Pavlopoulos, V., & Besevegis, E. (2012).
The hierarchical structure of childhood personality in five
countries: Continuity from early childhood to early adoles-
cence. Journal of Personality, 80, 847-879.
Tarter, R. E., Blackson, T., Brigham, J., Moss, H., & Caprara,
G. V. (1995). The association between childhood irritability
and liability to substance use in early adolescence: A 2-year
follow-up study of boys at risk for substance use. Drug and
Alcohol Dependence, 39, 253-261.
Tellegen, A., & Waller, N. G. (2008). Exploring personality
through test construction: Development of the multidimen-
sional personality questionnaire. In G. J. Boyle, G. Matthews,
& D. H. Saklofske (Eds.), The Sage handbook of personality
theory and assessment, vol. 2: Personality measurement and
testing (pp. 261-292). London, UK: Sage.
Terracciano, A., Lockenhoff, C. E., Crum, R. M., Bienvenu, O. J.,
& Costa, P. T. (2008). Five-Factor Model personality profiles
of drug users. BMC Psychiatry, 8, 22.
Uliaszek, A. A., Hauner, K. K. Y., Ziberg, R. E., Craske, M. G.,
Mineka, S., Griffith, J. W., & Rose, R. D. (2009). An exami-
nation of content overlap and disorder-specific predictions in
the associations of neuroticism with anxiety and depression.
Journal of Research in Personality, 43, 785-794.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthe-
sis of the measurement invariance literature: suggestions,
practices, and recommendations for organizational research.
Organizational Research Methods, 3, 4-70.
Watson, D., & Clark, L. A. (1997). Extraversion and its positive
emotional core. In R. Hogan, J. Johnson, & S. Briggs (Eds.),
Handbook of personality psychology (pp. 767-793). San
Diego, CA: Academic Press.
West, S. G., Taylor, A. B., & Wu, W. (2012). Model fit and model
selection in structural equation modeling. In R. H. Hoyle
606 Assessment 21(5)

(Ed.), Handbook of structural equation modeling (pp. 209-
231). New York, NY: Guilford.
Whiteside, S. P., & Lynam, D. R. (2001). The five factor model and
impulsivity: Using a structural model of personality to understand
impulsivity. Personality and Individual Differences, 30, 669-689.
Widiger, T. A., & Smith, G. T. (2008). Personality and psycho-
pathology. In O. P. John, R. Robins, & L. A. Pervin (Eds.),
Handbook of personality: Theory and research (3rd ed.,
pp. 743-769). New York, NY: Guilford.
Wilson, D. S., Near, D., & Miller, R. R. (1996). Machiavellianism:
A synthesis of the evolutionary and psychological literatures.
Psychological Bulletin, 119, 285-299.
Wilt, J., & Revelle, W. (2009). Extraversion. In M. Leary & R.
H. Hoyle (Eds.), Handbook of individual differences in social
behavior (pp. 27-45). New York, NY: Guilford.
Zuckerman, M. (2009). Sensation seeking. In M. R. Leary & R.
H. Hoyle (Eds.), Handbook of individual differences in social
behavior (pp. 455-465). New York, NY: Guilford.

Downloaded from asm.sagepub.com at Tumaini Uni-Ingringa University College on November 28, 2015