Health & Place 7 (2001) 225236

A brief observational measure for urban neighborhoods

Margaret O. Caughy
a,
*, Patricia J. OCampo
b
, Jacqueline Patterson
c
a
University of Texas School of Public Health, MPH Program at UTSW-Dallas, 5323 Harry Hines Blvd, V8.112A, Dallas,
TX 75390-9128, USA
b
Johns Hopkins University, USA
c
Center for Budget and Policy Priorities, USA
Received 15 February 2001; accepted 15 February 2001
Abstract
There is growing recognition that neighborhood context contributes to the health and well-being of residents over
and above individual characteristics and health behaviors. However, few published reports exist of methods for
documenting neighborhood characteristics which are easily administered with minimal outlay of resources. In this
paper, we present the development of a brief observational method for urban neighborhoods relevant to the health and
well-being of families and children. Data from a socieconomically diverse group of urban neighborhoods are used to
create theoretically grounded measures of neighborhood context, and the utility of these measures for discriminating
between and within urban neighborhoods is demonstrated. # 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Neighborhoods; Measures
Introduction
There is growing recognition that the neighborhood in
which one lives has important implications for the health
and well-being of families and children. In a review of
the literature on neighborhood eects and child devel-
opment, Furstenberg and Hughes (1994) advocated for
research studies which incorporate direct measurement
of neighborhood conditions. Such a research agenda
presupposes, however, that appropriate methods for
assessing relevant neighborhood conditions exist. Most
assessments of neighborhood conditions rely on meth-
ods falling into two broad categories: secondary data
sources or labor-intensive ethnographic methods. Rely-
ing solely on secondary sources provides an over-
simplied picture of urban neighborhoods and masks
the high degree of within-neighborhood variability that
exists (Caughy et al., 1999). In addition, it is not possible
to assess levels of neighborhood social interaction or
cohesion relying on secondary data sources alone. In
contrast, although ethnographic studies provide vivid
pictures of neighborhood life, they do not provide data
which are easily generalizable. Furthermore, the inten-
sity of data collection and analysis makes it dicult to
incorporate ethnographic methods into large-scale
research studies.
Several accounts of more rapid assessment methods of
neighborhood characteristics have recently been re-
ported in the literature (Buckner, 1988; Cohen et al.,
2000; Coulton et al., 1996; McGuire, 1997). The
measures proposed by Buckner (1988) and Coulton
et al. (1996) relied upon residents perceptions of their
neighborhood, which represent an important aspect of
neighborhood life that should be measured. Some
characteristics of neighborhoods, such as patterns of
social interaction and normative beliefs regarding
individual behaviors, can only be assessed by talking
to neighborhood residents. However, resident percep-
tions are limited by same source bias which arises
when the same informant provides information on both
the risk factor of interest (i.e., neighborhood conditions)
*Corresponding author. Tel.: +1-214-648-1052; fax: +1-
214-648-1081.
E-mail address: margaret.caughy@utsouthwestern.edu
(M.O. Caughy).
1353-8292/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved.
PII: S 1 3 5 3 - 8 2 9 2 ( 0 1 ) 0 0 0 1 2 - 0
as well as the outcome being studied (Duncan and
Raudenbush, 1999). Because of the limitations of relying
on the perceptions of neighborhood residents, a
comprehensive assessment of neighborhood conditions
should also include objective measures.
Cohen et al. (2000) utilized an observational protocol
to assess physical characteristics of streets using
videotapes, direct observation, and secondary data.
Videotaped images were entered into a visual database,
and each structure was coded for the presence of
structural damage. Direct observations were conducted
on foot, and observers recorded the presence of a wide
variety of physical incivilities such as garbage, grati
and abandoned cars. Finally, the authors obtained
inspection reports for local schools and coded them
for physical problems. The authors constructed a
broken windows index which was the sum of (1) the
percentage of homes with cosmetic or greater damage;
(2) the percentage of streets with trash, abandoned cars,
or grati; and (3) the number of physical problems and
building code violations in the public high schools.
Although the breadth of data is laudable, the authors
provide insucient information to judge the reliability
and validity of their measure. No information is
provided on interrater reliability, so one cannot judge
whether two independent observers would judge the
same building to have the same level of visible damage.
In addition, no information is provided regarding the
rationale for constructing the summary measure. For
example, it is unclear why a street in which the majority
of homes had only minor cosmetic damage would be
rated similarly poorly on the rst component of their
index as streets in which the same proportion of homes
had major structural damage. Furthermore, no informa-
tion is provided regarding the psychometric properties
of the index.
McGuire (1997) utilized a neighborhood assessment
protocol that included perceptions data as well as an
observational component based on the tradition of
systematic social observation to collect neighborhood
data in ve adjacent census tracts in a disadvantaged
area of Boston. The systematic social observation (SSO)
protocol utilized by McGuire (1997) was adapted from
the Block Environment Inventory developed by Ralph
Taylor and his colleagues in Baltimore (Taylor et al.,
1984). Four scales were created based on theoretical
considerations as well as factor analysis results: trac/
street disorder; trash; social disorder; and social cohe-
sion. Construct validity was demonstrated by estimating
correlations between the observational scales and
residents perceptions. McGuires approach had a
number of strengths, namely the utilization of standar-
dized procedures to increase reliability, and the use of
theoretical considerations in the development of indexes.
The most signicant limitation of her approach was that
data were only collected for a small, disadvantaged
community which may have limited the range of
neighborhood conditions being measured. Finally,
McGuires measure is limited by the analytic method
chosen. Because data were collected for multiple streets
within the same neighborhood and are therefore
correlated, ordinary factor analysis methods should
not be used.
The analytic challenge inherent in creating neighbor-
hood measures from data collected on streets within
neighborhoods was addressed by Raudenbush and
Sampson (1999) using an innovative analytic method
and data from the Project for Human Development in
Chicago Neighborhoods (PHDCN). The researchers
identied 343 neighborhood clusters (NCs) which
represented combinations of all 865 census tracts in
Chicago that were as ecologically meaningful as
possible, composed of geographically contiguous census
tracts, and internally homogeneous on key census
indicators (p. 12). Their observational protocol has
many of the same origins as the McGuire protocol used
in Boston. Observational data were collected for every
street in 196 census tracts in Chicago by simultaneously
videotaping both sides of a streets face blocks from a
sports utility vehicle (SUV) driven at a rate of ve miles
per hour. In addition to videotaped data, two coders in
the SUV coded their observations as the vehicle
proceeded down the street. Items coded included a wide
variety of indicators of physical and social disorder
including trash, grati, abandoned cars, and loitering
adults.
Raudenbush and Sampson (1999) demonstrated how
these data could be used to estimate two distinct
latent constructs, physical disorder and social dis-
order, by using a three-level item response model in
which items were embedded in face blocks which were in
turn embedded in neighborhood clusters (NCs). Results
indicated there was sucient variability in physical
disorder not only between neighborhoods but also
between face blocks within neighborhoods to yield a
highly reliable measure at the neighborhood level. The
social disorder scale demonstrated adequate reliability at
the neighborhood level but insucient variability
between face blocks within neighborhoods. Validity
was demonstrated by reporting correlations between
these measures and other indicators of physical/social
disorder.
Although the methods reported by Raudenbush and
Sampson (1999) represent a signicant advancement in
the eld of ecological assessment, this method has a
variety of shortcomings which limit its utility. First, the
cost of data collection and analysis is labor-intensive
and cost-prohibitive for the large proportion of indivi-
duals who might be interested in conducting such
assessments. Furthermore, we contend that driving any
sort of vehicle through a neighborhood at a slow rate of
speed for the purposes of videotaping is intrusive and
M.O. Caughy et al. / Health & Place 7 (2001) 225236 226
disrespectful of neighborhood residents and their com-
munity. Residents are videotaped without their knowl-
edge or consent, which raises interesting legal issues if
illegal activities are within view when the SUV drives by.
Early in our own data collection process, we found that
attempting to code neighborhood conditions from a
vehicle (e.g., a windshield assessment) was received
more negatively by neighborhood residents. As a result,
we switched our data collection process to one in which
coders parked their cars and got out to walk up and
down the street while coding. Such an approach allowed
neighborhood residents to approach us if they had
questions about our activities, which they did on
numerous occasions. Furthermore, because coders are
on the street, they have a virtually 3608 view of the block
while in the neighborhood. Driving an SUV through a
neighborhood precludes the coding of events that occur
either before the vehicle reaches that portion of the
block or after the vehicle has already passed by.
It is apparent from the strengths and limitations of
existing methods summarized in Table 1 that there is a
need for a neighborhood assessment tool which is simple
to implement yet yields reliable and valid measures. In
this paper we present the development and implementa-
tion of an observational tool for urban neighborhoods,
utilize data collected from a diverse set of neighbor-
hoods to generate conceptually sound factors, and
demonstrate the utility of these conceptually derived
factors for discriminating between and within urban
neighborhoods.
Methods
Instrument development
The observational protocol described here was devel-
oped as part of a larger project examining the impact of
neighborhood conditions on parenting and child devel-
opment outcomes. A number of steps were involved in
identifying the important components of urban neigh-
borhoods which should be assessed including building
upon previous research (Caughy et al., 1999; OCampo
et al., 1997a, b) as well as an extensive literature review
of the work of researchers who have studied the impact
of various neighborhood characteristics on adults and
children.
A majority of the items for the observational protocol
were adapted from the work of Ralph Taylor and his
colleagues (Perkins et al., 1992) as well as the work of
the Project for Human Development in Chicago
Neighborhoods (PHDCN) (National Opinion Research
Center, 1995). These two measurement strategies share
many of the same roots, both being focused on
understanding geographic variability in crime and
delinquency. This area of inquiry has the longest history
in the area of neighborhood assessment often referred to
as systematic social observation, and as such, we
thought it important to build upon this work. These
tools were modied to make them appropriate for the
purposes of the larger project and for the communities
included in the project. For example, some items were
Table 1
Strengths and limitations of existing neighborhood assessment methods
Author(s) Strength(s) Limitation(s)
Buckner (1988) Able to measure non-observable
neighborhood characteristics
Limited to perceptions of residents; subject to
same-source bias
Cohen et al. (2000) Considered a wide variety of physical
attributes
No psychometric data provided; rationale
for index creation is of questionable validity;
requires videotaping, which is intrusive
Coulton et al. (1996) Able to measure non-observable
neighborhood characteristics
Limited to perceptions of residents; subject
to same-source bias
McGuire (1997) Included both observational as well as
resident perceptions measures; theoretical
considerations used in developing measures
Only tested in a small, disadvantaged
community so utility in broader range of
socioeconomic conditions unknown.
Raudenbush and Sampson (1999) Theoretical considerations used in developing
measures; collected data on a broad spectrum
of neighborhoods; used a sophisticated analytic
method for assessing neighborhood measures
Method is costly and labor-intensive; requires
videotaping, which is intrusive; videotaping
from a vehicle limits view of neighborhood;
theoretical underpinnings have limited
application to public health
M.O. Caughy et al. / Health & Place 7 (2001) 225236 227
dropped due to lack of variation in the observations or
lack of relevance to the study neighborhoods. A few
items were added because of particular relevance to the
context of the study neighborhoods.
An initial draft of the protocol was pilot tested by
conducting drive-throughs in two socioeconomically
diverse neighborhoods. Based on the results of the pilot
test, the research team recommended further modica-
tion of the tool in terms of the addition and elimination
of items. The nal instrument included 45 objective
items that covered the type and condition of buildings,
the condition of grounds and undeveloped spaces,
indications of block uniformity/territoriality, type of
street, presence of grati/litter, neighborhood resources,
and presence and activities of people. In addition to
these items, the presence and types of nonresidential
land uses were tallied. A complete copy of the
instrument can be obtained from the authors.
Selection of study neighborhoods
As part of the larger research project, neighborhoods
were selected to represent the range of socioeconomic
status and racial composition of neighborhoods in
Baltimore City, the locale of the project. For the
purposes of this study, neighborhoods were dened as
census block groups. Twenty-four census tracts stratied
by average household wealth (quartiles) and racial
composition (African-American, European-American,
or racially mixed) were identied. From each of these
census tracts, between 1 and 3 block groups were
selected resulting in a total of 57 study neighborhoods.
The characteristics of these neighborhoods are displayed
in Table 2. All of the items in the table were derived
from census data with the exception of average house-
hold wealth, which was obtained from a commercial
data source. A comparison of the study neighborhoods
with Baltimore City as a whole appears to indicate that
the study neighborhoods were slightly more advantaged
economically. However, statistical tests indicated only a
marginal dierence in childhood poverty rates, t 1:82,
p 0:07. Therefore, it would appear that the study
neighborhoods are by and large comparable to the city.
Data collection methods
Neighborhood data collection is a sensitive and
complicated task that requires intensive training to be
executed correctly. It is important that every rater have
the same understanding of the various items on the
protocol. To this end, operational denitions were
established for each observational item. Raters were
trained on the implementation of the instrument during
30 h of training over six days which included a detailed
review of the instrument, review of slides of neighbor-
hoods, walk-throughs of neighborhoods, and practice
sessions to establish interrater reliability. Training was
considered complete when interrater reliability exceeded
85% based on percent agreement.
The observational instrument was administered for
each hundred block in each of the 57 study neighbor-
hoods. A hundred block was dened as the face block
(i.e., both sides of the street) which included residences
or businesses with addresses in the same hundred group.
For example, the 1400 block of Main Street included all
houses or businesses between 1400 and 1499. Ratings
were conducted by pairs of observers, and ratings were
arrived at by consensus between the two raters. Each
rating took between 5 and 10 min to complete depending
on the length of the block. All ratings were conducted on
weekdays between 11:00 a.m. and 4:00 p.m. during the
late summer/early fall to minimize dierences between
blocks resulting from diurnal or seasonal changes.
A total of 1135 hundred blocks were rated for the 57
study neighborhoods. The number of hundred blocks
rated per neighborhood ranged from 3 to 51 with an
average of 20. Fifty-one of the neighborhoods (91.2%)
had at least 10 blocks rated. The supervisor of the
neighborhood data collection completed duplicate
Table 2
Characteristics of study neighborhoods compared with all neighborhoods in Baltimore City
Baltimore city N 832 Study neighborhoods N 57
Average household wealth $107,262 $114,567
(0$416,456) ($7,478$273,652)
Poverty rate for children 05 29% 23%
(0100%) (091%)
Unemployment rate 10% 9%
(065%) (065%)
Proportion African American 62% 61%
(0100%) (1099%)
Per capita crime rate 0.21 0.12
(0.0112.10) (0.020.91)
M.O. Caughy et al. / Health & Place 7 (2001) 225236 228
assessments on 25% of the blocks to monitor interrater
reliability, and the average reliability was 87%.
Neighborhood scale creation
We built upon the existing literature on neighborhood
assessment to identify the constructs or factors to be
estimated from our observational data. The most
frequently reported characteristic of neighborhoods
based on observation is disorder, both physical and
social (Cohen et al., 2000; McGuire, 1997; Raudenbush
and Sampson, 1999). Physical disorder includes such
things as grati, trash, and/or vacant buildings. Social
disorder includes such things as loitering adults, drug
paraphernalia on the streets, and/or prostitution.
Because we did not record any neighborhood character-
istics of social disorder, we were limited to a measure of
physical disorder. We chose to call this rst factor
physical incivilities based on the work of Perkins et al.
(1992). Perkins et al. (1992) state that physical incivilities
are associated with increased fear of crime as well as a
perception of problems in the neighborhood, loss of
condence in [the] neighborhood and the decline of
resident-based informal social controls.
Although the majority of rapid observational tools
have focused on measures of disorder, there is a
substantial literature of other important characteristics
of neighborhoods that can be derived from observation,
based primarily on the work of Ralph Taylor and his
colleagues (Perkins et al., 1992; Taylor et al., 1984;
Taylor et al., 1985). Two other observable character-
istics identied by Taylor and his colleagues are
territorial functioning and defensible space. Perkins et al.
(1992) dene territorial functioning as those markers
which convey a non-verbal message of control, separa-
tion from outsiders, and investment in the locale (p.
22). Indicators of territorial functioning include prop-
erty maintenance, beautication, signs of personaliza-
tion, and symbols of protection. Defensible space is
similar in some ways to territorial functioning. Newman
(1972 as cited by Perkins et al., 1992) describes
defensible space as including those characteristics of
the physical environment which encourage residents to
exercise territorial control. Markers of defensible space
may include real barriers such as fences, walls and
security bars as well as symbolic barriers such as
gardens, low railings, and shrubbery. Because of the
strong relationship between these two constructs, we
chose to create a single indicator from our observational
data called territoriality, to include both markers of
territoriality as well as markers of defensible space.
Perkins et al. (1992) specify that territorial markers in a
neighborhood should be related to the exercise of
informal social control. In an example given by these
authors, if a potential criminal oender crosses territor-
ial boundaries, it is expected that neighborhood
residents would take some sort of defensive action.
Therefore, we hypothesized that indicators of territori-
ality would represent outward manifestations of neigh-
borhood cohesion.
The third factor we estimated from our observational
data was availability of play resources. Because one of
the primary interests of this study is the growth and
development of children, we felt it was important to
include a measure of resources which are particularly
relevant to the health and development of young
children. The target children participating in the larger
study are preschool age, and the ability to engage in play
is important for optimal motor and cognitive develop-
ment. In our qualitative work in economically impover-
ished Baltimore neighborhoods, we often had parents
tell us they restricted the outdoor activities of their
children because of safety concerns. The practice of
parents in high risk neighborhoods to restrict outside
activities of their children has been echoed in the work
of others who have conducted ethnographic studies of
urban neighborhoods (Brodsky, 1996; Furstenberg,
1993). Therefore, we decided to estimate a neighbor-
hood indicator of the availability of play resources that
might be related to dierences in parent monitoring
behavior and/or dierences in child development be-
tween neighborhoods.
With these neighborhood constructs as a back drop,
17 indicators were created by either dichotomizing single
items or by created dichotomous variables from a
combination of several items. Although some of the
items were originally ordinal in nature, it was necessary
to dichotomize them to facilitate the multilevel logit item
response model used for the analysis (see Analysis
Methods below). The 17 indicators and their frequency
distributions are displayed in Table 3.
Based on the neighborhood research literature, it is
possible to suggest the associations between the three
neighborhood scales. We hypothesized that the physical
incivilities scale would be negatively associated with both
territoriality and play resources. Specically, we hy-
pothesized that neighborhoods that had high rates of
trash, grati, and abandoned buildings would also have
fewer signs of beautication and other signs of personal
investment in the neighborhood. Likewise, we hypothe-
sized that neighborhoods with busy streets, few yards,
and few usable playgrounds would also be more likely to
have high rates of trash and grati.
Additional measures
Several additional neighborhood measures were used
to examine the construct validity of the derived
observational measures. A composite measure of im-
poverishment was derived from census data as the
standardized sum of the poverty rate, unemployment
rate, vacant housing rate, and proportion of households
M.O. Caughy et al. / Health & Place 7 (2001) 225236 229
with children under the age of 5 which were single-
headed based on recommendations by Korbin and
Coulton (1997). Neighborhood crime density was
calculated by geocoding all crimes reported by the city
police department in 1993, and calculating the density of
reported crimes per square mile.
Neighborhood perceptions data using the
measure reported by Coulton et al. (1996) were
collected as part of family interviews with 200
primary caregivers of preschoolers living in a subsample
of the study neighborhoods. Families were identied
through door-to-door canvassing, recruitment from
day care and Head Start centers, and targeted
mailings to families with young children. The following
nine subscales of the Coulton et al. (1996) measure
were used: residential mobility, social interaction, fear
of retaliation, willingness to intervene in acts of
delinquency, willingness to intervene in acts of
misbehavior, willingness to assist children in need,
perceptions of physical and social disorder, fear of
victimization, and overall neighborhood quality. Inter-
nal reliability ranged from 0.58 to 0.94. Only one
subscale (residential mobility) had an internal reliability
coecient of less than 0.75. As family interviews are
ongoing, these data are not yet available for all study
neighborhoods.
We hypothesized a number of relationships between
our observational measures and neighborhood structure
and perceived quality. We hypothesized that physical
incivilities would be positively correlated with impover-
ishment, crime, mobility, and perceived disorder and
negatively correlated with neighborhood quality and
social cohesion. In contrast, we hypothesized that
territoriality and availability of play resources would
increase with decreasing impoverishment and crime and
with increasing perceived social cohesion and neighbor-
hood quality.
Analytic methods
The analytic methods were modeled after those used
by Raudenbush and Sampson (1999). A three-level item
response model was dened in which level one was the
item, level two was the street, and level three was the
neighborhood. Let Z
ijk
represent the log odds of
observing a positive response for item i on street j in
neighborhood k. The level-1 model represents the
variation between items within streets and is represented
by the following equation:
Z
ijk
D
pijk
p
pjk

X
6
m1
a
mjk
X
mijk
!
D
tijk
p
tjk

X
5
m1
d
mjk
W
mijk
!
D
rijk
p
rjk

X
3
m1
t
mjk
Z
mijk
!
;
Table 3
Neighborhood indicators derived from systematic social observation of 1135 streets in 57 study neighborhoods
a
Factor/item Yes N (%) No N (%)
Physical incivilities
Grati present 444 (39.1) 691 (60.9)
Resident grounds poorly maintained 94 (9.8) 870 (90.2)
Resident buildings poorly maintained 110 (11.0) 891 (89.0)
Moderate or signicant litter 280 (24.7) 853 (75.3)
Vacant/burned residences present 351 (30.9) 780 (68.7)
Vacant/burned commercial establishments present 103 (9.1) 1009 (90.7)
Public spaces poorly maintained 370 (32.9) 738 (65.0)
Territoriality
Crime watch/security/no trespassing signs visible 836 (73.7) 299 (26.3)
Residents react to presence of raters 502 (61.1) 320 (38.9)
One third or more of homes with borders/hedges 405 (40.8) 587 (59.2)
One third or more of homes with security bars 251 (25.2) 746 (74.8)
One third or more of homes with decoration 610 (61.2) 386 (38.8)
Sign visible denoting neighborhood name 25 (2.5) 968 (97.5)
Available play resources
One third or more of homes with yards 764 (76.6) 234 (23.4)
Street is not a busy thoroughfare 802 (70.8) 330 (29.2)
Public playground in good condition present 20 (1.8) 1098 (98.2)
Children visible playing 162 (14.3) 973 (85.7)
a
The balance between the sum of the yes and no responses and 1135 represents missing data.
M.O. Caughy et al. / Health & Place 7 (2001) 225236 230
where D
pijk
is an indicator coded 1 if the item is part of
the physical incivilities scale and 0 if otherwise, D
tijk
is an
indicator coded 1 if the item is part of the territoriality
scale and 0 if otherwise, and D
rijk
is an indicator coded 1
if the item is part of the play resources scale and 0 if
otherwise. X
mijk
, W
mijk
, and Z
mijk
are a series of dummy
variables representing 6 of the 7 physical incivilities
items, 5 of the 6 territoriality items, and 3 of the 4 play
resources items, respectively. These dummy variables are
centered around their grand means, allowing the
intercept variables, p
pjk
, p
tjk
, and p
rjk
, to be interpreted
as the log-odds of observing a positive response on a
typical item on the physical incivilities scale, territori-
ality scale, and play resources scale, respectively. These
log-odds are allowed to vary at both the street and
neighborhood levels. The remaining parameters, a
mjk
,
d
mjk
, and t
mjk
, when multiplied by 1, represent the item
diculties or severities for item m in the physical
incivilities scale, the territoriality scale, and the play
resources scale, respectively. These item diculties are
considered as xed eects at the street and neighbor-
hood levels. For complete information on the analytic
method, please see Raudenbush and Sampson (1999).
The parameters for the three-level item response model
were estimated using MLwiN BETA version 1.10.0001
(Rabash et al., 1999).
Results
The estimated item diculties at the street level are
displayed in Table 4, and the variancecovariance
components are displayed in Table 5. For the item
diculties, one item per scale is chosen as the reference
item, usually the item with the median frequency, and
the other item diculties should be compared to it. The
item diculties show a nice bit of variation, one of the
characteristics of a good scale, and their ordering is
consistent with what one would expect from the
frequency distributions of the items. Items which were
observed more frequently are considered less severe or
dicult than items observed less frequently.
Turning now to the variancecovariance components
in Table 5, the estimated variation in physical incivilities
between streets in the same neighborhood is 0.657 while
the between neighborhood variation is 1.919, yielding an
intraclass correlation for this scale of 0.74. That is, 74%
of the variation in physical incivilities is estimated to be
between neighborhoods. This is substantially higher
than the corresponding amount reported by Rauden-
bush and Sampson (1999) of 39% for their physical
disorder scale. For the territoriality and play resources
scales, the within neighborhood variances as well as the
scale covariances were estimated to be zero. This is
Table 4
Item diculty at the street level
Scale/item Coecient SE
Physical incivilities
Intercept 1.375 0.195
Grati 0.966 0.111
Condition of public spaces 0.640 0.112
Vacant residences 0.450 0.113
Litter 0 (reference)
Condition of resident buildings 1.290 0.148
Condition of resident grounds 1.446 0.148
Vacant commercial establishments 1.547 0.142
Territoriality
Intercept 0.388 0.079
Crime watch/security/no trespassing signs 1.478 0.095
Reactions to raters 0.874 0.099
Residences with decoration 0.862 0.094
Residences with borders/hedges 0 (reference)
Residences with security bars 0.741 0.099
Sign denoting neighborhood name 3.343 0.214
Play resources
Intercept 1.805 0.139
Yards 3.245 0.121
Busy street 2.944 0.115
Children playing 0 (reference)
Presence of usable public playground 2.267 0.243
M.O. Caughy et al. / Health & Place 7 (2001) 225236 231
similar to the ndings of Raudenbush and Sampson
(1999) for their social disorder scale, who concluded that
the zero variance estimate resulted from the rarity of a
positive response on their social disorder scale rather
than from a true lack of variance in social disorder
within neighborhoods. It is unlikely this is a similar issue
for the data reported here. Whereas the proportion of
positive responses was extremely low for the items on
Raudenbush and Sampsons social disorder scale (range
0.075.7%), the proportions of positive responses for
the territoriality and play resources scales in this study
were much higher. Therefore, given the estimated
variances, it is not possible to reject the null hypothesis
that the true within neighborhood variances for
territoriality and play resources are zero.
Contrary to our prediction, the Territoriality scale
was not negatively correlated with the physical incivi-
lities. The territoriality scale was also not correlated with
the play resources scale. However, consistent with our
prediction, the correlation between physical incivilities
in a neighborhood and play resources was 0.29, which
was just short of statistical signicance (p 0:05),
suggesting that neighborhoods with a high degree of
trash, grati and abandoned buildings are also less
likely to have resources available for childrens play.
Raudenbush and Sampson (1999) provide methods
for calculating street-level as well as neighborhood-level
reliabilities. The neighborhood-level reliability for each of
the scales was very high at 0.96, 0.93, and 0.94 for the
physical incivilities, territoriality, and play resources
scales, respectively. At the street-level, the reliability of
the physical incivilities was 0.74, which is more than
adequate. However, the street-level reliability of the
territoriality and play resources scales was low at 0.33
and 0.42, respectively. These results indicate that
although there is sucient variation in territoriality
and play resources to distinguish between neighbor-
hoods, there is insucient variation to distinguish
between streets within the same neighborhood. In
contrast, the variability in physical incivilities was
sucient for distinguishing not only between neighbor-
hoods but also between streets within the same
neighborhood.
Construct validity was examined by estimating the
correlation between the three observational scales,
neighborhood structural characteristics and residents
perceptions of their neighborhood. These correlations
are displayed in Table 6. As predicted, impoverished
neighborhoods with higher crime density also had higher
rates of observed physical incivilities. Territoriality was
not correlated with neighborhood impoverishment but
was positively correlated with crime density. Although
this was not predicted, it may have been that selected
elements of the territoriality scale such as security bars
and/or crime watch signs may have contributed to the
positive association between territoriality and crime
because the presence of these items in a neighborhood
may have been a response of the residents to higher
crime rates. Also surprising was the positive correlation
between play resources and neighborhood impoverish-
ment.
The physical incivilities scale was correlated as
expected with residents perceptions of neighborhood
conditions. Physical incivilities were positively corre-
lated with residential mobility, fear of retaliation, and
perceived physical and social disorder and negatively
correlated with willingness to intervene in acts of
delinquency and/or misbehavior or provide aid to
children in need as well as with overall neighborhood
quality. Territoriality appeared to have the opposite
association with perceived neighborhood conditions,
having a negative association with residential mobility
and fear of retaliation, and positive associations with
perceived willingness of residents to collectively monitor
youth. As such, it conrms our hypothesis that observed
signs of territoriality may be an outward manifestation
Table 5
Variance-covariance components
Variancecovariance component Estimate SE
Between streets w/in neighborhoods Variance of physical incivilities 0.657 0.083
Variance of territoriality } }
Variance of play resources } }
Covariance of physical incivilities and territoriality } }
Covariance of physical incivilities and play resources } }
Covariance of territoriality and play resources } }
Between neighborhoods Variance of physical incivilities 1.919 0.384
Variance of territoriality 0.212 0.051
Variance of play resources 0.759 0.165
Covariance of physical incivilities and territoriality 0.001 0.099
Covariance of physical incivilities and play resources 0.358 0.184
Covariance of territoriality and play resources 0.057 0.065
M.O. Caughy et al. / Health & Place 7 (2001) 225236 232
of social cohesion in the community. In contrast, the
correlations between the play resources scale and the
neighborhood perceptions measures, while signicant,
were counterintuitive, with negative associations with
neighborhood monitoring of youth as well as overall
quality. It is unclear whether these associations may be
an artifact of the restricted range of neighborhoods for
which perceptions data are currently available compared
to the full range of neighborhoods providing observa-
tional data.
A further step was taken to examine the utility of the
observational methods of neighborhood conditions by
examining the variability of the indicators within
neighborhoods that are relatively homogeneous with
respect to socioeconomic status. We have argued else-
where that measures of socioeconomic status alone mask
a considerable amount of variability within neighbor-
hoods (Caughy et al., 1999). To examine this, the study
neighborhoods were stratied according to their quartile
of average household wealth with respect to the city as a
whole, and the distribution of each neighborhood
observational indicator within each wealth quartile is
displayed in the Fig. 1. As expected, there appears to be
a linear and negative association between physical
incivilities and average household wealth, with observed
physical incivilities decreasing with increasing average
household wealth. Notable, however, is the signicant
amount of variation within wealth quartiles in the
amount of physical incivilities that were observed. As
can be seen in the gure, there is a signicant amount of
overlap in the distributions, with some neighborhoods in
the lowest quartile of wealth having rates of physical
incivilities which are lower than neighborhoods in the
highest quartile. Thus, socioeconomic status is insu-
cient as a measure for discriminating between neighbor-
hood residential contexts.
The distributions of territoriality and play resources
by average household wealth are also displayed in the
gure. The correlation between territoriality and average
household wealth is non-signicant (r 0:218, p
0:12 after excluding outlier cases), and the variation
appears more restricted within wealth categories than
physical incivilities. Counterintuitively, availability of
play resources is negatively correlated with average
household wealth (r 0:345, p50:05 after excluding
outlier cases). The degree of variation in the play
resources scale in neighborhoods in the lowest quartile
of wealth is notable.
Discussion
We presented the results of the development of an
assessment tool for collecting information on neighbor-
hood characteristics of relevance to studies concerned
with the well-being of families and children. Our
instrument draws from previous eorts (NORC, 1995;
Perkins et al., 1992) and includes additional items
important for understanding neighborhood inuences
on parents and children. Our protocol can supplement
information on neighborhoods obtained from more
routine secondary sources of data as it is easy to
implement and yields factors with high reliability.
Previous contextual research which rely on secondary
sources of data for information on neighbor-
hood characteristics has narrowly conceptualized
Table 6
Correlations between observational measures of neighborhood, neighborhood structural characteristics, and residents perceptions of
neighborhood climate
a
Physical incivilities Territoriality Availability of
play resources
Neighborhood structural characteristics
Impoverishment 0.293** 0.120 0.279***
Crime density 0.407*** 0.346*** 0.109
Neighborhood perceptions scale
Residential mobility 0.341*** 0.183** 0.087
Social interaction 0.070 0.017 0.104
Fear of retaliation 0.389*** 0.193** 0.133*
Intervene delinquency 0.133* 0.131* 0.150**
Intervene misbehavior 0.189** 0.128* 0.030
Intervene assist children in need 0.253*** 0.174** 0.164**
Physical/social disorder 0.375*** 0.129 0.029
Fear of victimization 0.078 0.090 0.048
Overall neighborhood quality 0.175** 0.134 0.148**
a
Outliers on the territoriality and play resources scales were excluded, respectively, in the calculations of correlations between each
observational scale and perceptions scale. The correlations for the play resources scale were virtually unchanged, but the correlations
for the territoriality scale changed substantially. See gure for graphic display of outliers for each scale. *p50:10, **p50:05,
***p50:01.
M.O. Caughy et al. / Health & Place 7 (2001) 225236 233
neighborhood inuences on the health and well-being of
residents. Collection of primary data using rapid
assessment methods such as the observational method
presented here oers the opportunity to expand our
understanding of the relevant characteristics that aect
families with relatively little outlay of resources.
Collecting data on each block face took anywhere from
10 to 30 min depending on whether residents stopped to
talk to the neighborhood assessors. Data entry of the
instrument was straightforward as all items were close
ended or of a check box format.
The rapid assessment method presented here has a
number of advantages compared with other objective
assessment methods of neighborhood conditions re-
cently reported in the literature. Compared with
measures using videotaped data collection (Cohen
et al., 2000; Raudenbush and Sampson, 1999), our
method is quick and easy to implement. Despite this ease
of implementation, however, the factors it yields show a
high degree of reliability at a relatively small unit of
observation, the census block group. Furthermore, we
have identied two additional neighborhood constructs,
territoriality and availability of play resources, which
expand the variety of neighborhood characteristics that
are measured beyond a sole focus on physical and/or
social disorder. Future research will be needed to
determine if these neighborhood characteristics are
important correlates of individual health and well-being.
There are limitations of the proposed assessment
methodology that should be noted. A number of
limitations arise from the use of a three-level
item response model for the analysis. For one, the
requirement that all indicators be dichotomized
may have sacriced important variation in the
indicators. Hopefully, further advances in analytic
methods will permit us to capture more fully the
variation in these measures. Secondly, unlike other
methods of latent variable analysis, a three-level item
response model does not provide an index of model t or
other criteria to assist the investigator in determining
whether the factor structure chosen is the one which best
ts the data.
Fig. 1. Neighborhood observation indicators by average household wealth.
M.O. Caughy et al. / Health & Place 7 (2001) 225236 234
A second limitation of the measure was the lack of
items reecting social disorder such as observations of
illegal activity (i.e., drug dealing, prostitution). This
limitation was partly the result of conducting all
observations in the late morning and early afternoon,
thereby reducing the likelihood of observing illegal
social interactions. Finally, the generalizability of the
measure may have been compromised by its focus on an
older urban center in the northeastern United States.
Because the physical structure of neighborhoods varies
greatly by geographic region, features that were salient
for observation in Baltimore may be irrelevant in other
parts of the United States or in other countries.
Another limitation of the measure is a lack of
dierentiation between those neighborhood character-
istics which result from the behavior of residents from
those which are a function of municipal policies or
actions. In truth, the distinction between the two is often
dicult to make. For example, the presence of trash and
grati has been attributed primarily to the behavior of
residents and interpreted as evidence that residents
dont care about themselves or their community
(Cohen et al., 2000). However, the level of physical
incivilities in a neighborhood is not only a function of
the will of neighborhood residents but also that of
municipal agencies which may not provide the same
level of services through such things as garbage
collection or property owners who live outside the
neighborhood who do not maintain their properties to
an acceptable standard (i.e., slum landlords). In our
own experience doing neighborhood studies, both
processes have been observed operating in select
neighborhoods. Although some characteristics of neigh-
borhoods, such as trac patterns, are more likely to be
under the control of municipal entities while others, such
as resident decoration and plantings, are under the
control of residents themselves, the distinction between
the two is not always clear. More research is needed to
tease apart how the relationship between residents and
broader municipal processes give rise to neighborhood
physical and social processes.
All observational methods of neighborhoods are limited
in their application as cross-sectional snapshots of a
community. Conducting a single observation does not
capture the dynamics of the neighborhood that may
change on a daily basis, based on season of the year, or
over the course of a number of years. Conducting multiple
observations in a strategic manner could help to capture
the dynamic characteristics of community life.
The appropriate application of any neighborhood
observational assessment requires that it be imbedded
within a larger neighborhood assessment protocol.
Interviews with community leaders and/or community
organizations, for example, would provide important
information for the valid interpretation of observational
data. Without a broader understanding of the commu-
nity context, there is a real risk that observational data
will be misinterpreted. Furthermore, an observational
assessment method should be combined with a resident-
report measure such as that reported by Coulton et al.
(1996) to yield the most comprehensive assessment of
the neighborhood environment. There are some aspects
of neighborhoods such as social interaction patterns and
collective ecacy which are most eectively tapped by
resident report although some proxies for them, such as
our measure of territoriality, may be evident through
observation.
The increasing number of studies in the literature
which examine the inuence of neighborhood context on
individual health outcomes represents an exciting devel-
opment which promises to improve our understanding
of the complex interplay between individuals and their
communities and its contribution to the production of
health. This area of inquiry is still in its infancy, and
researchers are struggling with how best to conceptualize
neighborhoods and to measure their most important
features. A careful consideration of theory when
developing measures and interpreting ndings will
provide a critical foundation as the eld progresses. The
results of such research would help us to design eective
community revitalization eorts and other policies to
improve the lives of families and their children.
Acknowledgements
This research was supported by the Maternal and
Child Health Bureau, Grant MCJ-240731-01, and by the
Smith-Richardson Foundation, Grant 9701-936. The
authors would like to thank Sherry Bazinet, Deborah
Brothers, Bennette Drummond-Fitzgerald, Crystal
Evans, Danielle Fagan, Yvette Flowers, Christopher
Jeter, Terese Lowery, Samara Mays, Elizabeth Mon-
tgomery, Karen Ricks, John Rogers, and Katy Schalla
for their assistance in conducting the neighborhood
observations. The authors would also like to thank Yi-
Hua Chen, Patricia Chore, and Crystal Evans for their
assistance with data management. Finally, the authors
are indebted to Drs. Ron Harrist and Louise Masse for
their patience in providing statistical advice.
References
Brodsky, A.E., 1996. Resilient single mothers in risky
neighborhoods: negative psychological sense of community.
Journal of Community Psychology 24, 347363.
Buckner, J.C., 1988. The development of an instrument to
measure neighborhood cohesion. American Journal of
Community Psychology 16, 771791.
M.O. Caughy et al. / Health & Place 7 (2001) 225236 235
Caughy, M.O., OCampo, P.J., Brodsky, A.E., 1999. Neighbor-
hoods, families, and children: implications for policy and
practice. Journal of Community Psychology 27, 615633.
Cohen, D., Spear, S., Scribner, R., Kissinger, P., Mason, K.,
Wildgen, J., 2000. Broken windows and the risk of
gonorrhea. American Journal of Public Health 90, 230236.
Coulton, C.J., Korbin, J.E., Su, M., 1996. Measuring neighbor-
hood context for young children in an urban area. American
Journal of Community Psychology 24, 532.
Duncan, G.J., Raudenbush, S.W., 1999. Assessing the eects of
context in studies of child and youth development. Educa-
tional Psychologist 34 (1), 2941.
Furstenberg, F.F., 1993. How families manage risk and
opportunity in dangerous neighborhoods. In: Wilson, W.J.
(Ed.), Sociology and the Public Agenda. Sage Publications,
Newbury Park, CA, pp. 231258.
Furstenberg, F.F., Hughes, M.E., 1994. The inuence of
neighborhoods on childrens development: a theoretical
perspective and research agenda. Paper presented at the
University of Wisconsin Institute for Research on Poverty
Conference on Indicators of Childrens Well-Being,
Bethesda, MD, November 1718.
Korbin, J.E., Coulton, C.J., 1997. Understanding the neighbor-
hood context for children and families: combining epidemio-
logical and ethnographic approaches. In: Brooks-Gunn, J.,
Duncan, G.J., Aber, J.L. (Eds.), Neighborhood Poverty:
Policy Implications in Studying Neighborhoods. Russell Sage
Foundation, New York.
McGuire, J.B., 1997. The reliability and validity of a
questionnaire describing neighborhood characteristics rele-
vant to families and young children living in urban areas.
Journal of Community Psychology 25, 551566.
NORC (National Opinion Research Center), 1995. Project for
Human Development in Chicago Neighborhoods; Harvard
Project 4709. SSO (Systematic Social Observation) Coding
Manual, June 1995.
OCampo, P., Wang, M., Xue, X., Caughy, M.O., 1997a.
Neighbourhood risk factors for low birthweight in Baltimore
City: A multi-level analysis. American Journal of Public
Health 87, 11131118.
OCampo, P., Caughy, M.O., Aronson, R., Xue, X., Wang, M.,
1997b. A comparison of two analytic methods for the
identication of neighborhoods as intervention sites for
community-based programs. Evaluation and Program Plan-
ning 20, 405414.
Perkins, D., Meeks, J., Taylor, R.B., 1992. The physical
environment of street blocks and resident perceptions of
crime and disorder: implications for theory and measure-
ment. Journal of Environmental Psychology 12, 2134.
Rabash, J., Browne, W., Healy, M., Cameron, B., Charlton, C.,
1999. MlwiN BETA version 1.10.0001. Multilevel Models
Project. Insitute of Education, University of London.
Raudenbush, S.W., Sampson, R.J., 1999. Ecometrics: toward a
science of assessment ecological settings, with application to
the systematic social observation of neighborhoods. Socio-
logical Methodology 29, 141.
Taylor, R.B., Gottfredson, S.D., Brower, S., 1984. Block crime
and fear: defensible space, local social ties, and territorial
functioning. Journal of Research in Crime and Delinquency
21, 303331.
Taylor, R.B., Shumaker, S.A., Gottfredson, S.D., 1985.
Neighborhood-level links between physical features and local
sentiments: deterioration, fear or crime,and condence.
Journal of Architectural Planning and Research 2, 261275.
M.O. Caughy et al. / Health & Place 7 (2001) 225236 236